> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/huggingface/trl/llms.txt
> Use this file to discover all available pages before exploring further.

# SFT Trainer

> Supervised Fine-Tuning (SFT) trainer for adapting language models to follow instructions and complete text tasks.

## Overview

Supervised Fine-Tuning (SFT) is the simplest and most commonly used method to adapt a language model to a target dataset. The model is trained in a fully supervised fashion using pairs of input and output sequences. The goal is to minimize the negative log-likelihood (NLL) of the target sequence, conditioning on the input.

`SFTTrainer` supports both language modeling and prompt-completion datasets, and works with standard or conversational dataset formats. When provided with a conversational dataset, the trainer automatically applies the model's chat template.

## Quick start

```python theme={null}
from trl import SFTTrainer
from datasets import load_dataset

trainer = SFTTrainer(
    model="Qwen/Qwen3-0.6B",
    train_dataset=load_dataset("trl-lib/Capybara", split="train"),
)
trainer.train()
```

To launch distributed training:

```bash theme={null}
accelerate launch train_sft.py
```

## Dataset format

`SFTTrainer` accepts four dataset formats:

```python theme={null}
# Standard language modeling
{"text": "The sky is blue."}

# Conversational language modeling
{"messages": [{"role": "user", "content": "What color is the sky?"},
              {"role": "assistant", "content": "It is blue."}]}

# Standard prompt-completion
{"prompt": "The sky is",
 "completion": " blue."}

# Conversational prompt-completion
{"prompt": [{"role": "user", "content": "What color is the sky?"}],
 "completion": [{"role": "assistant", "content": "It is blue."}]}
```

<Note>
  For prompt-completion datasets, loss is computed only on the completion tokens by default. For language modeling datasets, loss is computed on the full sequence.
</Note>

If your dataset uses different column names, preprocess it to match the expected format:

```python theme={null}
from datasets import load_dataset

dataset = load_dataset("FreedomIntelligence/medical-o1-reasoning-SFT", "en")

def preprocess_function(example):
    return {
        "prompt": [{"role": "user", "content": example["Question"]}],
        "completion": [
            {"role": "assistant", "content": f"<think>{example['Complex_CoT']}</think>{example['Response']}"}
        ],
    }

dataset = dataset.map(preprocess_function, remove_columns=["Question", "Response", "Complex_CoT"])
```

## Key configuration parameters

<AccordionGroup>
  <Accordion title="Data preprocessing">
    <ParamField path="max_length" type="int | None" default="1024">
      Maximum length of the tokenized sequence. Sequences longer than this are truncated. Set to `None` to disable truncation (recommended for VLMs).
    </ParamField>

    <ParamField path="packing" type="bool" default="false">
      Whether to pack multiple short sequences into fixed-length blocks, improving GPU utilization and reducing padding waste. Uses `max_length` to define block size.
    </ParamField>

    <ParamField path="packing_strategy" type="str" default="bfd">
      Strategy for packing sequences: `"bfd"` (best-fit decreasing, truncates overflow), `"bfd_split"` (best-fit decreasing, splits overflow sequences), or `"wrapped"` (aggressive, cuts mid-sequence).
    </ParamField>

    <ParamField path="dataset_text_field" type="str" default="text">
      Name of the column containing text data for language modeling datasets.
    </ParamField>

    <ParamField path="truncation_mode" type="str" default="keep_start">
      Which end to truncate when a sequence exceeds `max_length`. Options: `"keep_start"` or `"keep_end"`.
    </ParamField>
  </Accordion>

  <Accordion title="Loss computation">
    <ParamField path="completion_only_loss" type="bool | None" default="None">
      Whether to compute loss only on the completion part. When `None`, defaults to `True` for prompt-completion datasets and `False` for language modeling datasets.
    </ParamField>

    <ParamField path="assistant_only_loss" type="bool" default="false">
      Whether to compute loss only on assistant responses in conversational datasets. Requires a chat template that supports the `{% generation %}` and `{% endgeneration %}` keywords.
    </ParamField>

    <ParamField path="loss_type" type="str" default="nll">
      Type of loss to use. Options: `"nll"` (standard negative log-likelihood) or `"dft"` (Dynamic Fine-Tuning, which rectifies the reward signal to improve generalization).
    </ParamField>
  </Accordion>

  <Accordion title="Model initialization">
    <ParamField path="model_init_kwargs" type="dict | None">
      Keyword arguments forwarded to `AutoModelForCausalLM.from_pretrained` when the `model` argument is a string. Useful for setting `dtype`, `device_map`, or `output_router_logits` for MoE models.
    </ParamField>

    <ParamField path="chat_template_path" type="str | None">
      Path to a tokenizer or a Jinja template file to set as the model's chat template. Useful when fine-tuning base models that do not have a chat template.
    </ParamField>

    <ParamField path="eos_token" type="str | None">
      Token used to indicate end of sequence. Required when the chat template uses a different EOS token than the tokenizer's default.
    </ParamField>
  </Accordion>

  <Accordion title="Memory optimization">
    <ParamField path="padding_free" type="bool" default="false">
      Perform forward passes without padding by flattening all sequences into a single continuous sequence. Requires FlashAttention 2 or 3. Automatically enabled when `packing="bfd"`.
    </ParamField>

    <ParamField path="activation_offloading" type="bool" default="false">
      Offload activations to CPU to reduce GPU memory usage.
    </ParamField>
  </Accordion>
</AccordionGroup>

<Note>
  `SFTConfig` also overrides some `TrainingArguments` defaults: `logging_steps=10`, `gradient_checkpointing=True`, `bf16=True`, and `learning_rate=2e-5`.
</Note>

## Instruction tuning

To turn a base model into an instruction-following model, provide a chat template and a conversational dataset:

```python theme={null}
from trl import SFTConfig, SFTTrainer
from datasets import load_dataset

trainer = SFTTrainer(
    model="Qwen/Qwen3-0.6B-Base",
    args=SFTConfig(
        output_dir="Qwen3-0.6B-Instruct",
        chat_template_path="HuggingFaceTB/SmolLM3-3B",
    ),
    train_dataset=load_dataset("trl-lib/Capybara", split="train"),
)
trainer.train()
```

<Warning>
  Some base models (such as Qwen models) already have a chat template in the tokenizer. In that case, you do not need to set `chat_template_path`, but you should align the EOS token. For example, for `Qwen/Qwen2.5-1.5B`, set `eos_token="<|im_end|>"` in `SFTConfig`.
</Warning>

## Dataset packing

Packing is a technique to increase training efficiency by grouping multiple short examples into a single fixed-length block, reducing wasted padding tokens.

```python theme={null}
from trl import SFTConfig, SFTTrainer
from datasets import load_dataset

trainer = SFTTrainer(
    model="Qwen/Qwen3-0.6B",
    args=SFTConfig(
        packing=True,
        max_length=2048,
    ),
    train_dataset=load_dataset("trl-lib/Capybara", split="train"),
)
trainer.train()
```

<Tip>
  For best performance with packing, use FlashAttention 2 or 3. The `padding_free` option is automatically enabled with `packing_strategy="bfd"`, eliminating padding overhead entirely.
</Tip>

## Training with PEFT/LoRA

Use the PEFT library to train only a small set of adapter parameters instead of the full model:

```python theme={null}
from datasets import load_dataset
from trl import SFTTrainer
from peft import LoraConfig

dataset = load_dataset("trl-lib/Capybara", split="train")

trainer = SFTTrainer(
    "Qwen/Qwen3-0.6B",
    train_dataset=dataset,
    peft_config=LoraConfig(),
)
trainer.train()
```

To continue training an existing PEFT model:

```python theme={null}
from datasets import load_dataset
from trl import SFTTrainer
from peft import AutoPeftModelForCausalLM

model = AutoPeftModelForCausalLM.from_pretrained("trl-lib/Qwen3-4B-LoRA", is_trainable=True)
dataset = load_dataset("trl-lib/Capybara", split="train")

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
)
trainer.train()
```

<Tip>
  When training adapters, use a higher learning rate (around `1e-4`) since only the new adapter parameters are being learned.
</Tip>

## Training Vision-Language Models

`SFTTrainer` supports VLMs. Provide a dataset with an `image` column (single image) or `images` column (list of images):

```python theme={null}
from trl import SFTConfig, SFTTrainer
from datasets import load_dataset

trainer = SFTTrainer(
    model="Qwen/Qwen2.5-VL-3B-Instruct",
    args=SFTConfig(max_length=None),
    train_dataset=load_dataset("trl-lib/llava-instruct-mix", split="train"),
)
trainer.train()
```

<Warning>
  For VLMs, set `max_length=None` to prevent truncation from removing image tokens, which causes errors during training.
</Warning>

## Logged metrics

| Metric                | Description                                                      |
| --------------------- | ---------------------------------------------------------------- |
| `loss`                | Average cross-entropy loss over non-masked tokens                |
| `entropy`             | Average entropy of the model's predicted token distribution      |
| `mean_token_accuracy` | Proportion of tokens where top-1 prediction matches ground truth |
| `learning_rate`       | Current learning rate                                            |
| `grad_norm`           | L2 norm of gradients before clipping                             |
| `num_tokens`          | Total tokens processed so far                                    |