> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/huggingface/trl/llms.txt
> Use this file to discover all available pages before exploring further.

# Use a model after training

> Load a fine-tuned TRL model for inference, merge LoRA adapters, push to the Hugging Face Hub, and run text generation.

Once you have trained a model with SFTTrainer, DPOTrainer, GRPOTrainer, or any other TRL trainer, you can load it and run inference like any other Transformers model.

## Load and generate

If you fine-tuned the model fully (without PEFT/LoRA), load it directly with the standard `AutoModelForCausalLM` class. Any trainer-specific components such as the value head from PPO training are automatically ignored:

```python theme={null}
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name_or_path = "Qwen/Qwen3-0.6B"  # or path/to/your/model
device = "cuda"  # or "cpu"

model = AutoModelForCausalLM.from_pretrained(model_name_or_path).to(device)
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)

inputs = tokenizer.encode("This movie was really", return_tensors="pt").to(device)
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))
```

Alternatively, use the `pipeline` API:

```python theme={null}
from transformers import pipeline

model_name_or_path = "Qwen/Qwen3-0.6B"  # or path/to/your/model
pipe = pipeline("text-generation", model=model_name_or_path)
print(pipe("This movie was really")[0]["generated_text"])
```

## Load and use PEFT adapters

If you trained with LoRA or another PEFT method, load the base model and then apply the adapter on top:

```python theme={null}
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_name = "Qwen/Qwen3-0.6B"   # base model used during training
adapter_model_name = "path/to/my/adapter"

model = AutoModelForCausalLM.from_pretrained(base_model_name)
model = PeftModel.from_pretrained(model, adapter_model_name)

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
```

With the adapter loaded, run generation as with a standard model.

## Merge LoRA adapters into the base model

Merging adapters into the base model weights produces a single self-contained checkpoint that behaves exactly like a standard Transformers model — no PEFT dependency required at inference time.

<Warning>
  Merged checkpoints are significantly larger than adapter-only checkpoints because they include all base model weights.
</Warning>

```python theme={null}
from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model_name = "Qwen/Qwen3-0.6B"
adapter_model_name = "path/to/my/adapter"

model = AutoModelForCausalLM.from_pretrained(base_model_name)
model = PeftModel.from_pretrained(model, adapter_model_name)

# Merge adapter weights into the base model
model = model.merge_and_unload()
model.save_pretrained("merged_model")
```

After merging and saving, load the merged model as any other standard model:

```python theme={null}
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("merged_model")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
```

## Push to the Hugging Face Hub

TRL trainers support pushing the trained model directly to the Hub at the end of training. Set `push_to_hub=True` in your training config:

```python theme={null}
from trl import SFTConfig

training_args = SFTConfig(
    ...,
    output_dir="my-model",
    push_to_hub=True,
)
```

Or push manually after training:

```python theme={null}
trainer.push_to_hub()
```

You can also use the standard Transformers API to push a loaded model:

```python theme={null}
model.push_to_hub("my-username/my-model")
tokenizer.push_to_hub("my-username/my-model")
```

## Run an inference server

For production inference, consider running a dedicated inference server. The [text-generation-inference](https://github.com/huggingface/text-generation-inference) library provides optimized serving for Transformers models, including models trained with TRL.