> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/huggingface/trl/llms.txt
> Use this file to discover all available pages before exploring further.

# Speeding up training

> Accelerate TRL training with vLLM generation, optimized attention kernels, Liger Kernel, and mixed precision.

This guide covers methods to accelerate training in TRL. Each technique includes minimal examples with links to more comprehensive documentation.

## vLLM for fast generation in online methods

[Online methods](/index#online-methods) such as GRPO or Online DPO require the model to generate completions, which is often the slowest step. [vLLM](https://github.com/vllm-project/vllm) speeds up generation significantly through PagedAttention and other optimizations.

Install vLLM:

```bash theme={null}
pip install trl[vllm]
```

<Steps>
  <Step title="Start a vLLM server">
    ```bash theme={null}
    trl vllm-serve --model <model_name>
    ```
  </Step>

  <Step title="Enable vLLM in your training config">
    <Tabs>
      <Tab title="GRPO">
        ```python theme={null}
        from trl import GRPOConfig

        training_args = GRPOConfig(..., use_vllm=True, vllm_mode="server")
        ```
      </Tab>

      <Tab title="Online DPO">
        ```python theme={null}
        from trl.experimental.online_dpo import OnlineDPOConfig

        training_args = OnlineDPOConfig(..., use_vllm=True, vllm_mode="server")
        ```
      </Tab>

      <Tab title="RLOO">
        ```python theme={null}
        from trl import RLOOConfig

        training_args = RLOOConfig(..., use_vllm=True, vllm_mode="server")
        ```
      </Tab>
    </Tabs>
  </Step>
</Steps>

<Warning>
  Ensure that GPUs assigned for training and generation are separate to avoid resource conflicts. For example, with 8 GPUs total:

  ```bash theme={null}
  # GPUs 0–3 for vLLM generation
  CUDA_VISIBLE_DEVICES=0,1,2,3 trl vllm-serve --model <model_name>

  # GPUs 4–7 for training
  CUDA_VISIBLE_DEVICES=4,5,6,7 accelerate launch train.py
  ```
</Warning>

For full configuration options, see the [vLLM Integration](/vllm-integration) guide.

## Optimized attention implementations

TRL supports optimized attention backends that speed up training while reducing memory usage.

<Tabs>
  <Tab title="Kernels from Hub (recommended)">
    Use pre-optimized attention kernels from the Hub without manual compilation:

    ```python theme={null}
    from trl import SFTConfig

    training_args = SFTConfig(
        ...,
        model_init_kwargs={"attn_implementation": "kernels-community/flash-attn2"},
    )
    ```

    Other available kernels include `kernels-community/vllm-flash-attn3` and `kernels-community/paged-attention`.

    For more details, see the [Kernels Hub Integration](https://github.com/huggingface/trl/blob/main/docs/source/kernels_hub.md) guide.
  </Tab>

  <Tab title="Manual build">
    <Warning>
      Manually building optimized attention backends is complex and time-consuming. Use Kernels from the Hub instead unless absolutely necessary.
    </Warning>

    If you have manually installed an optimized attention backend such as Flash Attention 2:

    ```python theme={null}
    from trl import SFTConfig

    training_args = SFTConfig(
        ...,
        model_init_kwargs={"attn_implementation": "flash_attention_2"},
    )
    ```
  </Tab>
</Tabs>

Optimized attention works across all TRL trainers.

## Liger Kernel

[Liger Kernel](https://github.com/linkedin/Liger-Kernel) is a collection of Triton kernels designed for LLM training. It can increase multi-GPU throughput by 20% and reduce memory usage by 60%.

<Tabs>
  <Tab title="SFT">
    ```python theme={null}
    from trl import SFTConfig

    training_args = SFTConfig(..., use_liger_kernel=True)
    ```
  </Tab>

  <Tab title="DPO">
    ```python theme={null}
    from trl import DPOConfig

    training_args = DPOConfig(..., use_liger_kernel=True)
    ```
  </Tab>

  <Tab title="GRPO">
    ```python theme={null}
    from trl import GRPOConfig

    training_args = GRPOConfig(..., use_liger_kernel=True)
    ```
  </Tab>

  <Tab title="KTO">
    ```python theme={null}
    from trl.experimental.kto import KTOConfig

    training_args = KTOConfig(..., use_liger_kernel=True)
    ```
  </Tab>

  <Tab title="GKD">
    ```python theme={null}
    from trl.experimental.gkd import GKDConfig

    training_args = GKDConfig(..., use_liger_kernel=True)
    ```
  </Tab>
</Tabs>

For more details, see the [Liger Kernel Integration](/liger-kernel-integration) guide.

## Mixed precision training

Mixed precision training using bf16 or fp16 can speed up training and reduce memory usage with minimal impact on model quality.

```python theme={null}
from trl import SFTConfig

# bfloat16 — recommended for Ampere (A100, RTX 30xx) or newer
training_args = SFTConfig(..., bf16=True)

# float16 — use for older GPUs
training_args = SFTConfig(..., fp16=True)
```

Mixed precision is supported across all TRL trainers.