> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/huggingface/trl/llms.txt
> Use this file to discover all available pages before exploring further.

# Installation

> Install TRL from PyPI or from source. Requires Python 3.10 or later.

## Requirements

* Python 3.10 or later
* PyTorch (installed automatically as a transitive dependency via Transformers)

## Install from PyPI

Install TRL using `pip` or `uv`:

<Tabs>
  <Tab title="pip">
    ```bash theme={null}
    pip install trl
    ```
  </Tab>

  <Tab title="uv">
    [uv](https://docs.astral.sh/uv/) is a fast Rust-based Python package manager. See the [uv installation guide](https://docs.astral.sh/uv/getting-started/installation/) to get started.

    ```bash theme={null}
    uv pip install trl
    ```
  </Tab>
</Tabs>

This installs TRL along with its core dependencies: `transformers`, `accelerate`, and `datasets`.

## Install from source

To use the latest unreleased features, install directly from the GitHub repository:

```bash theme={null}
pip install git+https://github.com/huggingface/trl.git
```

Alternatively, clone the repository and install in editable mode:

```bash theme={null}
git clone https://github.com/huggingface/trl.git
cd trl/
pip install -e .
```

## Optional dependencies

TRL provides optional dependency groups for specific use cases. Install them using the `extras` syntax:

```bash theme={null}
pip install "trl[<extra>]"
```

| Extra          | Packages installed                                              | Use case                                                          |
| -------------- | --------------------------------------------------------------- | ----------------------------------------------------------------- |
| `peft`         | `peft>=0.8.0`                                                   | LoRA and QLoRA training via PEFT                                  |
| `vllm`         | `vllm`, `fastapi`, `pydantic`, `aiohttp`, `uvicorn`, and others | Fast online generation for RL-based trainers                      |
| `deepspeed`    | `deepspeed>=0.14.4`                                             | DeepSpeed ZeRO distributed training                               |
| `liger`        | `liger-kernel>=0.7.0`                                           | Liger fused kernels for faster and more memory-efficient training |
| `quantization` | `bitsandbytes`                                                  | 4-bit and 8-bit quantization                                      |
| `judges`       | `openai>=1.23.2`, `llm-blender>=0.0.2`                          | LLM-based reward judges                                           |
| `vlm`          | `Pillow`, `torchvision`, `num2words`                            | Vision-language model training                                    |
| `math_verify`  | `math-verify>=0.5.2`                                            | Mathematical answer verification for reward functions             |

For example, to install TRL with PEFT and quantization support:

```bash theme={null}
pip install "trl[peft,quantization]"
```

<Tip>
  For most use cases, start with `pip install trl` and add optional dependencies as needed. The `peft` and `quantization` extras are recommended for training on consumer hardware.
</Tip>

## Developer install

If you want to contribute to TRL or run the test suite, install the development dependencies:

```bash theme={null}
git clone https://github.com/huggingface/trl.git
cd trl/
pip install -e ".[dev]"
```

The `dev` extra includes testing tools (`pytest`, `pytest-cov`, `pytest-xdist`), code quality tools (`pre-commit`, `hf-doc-builder`), and most optional dependency groups.

<Note>
  vLLM is not included in the `dev` extra by default due to CUDA compatibility constraints. Install it separately with `pip install "trl[vllm]"` if needed.
</Note>

See the [contributing guide](https://github.com/huggingface/trl/blob/main/CONTRIBUTING.md) for more details on setting up a development environment.