Installation

Install TRL from PyPI or from source. Requires Python 3.10 or later.

Requirements

• Python 3.10 or later
• PyTorch (installed automatically as a transitive dependency via Transformers)

Install from PyPI

Install TRL using pip or uv:

pip

pip install trl

uv

uv is a fast Rust-based Python package manager. See the uv installation guide to get started.

uv pip install trl

This installs TRL along with its core dependencies: transformers, accelerate, and datasets.

Install from source

To use the latest unreleased features, install directly from the GitHub repository:

pip install git+https://github.com/huggingface/trl.git

Alternatively, clone the repository and install in editable mode:

git clone https://github.com/huggingface/trl.git
cd trl/
pip install -e .

Optional dependencies

TRL provides optional dependency groups for specific use cases. Install them using the extras syntax:

pip install "trl[<extra>]"

Extra	Packages installed	Use case
peft	peft>=0.8.0	LoRA and QLoRA training via PEFT
vllm	vllm, fastapi, pydantic, aiohttp, uvicorn, and others	Fast online generation for RL-based trainers
deepspeed	deepspeed>=0.14.4	DeepSpeed ZeRO distributed training
liger	liger-kernel>=0.7.0	Liger fused kernels for faster and more memory-efficient training
quantization	bitsandbytes	4-bit and 8-bit quantization
judges	openai>=1.23.2, llm-blender>=0.0.2	LLM-based reward judges
vlm	Pillow, torchvision, num2words	Vision-language model training
math_verify	math-verify>=0.5.2	Mathematical answer verification for reward functions

For example, to install TRL with PEFT and quantization support:

pip install "trl[peft,quantization]"

For most use cases, start with pip install trl and add optional dependencies as needed. The peft and quantization extras are recommended for training on consumer hardware.

Developer install

If you want to contribute to TRL or run the test suite, install the development dependencies:

git clone https://github.com/huggingface/trl.git
cd trl/
pip install -e ".[dev]"

The dev extra includes testing tools (pytest, pytest-cov, pytest-xdist), code quality tools (pre-commit, hf-doc-builder), and most optional dependency groups.

vLLM is not included in the dev extra by default due to CUDA compatibility constraints. Install it separately with pip install "trl[vllm]" if needed.

See the contributing guide for more details on setting up a development environment.