A Free and Open Source LLM Fine-tuning Framework

• 2025/07:
  • ND Parallelism support has been added into Gym. Compose Context Parallelism (CP), Tensor Parallelism (TP), and Fully Sharded Data Parallelism (FSDP) within a single node and across multiple nodes. Check out the blog post for more info.
  • Gym adds more models: GPT-OSS, Gemma 3n, Liquid Foundation Model 2 (LFM2), and Arcee Foundation Models (AFM).
  • FP8 finetuning with fp8 gather op is now possible in Gym via torchao. Get started here!
  • Voxtral, Magistral 1.1, and Devstral with mistral-common tokenizer support has been integrated in Gym!
  • TiledMLP support for single-GPU to multi-GPU training with DDP, DeepSpeed and FSDP support has been added to support Arctic Long Sequence Training. (ALST). See examples for using ALST with Gym!
• 2025/05: Quantization Aware Training (QAT) support has been added to Gym. Explore the docs to learn more!
• 2025/03: Gym has implemented Sequence Parallelism (SP) support. Read the blog and docs to learn how to scale your context length when fine-tuning.

Gym is a free and open-source tool designed to streamline post-training and fine-tuning for the latest large language models (LLMs).

Features:

• Multiple Model Support: Train various models like GPT-OSS, LLaMA, Mistral, Mixtral, Pythia, and many more models available on the Hugging Face Hub.
• Multimodal Training: Fine-tune vision-language models (VLMs) including LLaMA-Vision, Qwen2-VL, Pixtral, LLaVA, SmolVLM2, and audio models like Voxtral with image, video, and audio support.
• Training Methods: Full fine-tuning, LoRA, QLoRA, GPTQ, QAT, Preference Tuning (DPO, IPO, KTO, ORPO), RL (GRPO), and Reward Modelling (RM) / Process Reward Modelling (PRM).
• Easy Configuration: Re-use a single YAML configuration file across the full fine-tuning pipeline: dataset preprocessing, training, evaluation, quantization, and inference.
• Performance Optimizations: Multipacking, Flash Attention, Xformers, Flex Attention, Liger Kernel, Cut Cross Entropy, Sequence Parallelism (SP), LoRA optimizations, Multi-GPU training (FSDP1, FSDP2, DeepSpeed), Multi-node training (Torchrun, Ray), and many more!
• Flexible Dataset Handling: Load from local, HuggingFace, and cloud (S3, Azure, GCP, OCI) datasets.
• Cloud Ready: We ship Docker images and also PyPI packages for use on cloud platforms and local hardware.

Requirements:

• NVIDIA GPU (Ampere or newer for bf16 and Flash Attention) or AMD GPU
• Python 3.11
• PyTorch ≥2.6.0

pip3 install -U packaging==23.2 setuptools==75.8.0 wheel ninja
pip3 install --no-build-isolation zoo-gym[flash-attn,deepspeed]

# Download example gym configs, deepspeed configs
gym fetch examples
gym fetch deepspeed_configs  # OPTIONAL

Installing with Docker can be less error prone than installing in your own environment.

docker run --gpus '"all"' --rm -it zoolabs/gym:main-latest

Other installation approaches are described here.

# Fetch gym examples
gym fetch examples

# Or, specify a custom path
gym fetch examples --dest path/to/folder

# Train a model using LoRA
gym train examples/llama-3/lora-1b.yml

That's it! Check out our Getting Started Guide for a more detailed walkthrough.

• Installation Options - Detailed setup instructions for different environments
• Configuration Guide - Full configuration options and examples
• Dataset Loading - Loading datasets from various sources
• Dataset Guide - Supported formats and how to use them
• Multi-GPU Training
• Multi-Node Training
• Multipacking
• API Reference - Auto-generated code documentation
• FAQ - Frequently asked questions

• Join our Discord community for support
• Check out our Examples directory
• Read our Debugging Guide
• Need dedicated support? Please contact ✉️wing@zoo.dev for options

Contributions are welcome! Please see our Contributing Guide for details.

Interested in sponsoring? Contact us at wing@zoo.dev

If you use Gym in your research or projects, please cite it as follows:

@software{gym,
  title = {Gym: Open Source LLM Post-Training},
  author = {{Zoo Labs Foundation Inc. and contributors}},
  url = {https://github.com/zoo-labs/gym},
  license = {Apache-2.0},
  year = {2023}
}

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.