Context

SFT with assistant_only_loss=True requires the chat template to include {% generation %} / {% endgeneration %} markers so that return_assistant_tokens_mask=True can produce correct masks. Very few models ship these markers natively.

TRL should provide training chat templates with these markers for known model families (via get_training_chat_template()), and the SFT trainer auto-applies them when assistant_only_loss=True.

This issue tracks adding training templates for all model families with chat template support.

What needs to happen for each model

1. Create <model>_training.jinja: the original template with {% generation %} / {% endgeneration %} added around assistant output (and prefix-preservation fixes if needed)
2. Load it in chat_template_utils.py and add a branch in get_training_chat_template()
3. Add a test verifying text output is identical to the original and masks are correct

Model families

Causal LMs with chat template

• Qwen2.5 feat: add Qwen2.5 training chat template with generation markers #5522
• Qwen3 Add {% generation %} support to training chat templates #5470
• Qwen3-2507 (Instruct) Add training chat template for Qwen3-2507 #5574
• Qwen3.5 Add Qwen3.5 Think/NoThink training chat templates with generation markers #5824
• Qwen3.6 Qwen3.6 integration #5642
• Cohere Add Cohere training chat template #5627
• Cohere2 Add {% generation %} markers for Cohere2 chat template #5675
• DeepSeek-V3 feat: add DeepSeek-V3 training chat template with generation markers #5527
• Gemma feat: add Gemma/Gemma2 training chat templates with generation markers #5523
• Gemma2 feat: add Gemma/Gemma2 training chat templates with generation markers #5523
• GLM-4-MoE feat(glm-4-moe): Add {% generation %} markers for training chat template #5519
• GPT-OSS feat(gpt-oss): Add {% generation %} markers for training chat template #5484
• Llama 3 feat: add Llama 3 training chat template with generation markers #5493
• Phi-3 feat: add Phi-3 training chat template with generation markers #5526
• Phi-3.5 feat: add Phi-3.5 training chat templates with generation markers #5746
• FalconMamba feat: add Falcon Mamba training chat templates with generation markers #5723

VLMs

• Gemma3 Add Gemma 3 training chat template #5685
• Qwen3-VL Add Qwen3-VL training chat template with generation markers #5764
• Qwen2-VL
• Qwen2.5-VL
• SmolVLM
• Idefics3
• LLaVA
• LLaVA-Next

No chat template (no action needed)

• Bloom, GPT2, GPTNeoX, OPT, T5

Notes

• VLMs currently don't support assistant_only_loss in SFT (blocked by a separate check). These should still be tracked so templates are ready when support lands.