feat: add Phi-3.5 training chat templates with generation markers#5746
Merged
qgallouedec merged 6 commits intoMay 11, 2026
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 1f26dea. Configure here.
…ns for assistant-only loss masking
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
qgallouedec
approved these changes
May 11, 2026
24 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
contributing to #5471
Adds a training-compatible variant of the Phi-3.5 chat template with
{% generation %} / {% endgeneration %}markers, enablingreturn_assistant_tokens_mask=Truefor assistant-only loss masking in SFT training.-
phi3_5.jinja: template frommicrosoft/Phi-3.5-mini-instruct-
phi3_5_training.jinja: modified the phi3_5.jinja to turn the training chat template-
get_training_chat_template: registered Phi-3.5-
tests/test_chat_template_utils.py: added param"trl-internal-testing/tiny-Phi3ForCausalLM-3.5", id="phi3.5"Before submitting
{% generation %}chat templates for common model families #5471AI writing disclosure
We welcome the use of AI tools to help with contributions. For transparency and to help us improve our review process, please indicate the level of AI involvement in this PR.
Who can review?
@qgallouedec (opened #5471)
Note
Low Risk
Low risk: additive support for a new model family template plus a small conditional in
get_training_chat_template, with coverage via existing template utility tests.Overview
Adds Phi-3.5 to TRL’s recognized chat-template families and introduces
phi3_5.jinjaplus a training variantphi3_5_training.jinjathat wraps assistant output in{% generation %}/{% endgeneration %}for assistant-only loss masking.Registers the new Phi-3.5 templates in
get_training_chat_template, updates docs/README to list the new family and training template, and extendstest_chat_template_utils.pyto exercise the Phi-3.5 tiny model in the existing training-template test matrix.Reviewed by Cursor Bugbot for commit 22c1909. Bugbot is set up for automated code reviews on this repo. Configure here.