Skip to content

feat(glm-4-moe): Add {% generation %} markers for training chat template#5519

Merged
qgallouedec merged 7 commits into
huggingface:mainfrom
casinca:glm4-generation-training-template
Apr 22, 2026
Merged

feat(glm-4-moe): Add {% generation %} markers for training chat template#5519
qgallouedec merged 7 commits into
huggingface:mainfrom
casinca:glm4-generation-training-template

Conversation

@casinca

@casinca casinca commented Apr 10, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

This PR add {% generation %} tags/markers for glm-4-moe: part of #5471

  • nit: sorted models list to lexicographical order

  • added think patch like in qwen3, same pattern, on top of {% generation %}

  • TestGetTrainingChatTemplate suite fully passes with transformers > 5

(not sure) I didn't add a skipif like in : added per codex review

pytest.param(
"trl-internal-testing/tiny-Glm4MoeForCausalLM",
id="glm4moe",
marks=pytest.mark.skipif(
Version(transformers.__version__) < Version("5.0.0"),
reason="GLM4 tokenizer requires transformers>=5.0.0",
),
),

 

Llama PR reverted

diff --git a/trl/chat_template_utils.py b/trl/chat_template_utils.py
--- a/trl/chat_template_utils.py
+++ b/trl/chat_template_utils.py
@@ -375,9 +375,7 @@ def get_training_chat_template(tokenizer: PreTrainedTokenizer) -> str | None:
 
     Returns a patched chat template that is prefix-preserving and includes `{%% generation %%}` / `{%% endgeneration
     %%}` markers for assistant-only loss masking. Returns `None` if the tokenizer's template already satisfies both
-    requirements. Currently supported:
-        - Qwen3
-        - GPT-OSS
+    requirements. Currently GPT-OSS, LLaMA 3 and Qwen3 are supported.
 
     Args:
         tokenizer (`PreTrainedTokenizer`):

This is going to be a long sentence with other models 😅

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline, Pull Request section?
  • Was this discussed/approved via a GitHub issue? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

AI writing disclosure

We welcome the use of AI tools to help with contributions. For transparency and to help us improve our review process, please indicate the level of AI involvement in this PR.

  • No AI usage: the PR was written entirely by a human.
  • AI-assisted: some parts were suggested or improved by AI, but the PR was written and reviewed by a human.
  • AI-generated: the PR was mostly or fully generated by an AI tool.

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

following tool call #5463 @qgallouedec


Note

Medium Risk
Changes prompt rendering for GLM-4-MoE during training, which can affect masking and prefix-preservation behavior; however the change is isolated to template selection/formatting and guarded by exact-template identity checks.

Overview
Adds GLM-4-MoE support to get_training_chat_template by introducing a new glm4moe_training.jinja template and returning it when the tokenizer matches the GLM-4-MoE base template.

The new training template wraps assistant output in {% generation %} / {% endgeneration %} for assistant-only loss masking and tightens <think> extraction to require both opening and closing tags to avoid mis-splitting partial generations. Tests are extended to cover GLM-4-MoE (skipped on transformers<5.0.0), and the chat-templates README is updated to document the new patch.

Reviewed by Cursor Bugbot for commit 6835d31. Bugbot is set up for automated code reviews on this repo. Configure here.

@qgallouedec

Copy link
Copy Markdown
Member

Thanks! lgtm!

@codex review

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@qgallouedec

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

An unknown error occurred
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@qgallouedec

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5a2ab2d109

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/test_chat_template_utils.py Outdated

@qgallouedec qgallouedec left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@qgallouedec qgallouedec merged commit 9a52d73 into huggingface:main Apr 22, 2026
1 check passed
@casinca casinca deleted the glm4-generation-training-template branch April 22, 2026 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants