Move disable_config=True from generate to GenerationConfig#5384
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 81d5f90e68
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if args.generation_kwargs is not None: | ||
| generation_kwargs.update(args.generation_kwargs) | ||
| self.generation_config = GenerationConfig(**generation_kwargs) | ||
| self.generation_config = GenerationConfig(**generation_kwargs, disable_compile=True) |
There was a problem hiding this comment.
Handle user-provided disable_compile without crashing
Because args.generation_kwargs is merged into generation_kwargs immediately above, this constructor will now fail with TypeError if a caller already passes disable_compile in generation_kwargs (for example from CLI/JSON config), since Python sees the key twice (**generation_kwargs and disable_compile=True). That makes training crash during init for a previously valid input path; the same duplicated pattern is also present in the corresponding RLOO and self-distillation constructors.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Before
After
Note
Low Risk
Low risk behavioral change that only affects how
disable_compileis passed into Transformers generation, primarily to avoid deprecation warnings; generation outputs should remain the same aside from compile/graph behavior.Overview
Moves
disable_compile=Truefrom individualmodel.generate(...)calls into the trainers’GenerationConfigconstruction (GRPO, RLOO, SDFT, and experimental self-distillation), and updates generation call sites to pass onlygeneration_config.This eliminates the Transformers deprecation warning about mixing
generation_configwith extra generation kwargs while keeping compilation disabled during rollout generation.Written by Cursor Bugbot for commit 528f44b. This will update automatically on new commits. Configure here.