revert ga fix#5030
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the patch_gradient_accumulation_fix function in unsloth/models/_utils.py to use a regular expression for patching the Trainer.init method, replacing the previous string-based substitution. Feedback suggests improving the robustness of the regex to handle different quote styles (single vs. double) and simplifying the whitespace matching pattern.
| init_function = re.sub( | ||
| r'if[\s]+hasattr\(\s*unwrapped_model\s*,\s*"accepts_loss_kwargs"\s*\)\s*:', | ||
| 'if hasattr(unwrapped_model, "accepts_loss_kwargs") and False:', | ||
| init_function, | ||
| ) |
There was a problem hiding this comment.
The regex used for patching Trainer.__init__ is currently sensitive to the quote style used in the transformers source code (it only matches double quotes for "accepts_loss_kwargs"). To ensure the patch remains robust across different versions or environments where single quotes might be used, it's recommended to support both quote types. Additionally, [\s]+ can be simplified to \s+.
| init_function = re.sub( | |
| r'if[\s]+hasattr\(\s*unwrapped_model\s*,\s*"accepts_loss_kwargs"\s*\)\s*:', | |
| 'if hasattr(unwrapped_model, "accepts_loss_kwargs") and False:', | |
| init_function, | |
| ) | |
| init_function = re.sub( | |
| r'if\s+hasattr\(\s*unwrapped_model\s*,\s*["\']accepts_loss_kwargs["\']\s*\)\s*:', | |
| 'if hasattr(unwrapped_model, "accepts_loss_kwargs") and False:', | |
| init_function, | |
| ) |
No description provided.