[compat] Allow for both warmup_steps and warmup_ratio until transformers v4 support is dropped#3645
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request adds a compatibility layer to handle the warmup_steps and warmup_ratio arguments across transformers v4 and v5. In transformers v4, warmup_ratio accepts float ratios and warmup_steps accepts integer step counts. In transformers v5+, warmup_ratio is deprecated and warmup_steps now accepts both integers and floats. This PR ensures users can use either argument with both transformers versions until transformers v4 support is dropped.
Changes:
- Added explicit
warmup_ratiofield toSentenceTransformerTrainingArgumentswith deprecation notice - Implemented version-specific logic in
__post_init__to convert betweenwarmup_stepsandwarmup_ratiobased on transformers version - Added comprehensive tests to verify the compatibility logic works correctly for both transformers v4 and v5
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| sentence_transformers/training_args.py | Added warmup_ratio field and post_init compatibility logic to handle warmup arguments across transformers versions |
| tests/test_training_args.py | Added parametrized test to verify warmup_steps/warmup_ratio compatibility across transformers versions |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Hello!
Pull Request overview
Details
Transformers v4.x supports only
warmup_ratiowith floats andwarmup_stepswith integers.Transformers v5.x supports
warmup_stepswith both floats and integers.warmup_ratiois now deprecated.I want to keep support for Transformers v4.x and v5.x for the time being, so I'm adding
warmup_ratioon the Sentence Transformers side, and pre-processing the Training Arguments so either can be used. I'll move to onlywarmup_stepswhen we drop Transformers v4 support in the next major version.