Skip to content

Make training args fully immutable#25435

Merged
muellerzr merged 11 commits intomainfrom
muellerzr-immutable
Aug 15, 2023
Merged

Make training args fully immutable#25435
muellerzr merged 11 commits intomainfrom
muellerzr-immutable

Conversation

@muellerzr
Copy link
Contributor

What does this PR do?

This PR ensures that the TrainingArguments are a fully immutable dataclass after the __post_init__ has been ran. We'll find that the tests suddenly fail now 😉 Should be merged after #25390

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts @sgugger


def __setattr__(self, name, value):
# Once fully through the `__post_init__`, `TrainingArguments` are immutable
if getattr(self, "_frozen", False):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since adding _frozen to the dataclass args earlier would make it show up as an available option to pass through, we don't set it until the very end of __post_init__, hence the getattr instead of setting it earlier.

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Aug 10, 2023

The documentation is not available anymore as the PR was closed or merged.

Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Now to see what breaks with this 😅

Copy link
Contributor

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice ❄️ !

@muellerzr muellerzr requested a review from sgugger August 10, 2023 18:18
exit(1)

trainer.args.eval_accumulation_steps = 2
trainer.args._set_value("eval_accumulation_steps", 2)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to create a new set of training args here. People sometimes look for inspiration in our tests and we definitely don't want to advertise that method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, there's also a method in dataclasses that lets us change frozen params the proper way with a new set of args overriding it

@muellerzr muellerzr force-pushed the muellerzr-immutable branch from 3e3e1ee to 02c2b62 Compare August 15, 2023 13:13
@muellerzr muellerzr requested a review from sgugger August 15, 2023 14:28
@muellerzr
Copy link
Contributor Author

@sgugger can you give it one final look please 😄

Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looking great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants