Update missing docs on activation_dropout and fix DropOut docs for SEW-D#26031
Merged
amyeroberts merged 3 commits intohuggingface:mainfrom Sep 8, 2023
gau-nernst:update_docs
Merged
Update missing docs on activation_dropout and fix DropOut docs for SEW-D#26031amyeroberts merged 3 commits intohuggingface:mainfrom gau-nernst:update_docs
activation_dropout and fix DropOut docs for SEW-D#26031amyeroberts merged 3 commits intohuggingface:mainfrom
gau-nernst:update_docs
Conversation
stevhliu
approved these changes
Sep 7, 2023
Member
stevhliu
left a comment
There was a problem hiding this comment.
Good catch, thanks for adding these!
| The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, | ||
| `"relu"`, `"selu"`, `"gelu_python"` and `"gelu_new"` are supported. | ||
| hidden_dropout (`float`, *optional*, defaults to 0.1): | ||
| Not used. |
Member
There was a problem hiding this comment.
If hidden_dropout is not used, maybe we can just remove it? cc @amyeroberts
Contributor
There was a problem hiding this comment.
We'd need to do a deprecation cycle in case any users use it in their own code. So something along the lines of:
class SEWDConfig(PretrainedConfig):
"""
...
hidden_dropout (`float`, *optional*, defaults to 0.1):
Deprecated. Not used by the model and will be removed in a future version.
...
"""
def __init__(...):
...
self._hidden_dropout = hidden_dropout
@property
def hidden_dropout(self):
logger.warning_once("hidden_dropout is not used by the model and will be removed as config attribute in v4.35")
return self._hidden_dropout
def to_dict(self):
"""
Serializes this instance to a Python dictionary.
"""
output = super().to_dict()
output["hidden_dropout"] = output.pop("_hidden_dropout")
return output |
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
amyeroberts
approved these changes
Sep 8, 2023
Contributor
amyeroberts
left a comment
There was a problem hiding this comment.
Thanks for fixing these!
Contributor
sanchit-gandhi
left a comment
There was a problem hiding this comment.
Thanks for the clean PR @gau-nernst!
| return functools.reduce(operator.mul, self.conv_stride, 1) | ||
|
|
||
| @property | ||
| def hidden_dropout(self): |
parambharat
pushed a commit
to parambharat/transformers
that referenced
this pull request
Sep 26, 2023
…SEW-D (huggingface#26031) * add missing doc for activation dropout * fix doc for SEW-D dropout * deprecate hidden_dropout for SEW-D
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #25854
Add doc for
activation_dropoutfor various audio models. Let me know if I miss out any.For SEW-D, document the behavior that
hidden_dropoutis not used, whileactivation_dropoutacts more likehidden_dropoutin other models.On a side note, it will be good if there is a test to catch undocumented config attributes in the future.
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@sanchit-gandhi