[DOCS] Update docstrings for GPT2 and Whisper tokenizer#26642
[DOCS] Update docstrings for GPT2 and Whisper tokenizer#26642ydshieh merged 4 commits intohuggingface:mainfrom
Conversation
|
@ydshieh Does this look good for you? |
ydshieh
left a comment
There was a problem hiding this comment.
A few nits but ready to go 🔥
| eos_token (`str`, *optional*, defaults to `"<|endoftext|>"`): | ||
| The end of sequence token. | ||
| pad_token (`str`, *optional*): | ||
| Id of the padding token in the vocabulary. |
There was a problem hiding this comment.
| Id of the padding token in the vocabulary. | |
| The token used for padding, for example when batching sequences of different lengths. |
| eos_token (`str`, *optional*, defaults to `"<|endoftext|>"`): | ||
| The end of sequence token. | ||
| pad_token (`str`, *optional*): | ||
| Id of the padding token in the vocabulary. |
There was a problem hiding this comment.
| Id of the padding token in the vocabulary. | |
| The token used for padding, for example when batching sequences of different lengths. |
|
@ydshieh I tried troubleshooting on my side I don't understanding the difference between |
|
Hi @McDonnellJoseph It's fine. No need to dive into those 2 files. I requests 2 tiny changes. Once you commit them, we are good to merge the PR. |
|
@ydshieh No problem I pushed the requested changes 😄 |
36e4143 to
04d9a82
Compare
|
Hi, thanks for the commit. May I know why there is irrelevant changes other than pad token id in the last commit pushed? |
64c8e08 to
b100203
Compare
|
Sorry my linter formatted automatically and I dind't notice it should be fixed now |
|
We are very close: just need to |
|
Ok I thing we're good now 😄 |
ydshieh
left a comment
There was a problem hiding this comment.
Very nice, thank you for the contribution 💯 !
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
What does this PR do?
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.