Hi 🤗 team,
I'm writing to inquire about the design choice to set the default top_k sampling parameter to 50 in the transformers library.
|
self.top_k = kwargs.pop("top_k", 50) |
It appears top_k is the only sampling parameter with an opinionated default value, as others like top_p are typically set to a neutral value (e.g., 1.0). For consistency and to allow for more flexible default behavior (i.e., no top_k filtering by default), I would personally advocate for a default value of -1, similar to how vLLM handles its sampling parameters (vLLM SamplingParams documentation).
Could you please clarify the reasoning behind this specific default?
Thank you for your time and consideration.
Hi 🤗 team,
I'm writing to inquire about the design choice to set the default top_k sampling parameter to 50 in the transformers library.
transformers/src/transformers/generation/configuration_utils.py
Line 431 in f4fc422
It appears top_k is the only sampling parameter with an opinionated default value, as others like top_p are typically set to a neutral value (e.g., 1.0). For consistency and to allow for more flexible default behavior (i.e., no top_k filtering by default), I would personally advocate for a default value of -1, similar to how vLLM handles its sampling parameters (vLLM SamplingParams documentation).
Could you please clarify the reasoning behind this specific default?
Thank you for your time and consideration.