Skip to content

Simplify logic for structured outputs across vLLM versions#5215

Merged
albertvillanova merged 7 commits into
huggingface:mainfrom
albertvillanova:fu-5155
Mar 5, 2026
Merged

Simplify logic for structured outputs across vLLM versions#5215
albertvillanova merged 7 commits into
huggingface:mainfrom
albertvillanova:fu-5155

Conversation

@albertvillanova

@albertvillanova albertvillanova commented Mar 4, 2026

Copy link
Copy Markdown
Member

Simplify logic for structured outputs across vLLM versions.

This PR refactors the handling of structured output parameters for vLLM generation to improve compatibility across different vLLM versions and simplify the logic for setting structured output options. The main changes focus on unifying the way structured output parameters are imported and used, and on reducing code duplication related to vLLM version checks.

vLLM structured output handling improvements:

  • Unified import and usage of StructuredOutputsParams and its key in generation_kwargs based on the vLLM version, ensuring compatibility with both pre- and post-0.10.2 versions.
  • Refactored logic for setting the structured output parameter in generation_kwargs, removing duplicated version checks and using a single structured_outputs_key variable for consistency.

Code cleanup:

  • Removed redundant imports of vLLM components and version checks from the top-level scope, moving them to where they are needed in the code.
  • Updated imports to include LLM, RequestOutput, and SamplingParams only when vLLM is available, improving import hygiene.

@qgallouedec qgallouedec left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! better

@albertvillanova albertvillanova merged commit 370eee7 into huggingface:main Mar 5, 2026
11 checks passed
catherinelee274 pushed a commit to catherinelee274/trl that referenced this pull request Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants