Kimi K2.6 can emit untagged machine-readable output when a request requires JSON, structured text, Responses text.format JSON/schema output, or a forced tool payload. The Kimi reasoning parser previously treated that untagged output as implicit reasoning until it saw a visible reasoning end token, so valid payloads such as {"answer": 42} or required tool-call JSON could be hidden from the OpenAI/Responses stream or handed to the wrong parser phase.

Make the request contract explicit and preserve it across parser request rewrites. Structured text contracts bypass implicit reasoning immediately, while forced tool contracts only move into content/tool parsing when the prefix is a plausible tool payload. This avoids treating ordinary assistant text that happens to contain JSON as a tool call under auto tools, and prevents tool-parser generated grammars from being mistaken for caller requested structured text.

Keep visible Kimi reasoning delimiters meaningful: complete <think>...</think> regions and implicit Kimi tool-section boundaries are still stripped as reasoning. The one intentionally ambiguous edge we handle is a constrained structured choice literal that itself starts with <think>, where the allowed choice lets us preserve literal content without changing generic JSON/schema semantics.

Render/disaggregated serving now carries request-scoped reasoning state through GenerateRequest: render marks machine-output contracts as reasoning_ended and forwards effective chat_template_kwargs; disagg passes those values to engine.generate so structured decoding in the worker uses the same Kimi thinking configuration as render.

Also keep tool_choice=none streaming out of tool-call parsing. This overlaps semantically with upstream PRs vllm-project#42752 and vllm-project#42868, which are narrower generic fixes for tool_choice=none; if either lands first, future rebases should drop the duplicate guard but keep the Kimi machine-output/request-contract handling.

Co-authored-by: OpenAI Codex <codex@openai.com>

Kimi K2.6 can emit untagged machine-readable output when a request requires JSON, structured text, Responses text.format JSON/schema output, or a forced tool payload. The Kimi reasoning parser previously treated that untagged output as implicit reasoning until it saw a visible reasoning end token, so valid payloads such as {"answer": 42} or required tool-call JSON could be hidden from the OpenAI/Responses stream or handed to the wrong parser phase.

Make the request contract explicit and preserve it across parser request rewrites. Structured text contracts bypass implicit reasoning immediately, while forced tool contracts only move into content/tool parsing when the prefix is a plausible tool payload. Preserve literal structured choices across rewrite as well, so a constrained choice such as <think>literal is not mistaken for hidden reasoning after structured decoding rewrites the request.

Keep visible Kimi reasoning delimiters meaningful: complete <think>...</think> regions and implicit Kimi tool-section boundaries are still stripped as reasoning. The intentionally ambiguous delimiter-literal edge is only handled when a constrained structured choice proves the literal is allowed, which avoids changing generic JSON/schema semantics.

Render/disaggregated serving now carries request-scoped reasoning state through GenerateRequest: render marks machine-output contracts as reasoning_ended and forwards effective chat_template_kwargs; disagg passes those values to engine.generate so structured decoding in the worker uses the same Kimi thinking configuration as render.

Also keep tool_choice=none streaming out of tool-call parsing. This overlaps semantically with upstream PRs vllm-project#42752 and vllm-project#42868, which are narrower generic fixes for tool_choice=none; if either lands first, future rebases should drop the duplicate guard but keep the Kimi machine-output/request-contract handling.

Co-authored-by: OpenAI Codex <codex@openai.com>

Kimi K2.6 can emit untagged machine-readable output when a request requires JSON, structured text, Responses text.format JSON/schema output, or a forced tool payload. The Kimi reasoning parser previously treated that untagged output as implicit reasoning until it saw a visible reasoning end token, so valid payloads such as {"answer": 42} or required tool-call JSON could be hidden from the OpenAI/Responses stream or handed to the wrong parser phase.

Make the request contract explicit and preserve it across parser request rewrites. Structured text contracts bypass implicit reasoning immediately, while forced tool contracts only move into content/tool parsing when the prefix is a plausible tool payload. Preserve literal structured choices across rewrite as well, so a constrained choice such as <think>literal is not mistaken for hidden reasoning after structured decoding rewrites the request.

Keep visible Kimi reasoning delimiters meaningful: complete <think>...</think> regions and implicit Kimi tool-section boundaries are still stripped as reasoning. The intentionally ambiguous delimiter-literal edge is only handled when a constrained structured choice proves the literal is allowed, which avoids changing generic JSON/schema semantics.

Render/disaggregated serving now carries request-scoped reasoning state through GenerateRequest: render marks machine-output contracts as reasoning_ended and forwards effective chat_template_kwargs; disagg passes those values to engine.generate so structured decoding in the worker uses the same Kimi thinking configuration as render.

Also keep tool_choice=none streaming out of tool-call parsing. This overlaps semantically with upstream PRs vllm-project#42752 and vllm-project#42868, which are narrower generic fixes for tool_choice=none; if either lands first, future rebases should drop the duplicate guard but keep the Kimi machine-output/request-contract handling.

Co-authored-by: OpenAI Codex <codex@openai.com>

Kimi K2.6 can emit untagged machine-readable output when a request requires JSON, structured text, Responses text.format JSON/schema output, or a forced tool payload. The Kimi reasoning parser previously treated that untagged output as implicit reasoning until it saw a visible reasoning end token, so valid payloads such as {"answer": 42} or required tool-call JSON could be hidden from the OpenAI/Responses stream or handed to the wrong parser phase.

Make the request contract explicit and preserve it across parser request rewrites. Structured text contracts bypass implicit reasoning immediately, while forced tool contracts only move into content/tool parsing when the prefix is a plausible tool payload. Preserve literal structured choices across rewrite as well, so a constrained choice such as <think>literal is not mistaken for hidden reasoning after structured decoding rewrites the request.

Keep visible Kimi reasoning delimiters meaningful: complete <think>...</think> regions and implicit Kimi tool-section boundaries are still stripped as reasoning. The intentionally ambiguous delimiter-literal edge is only handled when a constrained structured choice proves the literal is allowed, which avoids changing generic JSON/schema semantics.

Render/disaggregated serving now carries request-scoped reasoning state through GenerateRequest: render marks machine-output contracts as reasoning_ended and forwards effective chat_template_kwargs; disagg passes those values to engine.generate so structured decoding in the worker uses the same Kimi thinking configuration as render.

Also keep tool_choice=none streaming out of tool-call parsing. This overlaps semantically with upstream PRs vllm-project#42752 and vllm-project#42868, which are narrower generic fixes for tool_choice=none; if either lands first, future rebases should drop the duplicate guard but keep the Kimi machine-output/request-contract handling.

Co-authored-by: OpenAI Codex <codex@openai.com>

Kimi K2.6 can emit untagged machine-readable output when a request requires JSON, structured text, Responses text.format JSON/schema output, or a forced tool payload. The Kimi reasoning parser previously treated that untagged output as implicit reasoning until it saw a visible reasoning end token, so valid payloads such as {"answer": 42} or required tool-call JSON could be hidden from the OpenAI/Responses stream or handed to the wrong parser phase.

Make the request contract explicit and preserve it across parser request rewrites. Structured text contracts bypass implicit reasoning immediately, while forced tool contracts only move into content/tool parsing when the prefix is a plausible tool payload. Preserve literal structured choices across rewrite as well, so a constrained choice such as <think>literal is not mistaken for hidden reasoning after structured decoding rewrites the request.

Keep visible Kimi reasoning delimiters meaningful: complete <think>...</think> regions and implicit Kimi tool-section boundaries are still stripped as reasoning. The intentionally ambiguous delimiter-literal edge is only handled when a constrained structured choice proves the literal is allowed, which avoids changing generic JSON/schema semantics.

Render/disaggregated serving now carries request-scoped reasoning state through GenerateRequest: render marks machine-output contracts as reasoning_ended and forwards effective chat_template_kwargs; disagg passes those values to engine.generate so structured decoding in the worker uses the same Kimi thinking configuration as render.

Also keep tool_choice=none streaming out of tool-call parsing. This overlaps semantically with upstream PRs vllm-project#42752 and vllm-project#42868, which are narrower generic fixes for tool_choice=none; if either lands first, future rebases should drop the duplicate guard but keep the Kimi machine-output/request-contract handling.

Co-authored-by: OpenAI Codex <codex@openai.com>

Kimi K2 emits tool calls with native structural markers like <|tool_calls_section_begin|> and <|tool_call_begin|> functions.<name>:<id>, not the generic JSON payload used by the default required/named tool-choice path. When forced tool choices are guided and parsed as generic JSON, streamed responses can lose parsed tool calls or prevent visible reasoning before the native tool section.

Add a Kimi structural tag so required and named tool choices constrain generation to the same native format that KimiK2ToolParser already understands, and mark the parser as not supporting the generic required/named parser. The tag allows optional whitespace at the separator positions seen in Kimi K2.6 e2e output and already accepted by the parser regex, so guidance does not force the model away from its native distribution.

When structured outputs are enabled during reasoning, include a reasoning prefix that allows Kimi to complete its template-opened <think> block before the native tool-call section. Gate that prefix on the engine enable_in_reasoning setting and Kimi's thinking chat-template knob, not include_reasoning, because include_reasoning only controls response visibility.

Keep auto/none/no-tool behavior unchanged unless VLLM_ENFORCE_STRICT_TOOL_CALLING routes auto through structural tags, in which case Kimi now uses the same native tag builder as required/named. This change does not address the separate generic streaming parser issue where tool_choice="none" can still enter tool-call parsing; that is covered by vLLM PRs vllm-project#42752 and vllm-project#42868. Preserve strict=false tool definitions by disabling argument-schema guidance for that tool, and reject xgrammar-unsupported JSON schema features before installing the structural tag so unsupported schemas fail consistently with plain JSON structured outputs.

Tests cover Kimi structural-tag request adjustment, strict auto routing, strict=false tool schemas, xgrammar-unsupported schema rejection, opt-out from generic required/named parsing, replacement of conflicting structured-output constraints, structural-tag validation, reasoning-prefix gating by bitmask phase and Kimi thinking mode, and include_reasoning visibility not changing the grammar shape.

Co-authored-by: OpenAI Codex <codex@openai.com>

Signed-off-by: Ace Eldeib <aeldeib@coreweave.com>

Mirror of upstream vllm-project/vllm#42752 (fixes vllm-project/vllm#42747).

Streaming Chat Completions with tool_choice="none" (or omitted on a
no-tools request) could still produce delta.tool_calls and finish with
finish_reason="tool_calls" because DelegatingParser.parse_delta invokes
_extract_tool_calls_streaming unconditionally once the stream enters the
tool-call phase, ignoring request.tool_choice. Non-streaming already
short-circuits this in chat_completion/serving.py:1250:

    elif not request.tool_choice or request.tool_choice == "none":
        message = ChatMessage(role=role, reasoning=reasoning, content=content)

Replicate the same semantics on the streaming path: when tool_choice is
None or "none", skip the tool parser inside the tool-call phase and
surface the (post-reasoning) delta_text as plain DeltaMessage.content.

Effect: DSV4 DSML markup (and any other parser's tool-call-looking
output) stays in delta.content, matching the non-streaming behavior,
and finish_reason falls back to "stop".

Replaces the previous patch_dsv4_dsml_tool_choice_none.py approach,
which incorrectly stripped DSML markup from non-streaming content.
The new direction follows the upstream consensus in issue #42747:
both modes leave the markup in content, neither strips it.

Mirror of upstream vllm-project/vllm#42752 (fixes vllm-project/vllm#42747).

Streaming Chat Completions with tool_choice="none" (or omitted on a
no-tools request) could still produce delta.tool_calls and finish with
finish_reason="tool_calls" because DelegatingParser.parse_delta invokes
_extract_tool_calls_streaming unconditionally once the stream enters the
tool-call phase, ignoring request.tool_choice. Non-streaming already
short-circuits this in chat_completion/serving.py:1250:

    elif not request.tool_choice or request.tool_choice == "none":
        message = ChatMessage(role=role, reasoning=reasoning, content=content)

Replicate the same semantics on the streaming path: when tool_choice is
None or "none", skip the tool parser inside the tool-call phase and
surface the (post-reasoning) delta_text as plain DeltaMessage.content.

Effect: DSV4 DSML markup (and any other parser's tool-call-looking
output) stays in delta.content, matching the non-streaming behavior,
and finish_reason falls back to "stop".

Replaces the previous patch_dsv4_dsml_tool_choice_none.py approach,
which incorrectly stripped DSML markup from non-streaming content.
The new direction follows the upstream consensus in issue #42747:
both modes leave the markup in content, neither strips it.

Signed-off-by: liuchenbing <chenliumail@163.com>

DelegatingParser.parse_delta unconditionally invoked the configured
tool parser once the stream entered the tool-call phase, so streaming
Chat Completions could still emit delta.tool_calls and finish with
finish_reason="tool_calls" whenever a --tool-call-parser was configured
and the model output happened to match that parser format -- even when
the client disabled tools. The non-streaming path already short-circuits
both cases in chat_completion/serving.py:

    elif not request.tool_choice or request.tool_choice == "none":

Mirror that guard here. When `not request.tool_choice or
request.tool_choice == "none"` -- i.e. tool_choice="none" OR explicitly
disabled via JSON null (request.tool_choice is None) -- skip
extract_tool_calls_streaming and surface the accumulated post-reasoning
text as plain content. The tool parser is never invoked, so
function_name_returned/tools_streamed stay False and finish_reason falls
back to "stop". Reasoning extraction on boundary deltas is preserved.

This broadens the original tool_choice=="none"-only guard to also cover
request.tool_choice is None, per review feedback on vllm-project#42752, so streaming
matches the non-streaming semantics exactly.

Fixes vllm-project#42747

Signed-off-by: hoobnn <111053672+hoobnn@users.noreply.github.com>

DelegatingParser.parse_delta unconditionally invoked the configured
tool parser once the stream entered the tool-call phase, so streaming
Chat Completions could still emit delta.tool_calls and finish with
finish_reason="tool_calls" whenever a --tool-call-parser was configured
and the model output happened to match that parser format -- even when
the client disabled tools. The non-streaming path already short-circuits
both cases in chat_completion/serving.py:

    elif not request.tool_choice or request.tool_choice == "none":

Mirror that guard here. When `not request.tool_choice or
request.tool_choice == "none"` -- i.e. tool_choice="none" OR explicitly
disabled via JSON null (request.tool_choice is None) -- skip
extract_tool_calls_streaming and surface the accumulated post-reasoning
text as plain content. The tool parser is never invoked, so
function_name_returned/tools_streamed stay False and finish_reason falls
back to "stop". Reasoning extraction on boundary deltas is preserved.

This broadens the original tool_choice=="none"-only guard to also cover
request.tool_choice is None, per review feedback on vllm-project#42752, so streaming
matches the non-streaming semantics exactly.

Fixes vllm-project#42747

Signed-off-by: hoobnn <111053672+hoobnn@users.noreply.github.com>

…-project#42752)

Signed-off-by: hoobnn <111053672+hoobnn@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>

…-project#42752)

Signed-off-by: hoobnn <111053672+hoobnn@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: JisoLya <523420504@qq.com>

…-project#42752)

Signed-off-by: hoobnn <111053672+hoobnn@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>

… API

Three adaptations required after upstream refactors:

1. _WrappedParser removed (vllm-project#44279): replaced with an inline subclass
   _Gemma4Parser(DelegatingParser) with reasoning_parser_cls and
   tool_parser_cls set as class attributes directly.

2. parse_delta() gained a required `finished` kwarg (vllm-project#44017): updated
   _run_streaming to pass finished=(last token), _run_single_delta to
   pass finished=True, and the multi-turn loop to pass finished=False.

3. tool_choice="none" short-circuit added (vllm-project#42752): parse_delta now
   returns raw content immediately when request.tool_choice is "none",
   which is the default when no tools are specified. Fixed _make_request
   to include a dummy tool so tool_choice stays "auto" and the parser
   exercises its actual tool-extraction logic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…-project#42752)

Signed-off-by: hoobnn <111053672+hoobnn@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Waqar Ahmed <waqar.ahmed@amd.com>

Kimi K2 emits tool calls with native structural markers like <|tool_calls_section_begin|> and <|tool_call_begin|> functions.<name>:<id>, not the generic JSON payload used by the default required/named tool-choice path. When forced tool choices are guided and parsed as generic JSON, streamed responses can lose parsed tool calls or prevent visible reasoning before the native tool section.

Add a Kimi structural tag so required and named tool choices constrain generation to the same native format that KimiK2ToolParser already understands, and mark the parser as not supporting the generic required/named parser. The tag allows optional whitespace at the separator positions seen in Kimi K2.6 e2e output and already accepted by the parser regex, so guidance does not force the model away from its native distribution.

When structured outputs are enabled during reasoning, include a reasoning prefix that allows Kimi to complete its template-opened <think> block before the native tool-call section. Gate that prefix on the engine enable_in_reasoning setting and Kimi's thinking chat-template knob, not include_reasoning, because include_reasoning only controls response visibility.

Keep auto/none/no-tool behavior unchanged unless VLLM_ENFORCE_STRICT_TOOL_CALLING routes auto through structural tags, in which case Kimi now uses the same native tag builder as required/named. This change does not address the separate generic streaming parser issue where tool_choice="none" can still enter tool-call parsing; that is covered by vLLM PRs vllm-project#42752 and vllm-project#42868. Preserve strict=false tool definitions by disabling argument-schema guidance for that tool, and reject xgrammar-unsupported JSON schema features before installing the structural tag so unsupported schemas fail consistently with plain JSON structured outputs.

Tests cover Kimi structural-tag request adjustment, strict auto routing, strict=false tool schemas, xgrammar-unsupported schema rejection, opt-out from generic required/named parsing, replacement of conflicting structured-output constraints, structural-tag validation, reasoning-prefix gating by bitmask phase and Kimi thinking mode, and include_reasoning visibility not changing the grammar shape.

Co-authored-by: OpenAI Codex <codex@openai.com>

Signed-off-by: Ace Eldeib <aeldeib@coreweave.com>

… API

Three adaptations required after upstream refactors:

1. _WrappedParser removed (vllm-project#44279): replaced with an inline subclass
   _Gemma4Parser(DelegatingParser) with reasoning_parser_cls and
   tool_parser_cls set as class attributes directly.

2. parse_delta() gained a required `finished` kwarg (vllm-project#44017): updated
   _run_streaming to pass finished=(last token), _run_single_delta to
   pass finished=True, and the multi-turn loop to pass finished=False.

3. tool_choice="none" short-circuit added (vllm-project#42752): parse_delta now
   returns raw content immediately when request.tool_choice is "none",
   which is the default when no tools are specified. Fixed _make_request
   to include a dummy tool so tool_choice stays "auto" and the parser
   exercises its actual tool-extraction logic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

… API

Three adaptations required after upstream refactors:

1. _WrappedParser removed (vllm-project#44279): replaced with an inline subclass
   _Gemma4Parser(DelegatingParser) with reasoning_parser_cls and
   tool_parser_cls set as class attributes directly.

2. parse_delta() gained a required `finished` kwarg (vllm-project#44017): updated
   _run_streaming to pass finished=(last token), _run_single_delta to
   pass finished=True, and the multi-turn loop to pass finished=False.

3. tool_choice="none" short-circuit added (vllm-project#42752): parse_delta now
   returns raw content immediately when request.tool_choice is "none",
   which is the default when no tools are specified. Fixed _make_request
   to include a dummy tool so tool_choice stays "auto" and the parser
   exercises its actual tool-extraction logic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
(cherry picked from commit c37293e)