Skip to content

[Bugfix][Tool Parser] Fix Kimi-K2 tool call arguments when using tool_choice#44934

Open
ssam18 wants to merge 3 commits into
vllm-project:mainfrom
ssam18:fix/44906-kimi-k2-named-tool-choice
Open

[Bugfix][Tool Parser] Fix Kimi-K2 tool call arguments when using tool_choice#44934
ssam18 wants to merge 3 commits into
vllm-project:mainfrom
ssam18:fix/44906-kimi-k2-named-tool-choice

Conversation

@ssam18

@ssam18 ssam18 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Purpose

Fixes #44906.

Kimi-K2 emits tool calls as special-token markup (<|tool_calls_section_begin|> ... <|tool_call_argument_begin|> {...} <|tool_call_end|> ...) and relies on its tool parser to pull the JSON arguments back out. This works fine for tool_choice: "auto", but when the request names a function (or sets tool_choice: "required") the server takes the standard JSON path and copies the raw model output straight into the call arguments. So the arguments come back full of the markup instead of the actual JSON, which is exactly what the issue reports.

The Kimi parser was inheriting the default supports_required_and_named = True, which tells the server the model already produces clean JSON for named and required tool choice. That is not true for this format. Setting it to False routes named and required choices through extract_tool_calls instead, the same way the GLM4-MoE parsers already handle their markup-based format. The change is a one line flag on KimiK2ToolParser plus a regression test.

Test Plan

pytest tests/tool_parsers/test_kimi_k2_tool_parser.py

Test Result

The new test_named_and_required_routed_through_parser passes and the existing Kimi-K2 parser tests still pass. I also confirmed against the exact payload from the issue that once the named and required paths go through the parser, extract_tool_calls returns name=get_weather and arguments={"city": "Boston"} rather than the raw markup.

…_choice

Kimi-K2 emits tool calls as special-token markup and relies on its parser to
recover the JSON arguments. When tool_choice names a function or is set to
"required", the server took the standard JSON path and copied the raw model
output straight into the arguments, so the returned arguments came back full
of <|tool_calls_section_begin|> markup instead of the actual JSON. Setting
supports_required_and_named = False routes those cases through the parser, the
same way the GLM4-MoE parsers already handle this kind of markup.
Fixes vllm-project#44906
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>

Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
@mergify

mergify Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ssam18.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Jun 12, 2026
…ed-tool-choice

# Conflicts:
#	vllm/tool_parsers/kimi_k2_tool_parser.py
@mergify mergify Bot removed the needs-rebase label Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working tool-calling

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[Bug]: Kimi-k2.6 tool call parser bug when using tool_choice

1 participant