fix: preserve native tool call ID in multi-turn tool calling by wangln19 · Pull Request #32768 · vllm-project/vllm

wangln19 · 2026-01-21T09:21:12Z

In multi-turn tool calling scenarios, models like Kimi K2 generate tool calls with specific ID formats (e.g., 'functions.get_weather:0'). The model expects to see these IDs in subsequent tool results to match them correctly.

Previously, _parse_tool_calls_from_content() was discarding the native tool call ID parsed by the tool parser, and the serving layer was generating random IDs instead. This broke multi-turn tool calling for models that rely on consistent tool call IDs.

This fix:

Add optional 'id' field to FunctionCall class
Preserve native ID in _parse_tool_calls_from_content()
Use preserved ID in chat_completion_full_generator()
Add .strip() to clean whitespace in Kimi K2 tool parser

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

gemini-code-assist

Code Review

The pull request successfully addresses the issue of preserving native tool call IDs in multi-turn tool calling scenarios, particularly for models like Kimi K2. The addition of the id field to FunctionCall and its preservation in _parse_tool_calls_from_content() are crucial improvements. The .strip() calls in kimi_k2_tool_parser.py also contribute to cleaner data handling. However, there's a potential issue in chat_completion_full_generator() where make_tool_call_id() might not generate IDs in the expected format for Kimi K2 models if the original tc.id is None.

wangln19 · 2026-01-21T09:23:11Z

Related to: #32504 #32216 #30238 #29596

mergify · 2026-01-21T09:37:30Z

Hi @wangln19, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

mergify · 2026-01-21T09:46:09Z

Hi @wangln19, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

mergify · 2026-01-21T10:08:49Z

Hi @wangln19, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

wangln19 · 2026-01-21T10:32:55Z

Note: The mypy error at line 1736 is a pre-existing issue in the main branch (line 1712 in main). This PR does not introduce any new type errors.

daniel-salib

thanks for the fix! I tried the fix locally and it fixes the issue I've been having with Kimi k2 tool parsing

daniel-salib · 2026-01-21T11:15:40Z

            elif (
                request.tool_choice
                and type(request.tool_choice) is ChatCompletionNamedToolChoiceParam
            ):
                assert tool_calls is not None and len(tool_calls) > 0
+                tool_call_class_items = []
+                for tc in tool_calls:
+                    tool_call_class_items.append(


would we need to apply the same pattern in responses/serving.py to support responses API?

daniel-salib · 2026-01-21T11:20:02Z

+                    tool_call_class_items.append(
+                        tool_call_class(
+                            id=tc.id
+                            if tc.id


if tc.id is an empty string do we still want o call make_tool_call_id?

Thanks, applying the same pattern to responses/serving.py makes sense for consistency. And the if tc.id check handles empty strings correctly by falling back to make_tool_call_id.

mergify · 2026-01-21T11:47:07Z

Hi @wangln19, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

qandrew

thanks for putting this together! can you write a unit test to preserve behavior?

wangln19 · 2026-01-22T08:32:01Z

thanks for putting this together! can you write a unit test to preserve behavior?

I considered adding unit tests but realized the cost-benefit doesn't favor it here:
To properly test the serving layer, I would need to either:
Extract the loop logic into a separate function and refactor 3 places in serving.py to use it or mock the entire serving layer with complex setup.

mergify · 2026-01-22T08:34:04Z

Hi @wangln19, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

mergify · 2026-01-22T10:46:47Z

Hi @wangln19, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

mergify · 2026-01-22T11:08:46Z

Hi @wangln19, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

mergify · 2026-01-25T04:09:44Z

Hi @wangln19, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>

mergify · 2026-01-26T15:59:41Z

Hi @wangln19, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>

Co-authored-by: Isotr0py <2037008807@qq.com> Signed-off-by: Roger Wang <hey@rogerw.io>

Signed-off-by: Roger Wang <hey@rogerw.io>

ywang96 · 2026-01-26T23:21:21Z

@cursor review

cursor · 2026-01-26T23:31:03Z

+                            tool_call_class_items.append(
+                                tool_call_class(id=generated_id, function=tc)
+                            )
+                    history_tool_call_cnt += 1


Double-counting causes non-sequential tool call indices

Medium Severity

The tool call index calculation uses history_tool_call_cnt + idx where idx comes from enumerate(), but history_tool_call_cnt is also incremented inside the loop. This causes indices to skip values. For example, with 3 tool calls starting at history count 5, the indices would be 5, 7, 9 instead of the expected 5, 6, 7. For Kimi K2, this produces IDs like functions.get_weather:5, functions.get_weather:7, functions.get_weather:9 breaking the sequential indexing the model expects.

Additional Locations (2)

vllm/entrypoints/openai/chat_completion/serving.py#L1620-L1626

vllm/entrypoints/openai/chat_completion/serving.py#L1668-L1674

Signed-off-by: Roger Wang <hey@rogerw.io>

Co-authored-by: Isotr0py <2037008807@qq.com> Signed-off-by: Roger Wang <hey@rogerw.io>

ywang96 · 2026-01-27T00:30:43Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

…oject#32768) Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn> Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com> Signed-off-by: Roger Wang <hey@rogerw.io> Co-authored-by: Roger Wang <hey@rogerw.io> Co-authored-by: Isotr0py <2037008807@qq.com>

Audited recent tool parser bug-fix PRs and found that several landed without corresponding test coverage. Added unit tests for each fix to prevent regressions. - Mistral: fast detokenization text detection (PR vllm-project#37209) - Qwen3Coder: malformed XML crash, anyOf double-encoding, speculative decode streaming (PRs vllm-project#36774, vllm-project#36032, vllm-project#35615) - DeepSeekV32: delimiter preservation with fast detokenization, skip_special_tokens adjustment (PR vllm-project#33964) - GLM-4 MoE: zero-argument tool calls, transformers 5.x delimiter handling, Unicode character preservation (PRs vllm-project#32321, vllm-project#31622, vllm-project#30920) - MiniMax M2: anyOf nullable parameter handling for non-null and null values (PR vllm-project#32342) - Step3p5: MTP-style variable-chunk and multi-token streaming (PR vllm-project#33690) - Kimi K2: native tool call ID extraction and multi-turn ID continuity (PR vllm-project#32768) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ben Browning <bbrownin@redhat.com>

…oject#32768) Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn> Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com> Signed-off-by: Roger Wang <hey@rogerw.io> Co-authored-by: Roger Wang <hey@rogerw.io> Co-authored-by: Isotr0py <2037008807@qq.com>

wangln19 requested review from aarnphm and chaunceyjiang as code owners January 21, 2026 09:21

mergify Bot added the frontend label Jan 21, 2026

gemini-code-assist Bot reviewed Jan 21, 2026

View reviewed changes

Comment thread vllm/entrypoints/openai/chat_completion/serving.py Outdated

Comment thread vllm/entrypoints/openai/chat_completion/serving.py Outdated

cursor Bot reviewed Jan 21, 2026

View reviewed changes

Comment thread vllm/entrypoints/openai/engine/protocol.py Outdated

wangln19 force-pushed the fix/preserve-tool-call-id branch from 2ba5175 to c21c48b Compare January 21, 2026 09:33

wangln19 force-pushed the fix/preserve-tool-call-id branch from c21c48b to f5c8c24 Compare January 21, 2026 09:42

wangln19 force-pushed the fix/preserve-tool-call-id branch from f5c8c24 to 268ff59 Compare January 21, 2026 10:02

MoyanZitto mentioned this pull request Jan 21, 2026

[Frontend] Add dedicated KimiK2ReasoningParser for tool call handling #32216

Closed

daniel-salib reviewed Jan 21, 2026

View reviewed changes

daniel-salib mentioned this pull request Jan 21, 2026

[Frontend] Support OpenAI-style tool call IDs in Kimi K2 parser #32146

Closed

qandrew reviewed Jan 21, 2026

View reviewed changes

wangln19 requested review from DarkLight1337, NickLucche and robertgshaw2-redhat as code owners January 22, 2026 08:29

wangln19 force-pushed the fix/preserve-tool-call-id branch from 7009a7a to 17c8fb7 Compare January 22, 2026 10:42

wangln19 force-pushed the fix/preserve-tool-call-id branch from 17c8fb7 to b3435a8 Compare January 22, 2026 11:04

ywang96 approved these changes Jan 25, 2026

View reviewed changes

ywang96 added the ready ONLY add when PR is ready to merge/full CI is needed label Jan 25, 2026

wangln19 added 2 commits January 26, 2026 23:39

update

c31498d

Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>

final

5508de7

Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>

wangln19 force-pushed the fix/preserve-tool-call-id branch from b1cab5a to 5508de7 Compare January 26, 2026 15:55

wangln19 and others added 6 commits January 27, 2026 00:10

final

e03a8e2

Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>

fix(openai): resolve mypy type errors in chat completion serving

31cf03f

Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>

Merge branch 'main' into fix/preserve-tool-call-id

3420f3b

Update vllm/entrypoints/openai/chat_completion/serving.py

fa38bed

Co-authored-by: Isotr0py <2037008807@qq.com> Signed-off-by: Roger Wang <hey@rogerw.io>

update

165a58c

Signed-off-by: Roger Wang <hey@rogerw.io>

Merge branch 'main' into fix/preserve-tool-call-id

1953219

cursor Bot reviewed Jan 26, 2026

View reviewed changes

ywang96 and others added 3 commits January 26, 2026 16:22

fix mock

94b3230

Signed-off-by: Roger Wang <hey@rogerw.io>

Update vllm/entrypoints/openai/responses/serving.py

7cc9c9a

Co-authored-by: Isotr0py <2037008807@qq.com> Signed-off-by: Roger Wang <hey@rogerw.io>

Merge branch 'main' into fix/preserve-tool-call-id

7a60b0e

cursor Bot reviewed Jan 27, 2026

View reviewed changes

youkaichao merged commit 2d70534 into vllm-project:main Jan 27, 2026
46 of 49 checks passed

wangln19 mentioned this pull request Jan 27, 2026

Fix tool call indexing double-counting #33141

Merged

bbrowning mentioned this pull request Mar 26, 2026

[Misc] Add 20 regression tests for 11 tool parser bug fixes #38172

Merged

Uh oh!

Conversation

wangln19 commented Jan 21, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

wangln19 commented Jan 21, 2026

Uh oh!

Uh oh!

mergify Bot commented Jan 21, 2026

Uh oh!

mergify Bot commented Jan 21, 2026

Uh oh!

mergify Bot commented Jan 21, 2026

Uh oh!

wangln19 commented Jan 21, 2026

Uh oh!

daniel-salib left a comment

Choose a reason for hiding this comment

Uh oh!

daniel-salib Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

daniel-salib Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

wangln19 Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

mergify Bot commented Jan 21, 2026

Uh oh!

qandrew left a comment

Choose a reason for hiding this comment

Uh oh!

wangln19 commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify Bot commented Jan 22, 2026

Uh oh!

mergify Bot commented Jan 22, 2026

Uh oh!

mergify Bot commented Jan 22, 2026

Uh oh!

mergify Bot commented Jan 25, 2026

Uh oh!

mergify Bot commented Jan 26, 2026

Uh oh!

ywang96 commented Jan 26, 2026

Uh oh!

Uh oh!

cursor Bot Jan 26, 2026

Choose a reason for hiding this comment

Double-counting causes non-sequential tool call indices

Uh oh!

ywang96 commented Jan 27, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

wangln19 commented Jan 21, 2026 •

edited by github-actions Bot

Loading

wangln19 commented Jan 22, 2026 •

edited

Loading