Skip to content

[Tool Parser][1/3] Pass tools to ToolParser constructor#38029

Merged
robertgshaw2-redhat merged 4 commits into
vllm-project:mainfrom
sfeng33:tool_init
Mar 26, 2026
Merged

[Tool Parser][1/3] Pass tools to ToolParser constructor#38029
robertgshaw2-redhat merged 4 commits into
vllm-project:mainfrom
sfeng33:tool_init

Conversation

@sfeng33

@sfeng33 sfeng33 commented Mar 24, 2026

Copy link
Copy Markdown
Collaborator

Purpose

This is the first PR toward decoupling tool parsers from the API request object, which is necessary to reuse tool parsing logic in the unified output parser (#32713).

Today, tool parsers receive tool definitions only at extraction time via the request parameter in extract_tool_calls() /
extract_tool_calls_streaming(). This tight coupling to the request object prevents the parsers from being composed into a unified parser that handles reasoning, tool calls, and text output together — the unified parser shouldn't need to know about API request internals.

This is PR 1 of 3:

  • PR 1 (this): Add tools to ToolParser.init and pass it from all call sites
  • PR 2: Update parsers to use self.tools instead of request.tools
  • PR 3: Remove request from extract_tool_calls / extract_tool_calls_streaming signatures

No behavioral change in this PR — purely additive plumbing.

Test Plan

pytest tests/tool_parsers/ -v

Signed-off-by: sfeng33 <4florafeng@gmail.com>
@mergify

mergify Bot commented Mar 24, 2026

Copy link
Copy Markdown
Contributor

Hi @sfeng33, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

Signed-off-by: sfeng33 <4florafeng@gmail.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

@sfeng33

sfeng33 commented Mar 24, 2026

Copy link
Copy Markdown
Collaborator Author

/gemini review

@sfeng33

sfeng33 commented Mar 24, 2026

Copy link
Copy Markdown
Collaborator Author

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the ToolParser initialization by introducing a tools argument to the ToolParser constructor and its subclasses. This change allows the tool parsing logic to be aware of the specific tools provided in the request, updating constructor calls and type hints across various tool parser implementations and related serving files. The review suggests an improvement to enhance encapsulation by initializing the StreamingXMLToolCallParser with the tools argument directly in its constructor for better configuration and future simplification.

Comment thread vllm/tool_parsers/qwen3xml_tool_parser.py
Comment thread vllm/tool_parsers/step3p5_tool_parser.py
@mergify

mergify Bot commented Mar 24, 2026

Copy link
Copy Markdown
Contributor

Hi @sfeng33, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

Comment thread vllm/tool_parsers/abstract_tool_parser.py
Signed-off-by: sfeng33 <4florafeng@gmail.com>
@mergify

mergify Bot commented Mar 24, 2026

Copy link
Copy Markdown
Contributor

Hi @sfeng33, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

Signed-off-by: sfeng33 <4florafeng@gmail.com>

@bbrowning bbrowning left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we got all the parsers updated. All unit tests pass for me locally, and I also ran integration tests under tests/entrypoints/openai/tool_parsers and tests/tool_use and all of those that I could run on my hardware passed. The ones I couldn't run are just related to my resources available, and not to any change here.

So, this looks good to me! It sets us up for the next refactoring but as far as I can tell is completely transparent to the end users.

As you move on to phase 2 of this, another thing I realized to keep in mind is we have some external tool parser plugins. This new parameter is optional in the constructor, so that part should be fine. We'll just need to be mindful about whether we want to break or keep semantics working for external tool parsers as all this gets wired through. Until or unless those external tool parser plugins get updated, we'll have to always handle the case where our list of tools is None defensively.

@sfeng33

sfeng33 commented Mar 25, 2026

Copy link
Copy Markdown
Collaborator Author

It looks like we got all the parsers updated. All unit tests pass for me locally, and I also ran integration tests under tests/entrypoints/openai/tool_parsers and tests/tool_use and all of those that I could run on my hardware passed. The ones I couldn't run are just related to my resources available, and not to any change here.

So, this looks good to me! It sets us up for the next refactoring but as far as I can tell is completely transparent to the end users.

As you move on to phase 2 of this, another thing I realized to keep in mind is we have some external tool parser plugins. This new parameter is optional in the constructor, so that part should be fine. We'll just need to be mindful about whether we want to break or keep semantics working for external tool parsers as all this gets wired through. Until or unless those external tool parser plugins get updated, we'll have to always handle the case where our list of tools is None defensively.

Thanks for the thorough review and integration testing, Ben! Great point about external tool parser plugins, I'll make sure to handle tools=None defensively in PR 2 so we don't break any out-of-tree parsers.

@robertgshaw2-redhat robertgshaw2-redhat enabled auto-merge (squash) March 25, 2026 20:56
@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 25, 2026
Comment thread vllm/entrypoints/openai/engine/serving.py

@chaunceyjiang chaunceyjiang left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks~

@robertgshaw2-redhat robertgshaw2-redhat merged commit e2db2b4 into vllm-project:main Mar 26, 2026
50 checks passed
@sfeng33 sfeng33 deleted the tool_init branch March 26, 2026 02:30
RhizoNymph pushed a commit to RhizoNymph/vllm that referenced this pull request Mar 26, 2026
nithinvc pushed a commit to nithinvc/vllm that referenced this pull request Mar 27, 2026
…#38029)

Signed-off-by: sfeng33 <4florafeng@gmail.com>

Signed-off-by: Nithin Chalapathi <nithin.ch10@gmail.com>
JiantaoXu pushed a commit to JiantaoXu/vllm that referenced this pull request Mar 28, 2026
hospedales added a commit to hospedales/vllm that referenced this pull request Apr 2, 2026
The ToolParser base class now accepts (tokenizer, tools) from PR vllm-project#38029,
but Gemma4ToolParser only accepted (tokenizer). This caused a 400 error
when tool calls were attempted.

Signed-off-by: Michael Hospedales <hospedales@me.com>
puririshi98 pushed a commit to puririshi98/vllm that referenced this pull request Apr 7, 2026
…#38029)

Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Rishi Puri <riship@nvidia.com>
mtparet pushed a commit to blackfuel-ai/vllm that referenced this pull request Apr 9, 2026
mystous pushed a commit to mystous/vllm_hybrid that referenced this pull request May 10, 2026
my-other-github-account pushed a commit to my-other-github-account/vllm that referenced this pull request May 15, 2026
my-other-github-account pushed a commit to my-other-github-account/vllm that referenced this pull request May 15, 2026
mvanhorn pushed a commit to mvanhorn/vllm that referenced this pull request Jun 4, 2026
…#38029)

Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

frontend ready ONLY add when PR is ready to merge/full CI is needed tool-calling

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants