Skip to content

[Model Runner V2] Fix kv_connector pre_forward order#42676

Merged
njhill merged 1 commit into
mainfrom
wentao-fix-mrv2-kvconnector-preforward
May 15, 2026
Merged

[Model Runner V2] Fix kv_connector pre_forward order#42676
njhill merged 1 commit into
mainfrom
wentao-fix-mrv2-kvconnector-preforward

Conversation

@yewentao256

@yewentao256 yewentao256 commented May 14, 2026

Copy link
Copy Markdown
Member

Purpose

Part of #41286

VLLM_USE_V2_MODEL_RUNNER=1 pytest tests/v1/kv_connector/unit/test_multi_connector.py::test_multi_example_connector_consistency

Originally

        # First three events are from initialization (register_kv_caches,
        # set_host_xfer_buffer_ops, get_handshake_metadata), then generate() events.
>       assert events["storage1-WORKER"][:8] == [
            "register_kv_caches",
            "set_host_xfer_buffer_ops",
            "get_handshake_metadata",
            "handle_preemptions",
            "bind_connector_metadata",
            "start_load_kv",
            "wait_for_layer_load",
            "save_kv_layer",
        ]
E       AssertionError: assert ['register_kv...load_kv', ...] == ['register_kv...load_kv', ...]
E         
E         At index 3 diff: 'bind_connector_metadata' != 'handle_preemptions'
E         Use -v to get more diff

Now

======================================== 1 passed, 17 warnings in 15.16s =========================================

@yewentao256
fix mrv2 kvconnector preforward
6c3ba6a 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
@yewentao256 yewentao256 requested a review from orozery as a code owner May 14, 2026 20:01
@yewentao256 yewentao256 added the ready ONLY add when PR is ready to merge/full CI is needed label May 14, 2026
claude[bot]
claude Bot reviewed May 14, 2026
View reviewed changes

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@yewentao256 yewentao256 mentioned this pull request May 14, 2026
Open
30 tasks
@mergify mergify Bot added the v1 label May 14, 2026
gemini-code-assist[bot]
gemini-code-assist Bot reviewed May 14, 2026
View reviewed changes

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request reorders the method calls in the pre_forward function of the KVConnector class, ensuring that handle_preemptions is executed before bind_connector_metadata. As there were no review comments provided, I have no feedback to provide.

@njhill njhill enabled auto-merge (squash) May 14, 2026 21:10
@njhill njhill merged commit af9616d into main May 15, 2026
72 of 74 checks passed
@njhill njhill deleted the wentao-fix-mrv2-kvconnector-preforward branch May 15, 2026 15:14
omerpaz95 pushed a commit to omerpaz95/vllm that referenced this pull request May 18, 2026
@yewentao256 @omerpaz95
[Model Runner V2] Fix kv_connector pre_forward order (vllm-project#…
3051674 
…42676)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
omerpaz95 pushed a commit to omerpaz95/vllm that referenced this pull request May 18, 2026
@yewentao256 @omerpaz95
[Model Runner V2] Fix kv_connector pre_forward order (vllm-project#…
7d8caa8 
…42676)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
mfylcek pushed a commit to mfylcek/vllm that referenced this pull request May 19, 2026
@yewentao256 @mfylcek
[Model Runner V2] Fix kv_connector pre_forward order (vllm-project#…
add9e5e 
…42676)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
jhu960213 pushed a commit to jhu960213/vllm that referenced this pull request May 20, 2026
@yewentao256
[Model Runner V2] Fix kv_connector pre_forward order (vllm-project#…
6e7180c 
…42676)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
@njhill njhill added the v2 label May 20, 2026
h1t35h pushed a commit to h1t35h/vllm that referenced this pull request May 21, 2026
@yewentao256 @h1t35h
[Model Runner V2] Fix kv_connector pre_forward order (vllm-project#…
3c87a00 
…42676)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Liuweixiong0118 pushed a commit to Liuweixiong0118/vllm that referenced this pull request Jun 1, 2026
@yewentao256 @Liuweixiong0118
[Model Runner V2] Fix kv_connector pre_forward order (vllm-project#…
6466838 
…42676)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Liuweixiong0118 <lwx34158427@gmail.com>
mvanhorn pushed a commit to mvanhorn/vllm that referenced this pull request Jun 4, 2026
@yewentao256 @mvanhorn
[Model Runner V2] Fix kv_connector pre_forward order (vllm-project#…
418e524 
…42676)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
andakai pushed a commit to andakai/vllm that referenced this pull request Jun 4, 2026
@yewentao256
[Model Runner V2] Fix kv_connector pre_forward order (vllm-project#…
04baa45 
…42676)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
knight0528 pushed a commit to knight0528/vllm that referenced this pull request Jun 8, 2026
@yewentao256 @knight0528
[Model Runner V2] Fix kv_connector pre_forward order (vllm-project#…
31ebf8c 
…42676)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments

@njhill njhill njhill approved these changes

@orozery orozery Awaiting requested review from orozery orozery is a code owner

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

ready ONLY add when PR is ready to merge/full CI is needed v1 v2

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@yewentao256 @njhill