[Chore][CI] Force vLLM Model Runner V1 in the PD comprehensive test#3502
Merged
Merged
Conversation
The PD comprehensive test (k3-comprehensive-test) started timing out on 2026-06-02 when the floating vLLM nightly bumped from 0.22.1rc1.dev91 to .dev95. That delta includes vllm-project/vllm#43458, which enables Model Runner V2 (MRV2) by default for Llama/Mistral dense models -- including meta-llama/Llama-3.2-1B-Instruct, the model used by pd.yaml. MRV2 is incompatible with KV cache connectors, so the prefiller/decoder (both driven through LMCacheConnectorV1) hang on the first request and the job is killed by the 30-minute timeout. The vLLM servers start fine; only the disaggregated request path hangs. Pin the PD prefiller and decoder to MRV1 via VLLM_USE_V2_MODEL_RUNNER=0 until MRV2 supports KV connectors / reliably falls back to MRV1. Signed-off-by: Samuel Shen <slshen@tensormesh.ai>
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the Buildkite configuration in .buildkite/configs/pd.yaml to force the use of vLLM Model Runner V1 (VLLM_USE_V2_MODEL_RUNNER=0) for both the docker-prefiller and docker-decoder environments. This change prevents hangs caused by incompatibilities between Model Runner V2 and KV cache connectors. No review comments were provided, and the changes are clear and well-documented.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Lyj1007
pushed a commit
to Lyj1007/LMCache
that referenced
this pull request
Jun 3, 2026
…MCache#3502) [CI/CD] Force vLLM Model Runner V1 in the PD comprehensive test The PD comprehensive test (k3-comprehensive-test) started timing out on 2026-06-02 when the floating vLLM nightly bumped from 0.22.1rc1.dev91 to .dev95. That delta includes vllm-project/vllm#43458, which enables Model Runner V2 (MRV2) by default for Llama/Mistral dense models -- including meta-llama/Llama-3.2-1B-Instruct, the model used by pd.yaml. MRV2 is incompatible with KV cache connectors, so the prefiller/decoder (both driven through LMCacheConnectorV1) hang on the first request and the job is killed by the 30-minute timeout. The vLLM servers start fine; only the disaggregated request path hangs. Pin the PD prefiller and decoder to MRV1 via VLLM_USE_V2_MODEL_RUNNER=0 until MRV2 supports KV connectors / reliably falls back to MRV1. Signed-off-by: Samuel Shen <slshen@tensormesh.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The PD comprehensive test (k3-comprehensive-test) started timing out on 2026-06-02 when the floating vLLM nightly bumped from 0.22.1rc1.dev91 to .dev95. That delta includes vllm-project/vllm#43458, which enables Model Runner V2 (MRV2) by default for Llama/Mistral dense models -- including meta-llama/Llama-3.2-1B-Instruct, the model used by pd.yaml.
MRV2 is incompatible with KV cache connectors, so the prefiller/decoder (both driven through LMCacheConnectorV1) hang on the first request and the job is killed by the 30-minute timeout. The vLLM servers start fine; only the disaggregated request path hangs.
Pin the PD prefiller and decoder to MRV1 via VLLM_USE_V2_MODEL_RUNNER=0 until MRV2 supports KV connectors / reliably falls back to MRV1.
What this PR does / why we need it:
Special notes for your reviewers:
If applicable: