[MP] Lazy start heartbeat thread when first req coming#2943
Merged
maobaolong merged 2 commits intoLMCache:devfrom Apr 11, 2026
Merged
[MP] Lazy start heartbeat thread when first req coming#2943maobaolong merged 2 commits intoLMCache:devfrom
maobaolong merged 2 commits intoLMCache:devfrom
Conversation
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Contributor
There was a problem hiding this comment.
Code Review
This pull request implements lazy initialization for the heartbeat thread in the VLLMMultiProcessAdapter, moving its start from the registration phase to the first store or retrieve request. A review comment identifies an inaccuracy in the class constructor's documentation regarding which method triggers the heartbeat, providing a suggestion to align the comment with the actual implementation.
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Collaborator
Author
|
@cursor review |
Collaborator
Author
|
@sammshen @chunxiaozheng Would you like to take a look at this ? I've tested locally. |
maobaolong
added a commit
to maobaolong/LMCache
that referenced
this pull request
Apr 8, 2026
…che#2943 Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Oasis-Git
pushed a commit
to Oasis-Git/LMCache
that referenced
this pull request
Apr 13, 2026
* [MP] Lazy start heartbeat thread when first req coming Signed-off-by: baoloongmao <baoloongmao@tencent.com> * Update comment Signed-off-by: baoloongmao <baoloongmao@tencent.com> --------- Signed-off-by: baoloongmao <baoloongmao@tencent.com>
ftian1
pushed a commit
to ftian1/LMCache
that referenced
this pull request
Apr 20, 2026
* [MP] Lazy start heartbeat thread when first req coming Signed-off-by: baoloongmao <baoloongmao@tencent.com> * Update comment Signed-off-by: baoloongmao <baoloongmao@tencent.com> --------- Signed-off-by: baoloongmao <baoloongmao@tencent.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
I'm sorry #2798 is not good enough, the current PR make it lazy start heartbeat thread when first req coming.
Special notes for your reviewers:
If applicable:
Note
Medium Risk
Changes the timing of heartbeat thread startup in the vLLM multiprocess adapter, which can affect health detection and degraded-mode behavior during startup and early requests. Concurrency/thread-lifecycle adjustments may introduce subtle race or ordering issues.
Overview
Defers starting the LMCache
HeartbeatThreadinLMCacheMPWorkerAdapteruntil the firstsubmit_store_requestorsubmit_retrieve_request, instead of starting it immediately afterregister_kv_cachescompletes.Replaces the eager
_start_heartbeatcall with an idempotent_ensure_heartbeat_startedhelper and updates comments to clarify the new “start on first use” behavior once vLLM initialization (model load, KV allocation, warmup/CUDA graph capture) is complete.Written by Cursor Bugbot for commit 2775a48. This will update automatically on new commits. Configure here.