Remove vllm deps from lookup client/server by maobaolong · Pull Request #2376 · LMCache/LMCache

maobaolong · 2026-01-09T07:17:09Z

What this PR does / why we need it:

Special notes for your reviewers:

If applicable:

this PR contains user facing changes - docs added
this PR contains unit tests

gemini-code-assist · 2026-01-09T07:17:39Z

Summary of Changes

Hello @maobaolong, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request aims to enhance the modularity and reduce the coupling of the LMCache lookup client and server with the vLLM framework. By extracting vLLM-specific configuration parameters into the LMCacheEngineMetadata object, the lookup components can now operate with a more generalized set of metadata, making them more independent and potentially reusable in different contexts. This change primarily involves updating function signatures and parameter passing across several files related to lookup client/server creation and RPC utility functions.

Highlights

Decoupling from vLLM Config: The lookup client and server components no longer directly depend on vllm_config, promoting greater modularity.
Centralized Metadata for RPC: Essential configuration details (like engine_id, num_ranks, kv_connector_extra_config) are now encapsulated and passed via the LMCacheEngineMetadata object.
Refactored RPC Path Generation: The get_zmq_rpc_path_lmcache utility function has been updated to accept engine_id directly, further reducing vLLM specific dependencies and improving clarity.
Cleanup of Type Hinting: Unnecessary TYPE_CHECKING imports related to vllm.config have been removed from several files, simplifying the codebase and reducing potential import issues.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request effectively decouples the lookup client and server from vllm dependencies by introducing new fields in LMCacheEngineMetadata and passing this object instead of the vllm_config. This is a great refactoring that improves modularity and maintainability. The changes are applied consistently across all relevant files. I also noticed and appreciate that a bug in LMCacheAsyncLookupServer.close() has been fixed and logging statements have been updated to use %-style formatting, which is a good practice for performance. I've found one minor issue with a misleading error message and have left a comment with a suggestion to fix it. Overall, this is a high-quality contribution.

Signed-off-by: baoloongmao <baoloongmao@tencent.com>

maobaolong · 2026-01-11T06:04:50Z

@sammshen Would you like to take a look at this PR? Thanks!

sammshen · 2026-01-11T08:51:45Z

    head_size = model_cfg.get_head_size()
    kv_shape = (num_layer, 1 if use_mla else 2, chunk_size, num_kv_head, head_size)

+    # Extract engine_id from vllm_config if available


what cases is the engine_id unavailable?

engine_id is introduced by vllm-project/vllm#17751 , before this PR, there is no engine_id within KVTransferConfig

sammshen

LGTM! This is great!

Signed-off-by: baoloongmao <baoloongmao@tencent.com>

chunxiaozheng

lgtm

* Remove vllm deps from lookup client/server Signed-off-by: baoloongmao <baoloongmao@tencent.com> * fix Signed-off-by: baoloongmao <baoloongmao@tencent.com> * fix Signed-off-by: baoloongmao <baoloongmao@tencent.com> * fix Signed-off-by: baoloongmao <baoloongmao@tencent.com> --------- Signed-off-by: baoloongmao <baoloongmao@tencent.com>

* Remove vllm deps from lookup client/server Signed-off-by: baoloongmao <baoloongmao@tencent.com> * fix Signed-off-by: baoloongmao <baoloongmao@tencent.com> * fix Signed-off-by: baoloongmao <baoloongmao@tencent.com> * fix Signed-off-by: baoloongmao <baoloongmao@tencent.com> --------- Signed-off-by: baoloongmao <baoloongmao@tencent.com> Signed-off-by: shaoxiawjc <wjc2800@163.com>

gemini-code-assist Bot reviewed Jan 9, 2026

View reviewed changes

Comment thread lmcache/v1/rpc_utils.py

sammshen mentioned this pull request Jan 11, 2026

[RFC]: northbound clean up of adapter and LMCacheManager #2384

Closed

Remove vllm deps from lookup client/server

ca94d41

Signed-off-by: baoloongmao <baoloongmao@tencent.com>

maobaolong force-pushed the removeVllmFromLookupClient branch from ba54e88 to ca94d41 Compare January 11, 2026 05:47

fix

9a6095d

Signed-off-by: baoloongmao <baoloongmao@tencent.com>

sammshen reviewed Jan 11, 2026

View reviewed changes

sammshen approved these changes Jan 11, 2026

View reviewed changes

maobaolong added 2 commits January 11, 2026 20:16

fix

abb4ee2

Signed-off-by: baoloongmao <baoloongmao@tencent.com>

fix

fecc820

Signed-off-by: baoloongmao <baoloongmao@tencent.com>

maobaolong added the full Run comprehensive tests on this PR label Jan 12, 2026

chunxiaozheng approved these changes Jan 12, 2026

View reviewed changes

chunxiaozheng merged commit ff2b40e into LMCache:dev Jan 12, 2026
25 of 26 checks passed

maobaolong mentioned this pull request Mar 12, 2026

fix(storage_manager): re-raise exception in read_prefetched_results #2737

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove vllm deps from lookup client/server#2376

Remove vllm deps from lookup client/server#2376
chunxiaozheng merged 4 commits intoLMCache:devfrom
maobaolong:removeVllmFromLookupClient

maobaolong commented Jan 9, 2026

Uh oh!

gemini-code-assist Bot commented Jan 9, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

maobaolong commented Jan 11, 2026

Uh oh!

sammshen Jan 11, 2026

Uh oh!

maobaolong Jan 11, 2026

Uh oh!

sammshen left a comment

Uh oh!

chunxiaozheng left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

maobaolong commented Jan 9, 2026

Uh oh!

gemini-code-assist Bot commented Jan 9, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

maobaolong commented Jan 11, 2026

Uh oh!

sammshen Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

maobaolong Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

sammshen left a comment

Choose a reason for hiding this comment

Uh oh!

chunxiaozheng left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants