[CLI]: Server command by sammshen · Pull Request #2836 · LMCache/LMCache

sammshen · 2026-03-20T22:39:42Z

lmcache server wraps python -m lmcache.v1.multiprocess.http_server

Note

Low Risk
Adds a new CLI entrypoint and documentation updates without modifying underlying server/runtime logic; main risk is mismatched/incorrect CLI wiring or defaults causing launch failures.

Overview
Adds a new lmcache server subcommand that composes existing MP server/storage/HTTP frontend/Prometheus/telemetry argparse helpers and calls run_http_server() with configs parsed from CLI args.

Registers the new command in the CLI command registry, adds unit tests verifying argument registration/defaults and that execute() invokes run_http_server, and updates CLI/MP docs to reference lmcache server instead of python -m lmcache.v1.multiprocess.http_server (including quickstart/deployment/config examples and command listings).

^{Written by Cursor Bugbot for commit 5c72a69. This will update automatically on new commits. Configure here.}

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

gemini-code-assist · 2026-03-20T22:39:56Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates the LMCache server launch functionality directly into the command-line interface. By introducing a dedicated lmcache server command, users can now easily start the LMCache server with various configuration options, replacing the previous method of directly invoking a Python module. This enhancement simplifies the user experience for deploying and managing the LMCache server.

Highlights

CLI Server Command: A new lmcache server command has been introduced to the CLI, enabling direct launch of the LMCache server.
Server Wrapper: The new server command acts as a wrapper for python -m lmcache.v1.multiprocess.http_server, streamlining server deployment.
Documentation Updates: Design and user documentation have been updated to reflect the addition and completion of the server command.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a new server command to the lmcache CLI, providing a user-friendly way to launch the LMCache server. The implementation correctly wraps the existing server logic. My review focuses on adherence to the project's style guide. The main points are the need for tests for this new feature and the addition of docstrings for public methods in the new command class, both of which are required by the style guide.

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

cursor · 2026-03-21T07:01:38Z

+            storage_manager_config=parse_args_to_config(args),
+            prometheus_config=parse_args_to_prometheus_config(args),
+            telemetry_config=parse_args_to_telemetry_config(args),
+        )


Server command inherits unused metrics output flags

Low Severity

ServerCommand extends BaseCommand, whose register() unconditionally calls _add_output_args(parser), adding --format and --output flags. However, execute() never calls create_metrics() and never reads these attributes — it only delegates to run_http_server. These dead flags will appear in lmcache server -h, misleading users into thinking they have an effect on a long-running server process.

^{Triggered by project rule: LMCache Code Review Style Guide}

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

KuntaiDu · 2026-03-23T07:47:20Z

   lmcache mock --name my-run --num-items 5

+   # Launch the LMCache server (ZMQ + HTTP)
+   lmcache server --engine-type blend --host 0.0.0.0 --port 5555


Why we have --engine-type blend here?

KuntaiDu

Otherwise lgtm

Removed engine-type option from server command.

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

…p-server-cli

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

* initial commit Signed-off-by: Samuel Shen <slshen@tensormesh.ai> * fix imports Signed-off-by: Samuel Shen <slshen@tensormesh.ai> * add unit tests and doc strings Signed-off-by: Samuel Shen <slshen@tensormesh.ai> * update documentation to use lmcache server Signed-off-by: Samuel Shen <slshen@tensormesh.ai> * fix non CUDA UT Signed-off-by: Samuel Shen <slshen@tensormesh.ai> * fix UT stub Signed-off-by: Samuel Shen <slshen@tensormesh.ai> * Update CLI command by removing engine-type option Removed engine-type option from server command. --------- Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

sammshen added 2 commits March 20, 2026 22:36

initial commit

9c11a1f

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

fix imports

7fb8a1a

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

cursor Bot reviewed Mar 20, 2026

View reviewed changes

Comment thread lmcache/cli/commands/server.py

Comment thread lmcache/cli/commands/server.py

gemini-code-assist Bot reviewed Mar 20, 2026

View reviewed changes

Comment thread lmcache/cli/commands/server.py

Comment thread lmcache/cli/commands/server.py

deng451e self-requested a review March 21, 2026 01:01

deng451e approved these changes Mar 21, 2026

View reviewed changes

sammshen added 2 commits March 21, 2026 06:53

add unit tests and doc strings

6159ac5

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

update documentation to use lmcache server

4d07674

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

cursor Bot reviewed Mar 21, 2026

View reviewed changes

sammshen added 3 commits March 21, 2026 07:41

fix non CUDA UT

0c4bbee

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

fix UT stub

95bc567

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

Merge branch 'dev' into mp-server-cli

3e7f102

KuntaiDu reviewed Mar 23, 2026

View reviewed changes

KuntaiDu approved these changes Mar 23, 2026

View reviewed changes

Update CLI command by removing engine-type option

40c7da4

Removed engine-type option from server command.

sammshen enabled auto-merge (squash) March 23, 2026 17:56

github-actions Bot added the full Run comprehensive tests on this PR label Mar 23, 2026

sammshen added the integration_testing Run the functional test for container image label Mar 23, 2026

sammshen added 2 commits March 24, 2026 12:09

Merge branch 'dev' into mp-server-cli

c4d0b39

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

Fix import order for ServerCommand

533a1e6

cursor Bot reviewed Mar 24, 2026

View reviewed changes

Comment thread tests/cli/commands/test_server.py Outdated

sammshen added 3 commits March 25, 2026 07:09

fix UT

5a75acc

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

Merge branch 'mp-server-cli' of github-samuel:sammshen/LMCache into m…

e700def

…p-server-cli

rebase again

5c72a69

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

cursor Bot reviewed Mar 25, 2026

View reviewed changes

Comment thread docs/source/getting_started/cli.rst Outdated

fix docs

fb38c8e

Signed-off-by: Samuel Shen <slshen@tensormesh.ai>

sammshen merged commit 61b8ba5 into LMCache:dev Mar 25, 2026
35 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CLI]: Server command#2836

[CLI]: Server command#2836
sammshen merged 14 commits intoLMCache:devfrom
sammshen:mp-server-cli

sammshen commented Mar 20, 2026 •

edited by cursor Bot

Loading

Uh oh!

gemini-code-assist Bot commented Mar 20, 2026

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

cursor Bot Mar 21, 2026

Uh oh!

KuntaiDu Mar 23, 2026

Uh oh!

sammshen Mar 23, 2026

Uh oh!

KuntaiDu left a comment

Uh oh!

Uh oh!

cursor Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

sammshen commented Mar 20, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot commented Mar 20, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

cursor Bot Mar 21, 2026

Choose a reason for hiding this comment

Server command inherits unused metrics output flags

Uh oh!

KuntaiDu Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

sammshen Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

KuntaiDu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sammshen commented Mar 20, 2026 •

edited by cursor Bot

Loading