[CLI] Implement initial framework of LMCache CLI by KuntaiDu · Pull Request #2775 · LMCache/LMCache

KuntaiDu · 2026-03-14T08:06:57Z

What this PR does / why we need it:

This PR implements LMCache CLI described in #2748 .

Summary

Introduces the lmcache CLI framework with a pluggable subcommand architecture using BaseCommand ABC and
explicit registration via ALL_COMMANDS in commands/init.py
Adds a hierarchical Metrics system that separates machine keys (for JSON) from human-readable labels
(for terminal), with pluggable rendering styles (defaulting to vLLM-style output), configurable via
LMCACHE_CLI_METRICS_STYLE env var through the centralized config system
Includes lmcache mock as a reference command demonstrating the full framework

Example usage

  # Terminal output (human-readable labels)
  $ lmcache mock --name test-run --num-items 5

  ============= Mock Result ==============
  ----------- Input Parameters -----------
  Name:                           test-run
  Num items:                             5
  ------------- Mock Metrics -------------
  Items processed:                      42
  Total time (ms):                   12.34
  Throughput (items/s):            3403.73
  -------------- Validation --------------
  Status:                               OK
  ========================================

  # JSON output (machine keys)
  $ lmcache mock --name test-run --num-items 5 --output result.json

How to add a new subcommand

Create lmcache/cli/commands/my_cmd.py with a BaseCommand subclass
Add one import + one entry to ALL_COMMANDS in commands/init.py

Special notes for your reviewers:

If applicable:

this PR contains user facing changes - docs added
this PR contains unit tests

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

gemini-code-assist · 2026-03-14T08:07:19Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request lays the foundational framework for the LMCache command-line interface, enabling a structured and extensible approach to managing LMCache components. It introduces a clear mechanism for defining and registering new subcommands, along with a sophisticated, yet lightweight, system for collecting and presenting operational metrics in both human-friendly and machine-parseable formats. This significantly streamlines the process of adding future CLI functionalities and provides users with consistent, informative feedback.

Highlights

New CLI Framework: Implemented a new CLI framework for LMCache, featuring an explicit command registration system using a BaseCommand abstract class.
Hierarchical Metrics System: Introduced a hierarchical metrics logging system that supports both human-readable terminal output and machine-readable JSON output, with pluggable rendering styles.
Example Mock Command: Added a 'mock' command as a concrete example to demonstrate the full capabilities of the new CLI framework and metrics system.
Centralized Configuration: Established a centralized configuration system for CLI settings, allowing for environment variable overrides.
Documentation Updates: Updated documentation to reflect the new CLI architecture, file layout, and design principles for command registration and metrics.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

docs/design/cli/commands.md
- Updated the CLI architecture description to reflect explicit command registration instead of auto-discovery.
- Revised the file layout to include new CLI components like main.py, metrics.py, config.py, and the commands subdirectory structure.
- Adjusted the CLI development phases, introducing a "Phase 0" for the framework and mock command.
docs/design/cli/framework-and-metrics.md
- Added a new design document detailing the explicit command registration mechanism and the hierarchical metrics logging system.
- Provided examples for adding new subcommands and using the Metrics API.
- Described the lmcache mock example command and shared CLI conventions like --output and error handling.
lmcache/cli/init.py
- Added the init.py file to establish lmcache/cli as a Python package.
lmcache/cli/commands/init.py
- Added the init.py file for the lmcache.cli.commands package, defining ALL_COMMANDS for explicit registration and importing BaseCommand and MockCommand.
lmcache/cli/commands/base.py
- Added the BaseCommand abstract base class, which all CLI subcommands must inherit from, enforcing name, help, add_arguments, and handler methods.
- Included a helper function add_output_arg for consistent --output flag implementation across commands.
lmcache/cli/commands/mock.py
- Added the MockCommand implementation, serving as a functional example for new CLI commands, demonstrating argument parsing, metrics collection, and output generation.
lmcache/cli/config.py
- Added a new module for centralized CLI configuration, allowing settings like metrics_style to be configured via environment variables.
lmcache/cli/main.py
- Added the main entry point for the lmcache CLI, responsible for parsing arguments, registering commands from ALL_COMMANDS, and dispatching to the appropriate command handler.
lmcache/cli/metrics.py
- Added the Metrics class, a hierarchical metrics collector supporting sectioned and flat metrics, with pluggable terminal rendering styles (e.g., vllm style) and JSON export capabilities.
pyproject.toml
- Added lmcache="lmcache.cli.main:main" to [project.scripts], making the new CLI executable.

Activity

No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a well-designed and robust framework for the LMCache CLI. The move to explicit command registration via BaseCommand improves maintainability, and the hierarchical Metrics system provides a flexible way to handle both human-readable terminal output and machine-readable JSON output. The new lmcache mock command serves as a great reference for adding new subcommands. The overall implementation is clean and well-documented. I have one suggestion to improve the consistency of the mock command's output.

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

…ior aligned Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

sammshen

LGTM!

ApostaC

Otherwise LGTM!

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

* initial commit --- implemented the CLI system Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * add sphinx doc Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * make linting happy Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * fix linter and clarify that url must be http Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * add formatter-handler separation, and make handler registration behavior aligned Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * update design doc Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * fix comments from Yihua Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * make sure that --format and --output are independent Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * clean up logic on getting the handler Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * avoid letting people add output args by themselves Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * use logging.exception Signed-off-by: KuntaiDu <kuntai@uchicago.edu> --------- Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

* initial commit --- implemented the CLI system Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * add sphinx doc Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * make linting happy Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * fix linter and clarify that url must be http Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * add formatter-handler separation, and make handler registration behavior aligned Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * update design doc Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * fix comments from Yihua Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * make sure that --format and --output are independent Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * clean up logic on getting the handler Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * avoid letting people add output args by themselves Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * use logging.exception Signed-off-by: KuntaiDu <kuntai@uchicago.edu> --------- Signed-off-by: KuntaiDu <kuntai@uchicago.edu> Signed-off-by: Aaron Wu <aaron.wu@dell.com>

* initial commit --- implemented the CLI system Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * add sphinx doc Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * make linting happy Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * fix linter and clarify that url must be http Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * add formatter-handler separation, and make handler registration behavior aligned Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * update design doc Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * fix comments from Yihua Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * make sure that --format and --output are independent Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * clean up logic on getting the handler Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * avoid letting people add output args by themselves Signed-off-by: KuntaiDu <kuntai@uchicago.edu> * use logging.exception Signed-off-by: KuntaiDu <kuntai@uchicago.edu> --------- Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

initial commit --- implemented the CLI system

9285a8a

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

gemini-code-assist Bot reviewed Mar 14, 2026

View reviewed changes

Comment thread lmcache/cli/commands/mock.py

KuntaiDu added 6 commits March 14, 2026 08:09

add sphinx doc

03b6c56

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

make linting happy

e11b269

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

fix linter and clarify that url must be http

a07d4d5

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

Merge branch 'dev' into kuntai-cli-initial

0304594

add formatter-handler separation, and make handler registration behav…

a59021e

…ior aligned Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

update design doc

0635eba

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

ApostaC added the full Run comprehensive tests on this PR label Mar 17, 2026

sammshen approved these changes Mar 17, 2026

View reviewed changes

ApostaC approved these changes Mar 17, 2026

View reviewed changes

KuntaiDu added 5 commits March 17, 2026 23:43

fix comments from Yihua

6274ad6

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

make sure that --format and --output are independent

1658ae2

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

clean up logic on getting the handler

b3ba86d

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

avoid letting people add output args by themselves

491c2de

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

use logging.exception

d53477f

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

KuntaiDu enabled auto-merge (squash) March 18, 2026 01:55

Merge branch 'dev' into kuntai-cli-initial

8eb5ac7

KuntaiDu merged commit 76ae806 into LMCache:dev Mar 18, 2026
23 of 28 checks passed

KuntaiDu deleted the kuntai-cli-initial branch March 18, 2026 20:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CLI] Implement initial framework of LMCache CLI#2775

[CLI] Implement initial framework of LMCache CLI#2775
KuntaiDu merged 13 commits intoLMCache:devfrom
KuntaiDu:kuntai-cli-initial

KuntaiDu commented Mar 14, 2026

Uh oh!

gemini-code-assist Bot commented Mar 14, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

sammshen left a comment

Uh oh!

ApostaC left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

KuntaiDu commented Mar 14, 2026

Uh oh!

gemini-code-assist Bot commented Mar 14, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

sammshen left a comment

Choose a reason for hiding this comment

Uh oh!

ApostaC left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants