[Chore][Docs] Update docs for MP mode by ApostaC · Pull Request #2708 · LMCache/LMCache

ApostaC · 2026-03-06T23:48:27Z

What this PR does / why we need it:

Overhauls the multiprocess mode documentation. The old single page (kv_cache/multiprocess_mode.rst) was outdated — it didn't cover L2 storage, observability/telemetry, the HTTP server variant, hash algorithm selection, or the internal architecture. The "Future Work" section listed items that are already implemented.

This PR replaces it with a new top-level "Multiprocess Mode" section (docs/source/mp/) containing 7 pages:

index.rst — Landing page with overview, benefits, prerequisites, server variants table
quickstart.rst — Local, Docker, and HTTP server quick starts
configuration.rst — Complete CLI reference for all config groups (MP server, HTTP frontend, L1 memory/TTLs, eviction, L2 adapters with JSON examples for all NIXL backends, Prometheus, telemetry, vLLM client, env vars)
l2_storage.rst — Two-tier architecture, data flow, all adapter types and backends, cascade pattern, verification
deployment.rst — Docker, Kubernetes (DaemonSet + Deployment), health checking, monitoring, production best practices
observability.rst — Prometheus metrics (all SM/L1 counters), telemetry event system, logging
architecture.rst — Internals: server variants, ZMQ protocol (all 16 RequestTypes), config system, distributed storage (StorageManager, L1Manager, L2 adapters, controllers), request flow diagrams, extension guide

The old page is replaced with a redirect stub (:orphan:) pointing to the new section.

Special notes for your reviewers:

All CLI args and metric names were cross-referenced against the actual source files. Sphinx build succeeds with 0 new warnings.

If applicable:

this PR contains user facing changes - docs added
this PR contains unit tests

Signed-off-by: ApostaC <yihua98@uchicago.edu>

gemini-code-assist · 2026-03-06T23:48:47Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the documentation for LMCache's multiprocess mode. It replaces an outdated single page with a comprehensive, multi-page section that provides in-depth coverage of various aspects, including architecture, configuration, deployment, and observability. The update aims to provide clearer, more current, and complete guidance for users and developers interacting with the multiprocess caching system.

Highlights

Documentation Overhaul: The multiprocess mode documentation has been completely overhauled, replacing an outdated single page with a new, comprehensive top-level section.
Expanded Coverage: New documentation pages cover critical aspects such as L2 storage, observability (Prometheus, telemetry), the HTTP server variant, hash algorithm selection, and detailed internal architecture.
Structured Content: The new section is organized into 7 dedicated pages: index, quickstart, configuration, L2 storage, deployment, observability, and architecture, providing a more structured and user-friendly experience.
Accuracy and Quality: All CLI arguments and metric names were cross-referenced against source files, and the Sphinx build successfully completed with no new warnings, ensuring high documentation quality.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

docs/source/index.rst
- Updated the main documentation index to include the new multiprocess mode section and remove the old link.
docs/source/kv_cache/multiprocess_mode.rst
- Converted the old multiprocess mode documentation page into a redirect stub, pointing to the new comprehensive section.
docs/source/mp/architecture.rst
- Added a new document detailing the internal architecture, server variants, ZMQ protocol, config system, distributed storage components, request flow diagrams, and an extension guide.
docs/source/mp/configuration.rst
- Added a new document providing a comprehensive reference for all CLI arguments, grouped by config module, including MP server, HTTP frontend, L1 memory/TTLs, eviction, L2 adapters, Prometheus, telemetry, vLLM client, and environment variables.
docs/source/mp/deployment.rst
- Added a new document covering deployment strategies for Docker and Kubernetes (DaemonSet + Deployment pattern), including health checking, monitoring integration, and production best practices.
docs/source/mp/index.rst
- Added a new landing page for the multiprocess mode documentation, offering an overview, key benefits, prerequisites, and a table of server variants, along with a table of contents for the new section.
docs/source/mp/l2_storage.rst
- Added a new document explaining the two-tier storage architecture (L1/L2), data flow, various L2 adapter types (NIXL-based, mock), and how to configure multiple adapters for cascading.
docs/source/mp/observability.rst
- Added a new document detailing the observability features, including Prometheus metrics (StorageManager, L1) and the telemetry event system (event model, processors, configuration), along with logging guidance.
docs/source/mp/quickstart.rst
- Added a new document providing quick start guides for running LMCache multiprocess mode locally, in Docker, and using the HTTP server variant, with step-by-step instructions and example commands.

Activity

No human activity (comments, reviews, approvals) has been recorded for this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This is an excellent and comprehensive documentation overhaul for the multiprocess mode. The new structure is logical, and the content is detailed and clear, covering everything from quick starts to deployment and internal architecture. I found one minor inconsistency in an example configuration that could be confusing for users. Otherwise, the new documentation is a fantastic improvement.

gemini-code-assist · 2026-03-06T23:50:50Z

+
+.. code-block:: bash
+
+    kubectl apply -f examples/multi_process/vllm-deployment.yaml


There appears to be an inconsistency in the lmcache.mp.host configuration within the referenced examples/multi_process/vllm-deployment.yaml file.

In vllm-deployment.yaml, the host is specified as "tcp://${HOST_IP}". However, other examples in the documentation (like in configuration.rst) specify only the hostname or IP address, without the tcp:// prefix (e.g., "127.0.0.1").

This could be confusing for users and might lead to connection issues depending on how the vLLM client handles the connection string. For consistency across the documentation, it would be best if all examples used the same format. I'd recommend removing the tcp:// prefix from vllm-deployment.yaml to align with the other examples.

Signed-off-by: ApostaC <yihua98@uchicago.edu>

KuntaiDu

LGTM!

sammshen

LGTM!

* [Add] new docs for multi-process mode Signed-off-by: ApostaC <yihua98@uchicago.edu> * [Add] docs for mp mode Signed-off-by: ApostaC <yihua98@uchicago.edu> --------- Signed-off-by: ApostaC <yihua98@uchicago.edu> Signed-off-by: shaoxiawjc <wjc2800@163.com>

* [Add] new docs for multi-process mode Signed-off-by: ApostaC <yihua98@uchicago.edu> * [Add] docs for mp mode Signed-off-by: ApostaC <yihua98@uchicago.edu> --------- Signed-off-by: ApostaC <yihua98@uchicago.edu> Signed-off-by: Aaron Wu <aaron.wu@dell.com>

* [Add] new docs for multi-process mode Signed-off-by: ApostaC <yihua98@uchicago.edu> * [Add] docs for mp mode Signed-off-by: ApostaC <yihua98@uchicago.edu> --------- Signed-off-by: ApostaC <yihua98@uchicago.edu>

[Add] new docs for multi-process mode

8ba4cff

Signed-off-by: ApostaC <yihua98@uchicago.edu>

gemini-code-assist Bot reviewed Mar 6, 2026

View reviewed changes

ApostaC commented Mar 7, 2026

View reviewed changes

Comment thread docs/source/mp/index.rst

Comment thread docs/source/mp/index.rst Outdated

Comment thread docs/source/mp/quickstart.rst

[Add] docs for mp mode

040ccde

Signed-off-by: ApostaC <yihua98@uchicago.edu>

KuntaiDu approved these changes Mar 7, 2026

View reviewed changes

sammshen approved these changes Mar 7, 2026

View reviewed changes

ApostaC enabled auto-merge (squash) March 9, 2026 17:51

github-actions Bot added the full Run comprehensive tests on this PR label Mar 9, 2026

ApostaC merged commit ed20f7a into LMCache:dev Mar 9, 2026
21 of 25 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Chore][Docs] Update docs for MP mode#2708

[Chore][Docs] Update docs for MP mode#2708
ApostaC merged 2 commits intoLMCache:devfrom
ApostaC:local-dev/mp-doc

ApostaC commented Mar 6, 2026

Uh oh!

gemini-code-assist Bot commented Mar 6, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KuntaiDu left a comment

Uh oh!

sammshen left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		.. code-block:: bash

		kubectl apply -f examples/multi_process/vllm-deployment.yaml

Conversation

ApostaC commented Mar 6, 2026

Uh oh!

gemini-code-assist Bot commented Mar 6, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KuntaiDu left a comment

Choose a reason for hiding this comment

Uh oh!

sammshen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants