[MP][UX][Docs] Enhance http server and its docs for MP mode by ApostaC · Pull Request #2722 · LMCache/LMCache

ApostaC · 2026-03-09T23:21:55Z

What this PR does / why we need it:

Adds a POST /api/clear-cache endpoint to the MP mode HTTP server that force-clears all KV cache data stored in L1 (CPU) memory. Also simplifies the /api/healthcheck endpoint by removing the expensive memcheck() call — it now only checks that the engine is initialized.

Updates the MP mode documentation (quickstart, architecture, deployment) to cover all HTTP API endpoints including the new clear-cache, the upstream status endpoint, and the simplified healthcheck.

Special notes for your reviewers:

The /api/status endpoint was added upstream — this PR only adds its documentation.

If applicable:

this PR contains user facing changes - docs added
this PR contains unit tests

Signed-off-by: ApostaC <yihua98@uchicago.edu>

gemini-code-assist · 2026-03-09T23:22:16Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the multiprocess (MP) mode HTTP server by introducing a new API endpoint for cache management and streamlining an existing health check. It also provides comprehensive documentation updates across various guides to ensure all HTTP API functionalities are clearly described, improving the server's operational utility and clarity for users.

Highlights

New API Endpoint: Added a POST /api/clear-cache endpoint to the MP mode HTTP server, allowing for the force-clearing of all KV cache data stored in L1 (CPU) memory.
Simplified Healthcheck: The /api/healthcheck endpoint was simplified by removing the expensive memcheck() call, now only verifying that the engine is initialized.
Enhanced Documentation: Updated the MP mode documentation (quickstart, architecture, deployment) to comprehensively cover all HTTP API endpoints, including the new clear-cache, the upstream status endpoint, and the simplified healthcheck.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

docs/source/mp/architecture.rst
- Updated description of http_server.py to include POST /api/clear-cache and /api/status endpoints.
- Revised the description for lmcache/v1/multiprocess/http_server.py in the key source files table to reflect its expanded API functionality.
docs/source/mp/deployment.rst
- Modified the HTTP server variant description to highlight support for cache management API.
docs/source/mp/quickstart.rst
- Updated the introductory text for the HTTP server quick start to mention HTTP management endpoints.
- Replaced the simple health check example with a detailed list-table of all HTTP endpoints (/api/healthcheck, /api/clear-cache, /api/status) and provided corresponding curl examples.
lmcache/v1/multiprocess/http_server.py
- Removed the engine.storage_manager.memcheck() call from the /api/healthcheck endpoint, simplifying its logic to only check engine initialization.
- Implemented a new POST /api/clear-cache endpoint that triggers engine.clear() to force-clear L1 KV cache data.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request enhances the multi-process HTTP server by adding a /api/clear-cache endpoint and simplifying the /api/healthcheck endpoint. However, the new /api/clear-cache endpoint lacks authentication, which could allow unauthorized users to perform destructive operations on the cache, leading to a potential Denial of Service (DoS). Additionally, there is an inconsistency in the quickstart guide regarding the health check's behavior that needs addressing.

gemini-code-assist · 2026-03-09T23:33:50Z

+@app.post("/api/clear-cache")
+async def clear_cache(request: Request):
+    """
+    Force-clear all KV cache data stored in L1 (CPU) memory.
+
+    This clears all objects including those with active read/write locks.
+    In-flight store or prefetch operations may be corrupted.
+    """
+    engine = getattr(request.app.state, "engine", None)
+    if engine is None:
        return JSONResponse(
            status_code=503,
-            content={"status": "unhealthy", "reason": "memory check failed"},
+            content={"status": "error", "reason": "engine not initialized"},
        )

-    return {"status": "healthy"}
+    engine.clear()
+    logger.info("Cache cleared via HTTP API")
+    return {"status": "ok"}


The new /api/clear-cache endpoint allows any user who can reach the HTTP server to force-clear all KV cache data in L1 memory. This is a destructive operation that can also corrupt in-flight store or prefetch operations. Exposing such a management endpoint without any authentication or authorization mechanism poses a risk of Denial of Service (DoS). It is highly recommended to implement an authentication mechanism (e.g., API key, bearer token) for administrative endpoints like this.

gemini-code-assist · 2026-03-09T23:33:50Z

+     - Returns ``{"status": "healthy"}`` when the engine is initialized and
+       memory checks pass. Suitable for Kubernetes liveness/readiness probes.


The description for the /api/healthcheck endpoint is no longer accurate. This pull request removes the memcheck() call from the health check, so it no longer needs to 'memory checks pass'. The documentation should be updated to reflect that it only checks for engine initialization.

Suggested change

- Returns ``{"status": "healthy"}`` when the engine is initialized and

memory checks pass. Suitable for Kubernetes liveness/readiness probes.

- Returns ``{"status": "healthy"}`` when the engine is initialized.

Suitable for Kubernetes liveness/readiness probes.

ApostaC · 2026-03-10T01:09:01Z

Need to merge #2723 first

Signed-off-by: ApostaC <yihua98@uchicago.edu>

…2722) * update http server for clear-cache and update docs * update docs * add config to specify engine Signed-off-by: ApostaC <yihua98@uchicago.edu> Signed-off-by: shaoxiawjc <wjc2800@163.com>

…2722) * update http server for clear-cache and update docs * update docs * add config to specify engine Signed-off-by: ApostaC <yihua98@uchicago.edu> Signed-off-by: Aaron Wu <aaron.wu@dell.com>

…2722) * update http server for clear-cache and update docs * update docs * add config to specify engine Signed-off-by: ApostaC <yihua98@uchicago.edu>

ApostaC added 2 commits March 9, 2026 23:20

update http server for clear-cache and update docs

33a2ede

Signed-off-by: ApostaC <yihua98@uchicago.edu>

update docs

1580097

Signed-off-by: ApostaC <yihua98@uchicago.edu>

ApostaC requested review from Oasis-Git and YaoJiayi March 9, 2026 23:25

gemini-code-assist Bot reviewed Mar 9, 2026

View reviewed changes

YaoJiayi approved these changes Mar 10, 2026

View reviewed changes

Oasis-Git approved these changes Mar 10, 2026

View reviewed changes

Merge branch 'dev' into local-dev/clear-cache-endpoint

c94fdfa

ApostaC added the full Run comprehensive tests on this PR label Mar 10, 2026

ApostaC added 2 commits March 10, 2026 19:03

add config to specify engine

e79dcd8

Signed-off-by: ApostaC <yihua98@uchicago.edu>

Merge branch 'dev' into local-dev/clear-cache-endpoint

1816e81

ApostaC merged commit c2642c9 into LMCache:dev Mar 10, 2026
24 of 28 checks passed

ApostaC mentioned this pull request Mar 10, 2026

[MP] Update the MP docs and pass telemetry config into http_server #2730

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MP][UX][Docs] Enhance http server and its docs for MP mode#2722

[MP][UX][Docs] Enhance http server and its docs for MP mode#2722
ApostaC merged 5 commits intoLMCache:devfrom
ApostaC:local-dev/clear-cache-endpoint

ApostaC commented Mar 9, 2026

Uh oh!

gemini-code-assist Bot commented Mar 9, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Mar 9, 2026

Uh oh!

gemini-code-assist Bot Mar 9, 2026

Uh oh!

ApostaC commented Mar 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		- Returns ``{"status": "healthy"}`` when the engine is initialized and
		memory checks pass. Suitable for Kubernetes liveness/readiness probes.

Conversation

ApostaC commented Mar 9, 2026

Uh oh!

gemini-code-assist Bot commented Mar 9, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

ApostaC commented Mar 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants