[Feat] LMCache MP mode k8s operator by royyhuang · Pull Request #2701 · LMCache/LMCache

royyhuang · 2026-03-06T01:01:48Z

Implementation PR for #2693: implementing K8s operator for LMCache multiprocess mode.

If applicable:

this PR contains user facing changes - docs added
this PR contains unit tests

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

gemini-code-assist · 2026-03-06T01:03:16Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a Kubernetes operator for LMCache, addressing manual configuration complexities and enhancing deployment automation. The operator simplifies management, enforces best practices, and provides seamless integration with vLLM and Prometheus, improving the overall user experience and operational efficiency.

Highlights

Kubernetes Operator Implementation: This PR implements a Kubernetes operator for LMCache multiprocess mode, automating lifecycle management and simplifying deployment.
CRD Definition: Introduces a new CRD, LMCacheEngine, to declaratively manage LMCache server configurations within Kubernetes.
Automated Pod Settings: Automatically injects necessary pod settings like hostIPC: true and --host 0.0.0.0, reducing manual configuration and potential errors.
Service Discovery: Creates a node-local Service with a connection ConfigMap, providing a stable discovery mechanism for vLLM instances.
Prometheus Integration: Declaratively integrates Prometheus monitoring via a ServiceMonitor CR, simplifying metrics collection.
CRD Validation: Implements OpenAPI schema validation and a validating webhook to catch misconfigurations at apply time.
Resource Sizing: Automatically computes resource requests and limits based on the L1 cache size, optimizing resource allocation.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Ignored Files

Ignored by pattern: .github/workflows/** (4)
- .github/workflows/operator_ci.yml
- .github/workflows/operator_nightly.yml
- .github/workflows/operator_release.yml
- .github/workflows/operator_test_e2e.yml

Activity

Implemented the core operator logic and CRD definitions.
Added automated pod settings injection.
Configured node-local service discovery.
Integrated Prometheus monitoring.
Implemented CRD validation and resource sizing.
Added various configuration files for linting, pre-commit hooks, and Docker.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a comprehensive Kubernetes operator for LMCache, including the controller implementation, CRD, RBAC, and extensive documentation. The overall structure and design are solid, following modern operator development practices. I've identified a few issues, primarily related to dependency pinning for reproducibility and a critical bug in the default container image used for the LMCache DaemonSet. Once these are addressed, this will be a great addition.

gemini-code-assist · 2026-03-06T01:06:12Z

+type ImageSpec struct {
+	// repository is the container image repository.
+	// +optional
+	// +kubebuilder:default="lmcache/vllm-openai"


The default image repository is set to lmcache/vllm-openai. This seems incorrect for the LMCache server pods managed by this operator. Based on the design documents and sample manifests, the default should be lmcache/standalone. The lmcache/vllm-openai image is for vLLM deployments that connect to LMCache.

Suggested change

// +kubebuilder:default="lmcache/vllm-openai"

// +kubebuilder:default="lmcache/standalone"

gemini-code-assist · 2026-03-06T01:06:12Z

+	podAnnotations := spec.PodAnnotations
+
+	serverPort := derefInt32(getServerPort(spec), 5555)
+	imgRepo := "lmcache/vllm-openai"


The image repository is hardcoded to lmcache/vllm-openai. This is incorrect for the LMCache DaemonSet, which should be running the LMCache server. The image should be lmcache/standalone to match the project's documentation and intended purpose.

Suggested change

imgRepo := "lmcache/vllm-openai"

imgRepo := "lmcache/standalone"

gemini-code-assist · 2026-03-06T01:06:13Z

+  # logcheck validates structured logging calls and parameters (e.g., balanced key-value pairs)
+  - module: "sigs.k8s.io/logtools"
+    import: "sigs.k8s.io/logtools/logcheck/gclplugin"
+    version: latest


Using version: latest can lead to non-reproducible builds if a new version of the plugin is released with breaking changes. It's a best practice to pin dependencies to a specific version to ensure build stability.

version: "v0.1.0" # Or any other specific version tag

gemini-code-assist · 2026-03-06T01:06:13Z

+# Install kind
+if ! command -v kind &> /dev/null; then
+  echo "Installing kind..."
+  curl -Lo /usr/local/bin/kind "https://kind.sigs.k8s.io/dl/latest/kind-linux-${ARCH}"


The script downloads the latest versions of kind and kubebuilder, and the stable version of kubectl. For a reproducible development environment, it's better to pin these to specific versions. This ensures all developers are using the same toolset and avoids unexpected issues from tool updates.

gemini-code-assist · 2026-03-06T01:06:13Z

@@ -0,0 +1,61 @@
+version: "2"


The configuration key version: "2" is not a standard part of the .golangci.yml schema and will likely be ignored or cause an error. Please remove this line. The version of golangci-lint itself is managed in the Makefile.

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

…into feat/lmcache-operator

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

KuntaiDu

LGTM!

* [feat] add k8s opeartor for MP mode Signed-off-by: royyhuang <roy.y.huang@gmail.com> * [misc] remove unncessary hostPID Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix false positive errors in precommit spellchecks Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix lint Signed-off-by: royyhuang <roy.y.huang@gmail.com> * skip python tests and builds for operator only changes Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix flaky e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com>

* [feat] add k8s opeartor for MP mode Signed-off-by: royyhuang <roy.y.huang@gmail.com> * [misc] remove unncessary hostPID Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix false positive errors in precommit spellchecks Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix lint Signed-off-by: royyhuang <roy.y.huang@gmail.com> * skip python tests and builds for operator only changes Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix flaky e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com> Signed-off-by: shaoxiawjc <wjc2800@163.com>

* [feat] add k8s opeartor for MP mode Signed-off-by: royyhuang <roy.y.huang@gmail.com> * [misc] remove unncessary hostPID Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix false positive errors in precommit spellchecks Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix lint Signed-off-by: royyhuang <roy.y.huang@gmail.com> * skip python tests and builds for operator only changes Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix flaky e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com> Signed-off-by: Aaron Wu <aaron.wu@dell.com>

* [feat] add k8s opeartor for MP mode Signed-off-by: royyhuang <roy.y.huang@gmail.com> * [misc] remove unncessary hostPID Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix false positive errors in precommit spellchecks Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix lint Signed-off-by: royyhuang <roy.y.huang@gmail.com> * skip python tests and builds for operator only changes Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix flaky e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix e2e tests Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com>

royyhuang added 3 commits March 5, 2026 21:44

[feat] add k8s opeartor for MP mode

47de524

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

[misc] remove unncessary hostPID

9209630

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

Merge branch 'dev' into feat/lmcache-operator

4957c12

royyhuang changed the title ~~Feat/lmcache operator~~ [Feat] LMCache MP mode k8s operator Mar 6, 2026

gemini-code-assist Bot reviewed Mar 6, 2026

View reviewed changes

royyhuang added 8 commits March 6, 2026 01:09

fix false positive errors in precommit spellchecks

42db8ac

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

fix lint

60db7fa

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

skip python tests and builds for operator only changes

919a126

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

fix flaky e2e tests

b17c1ae

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

Merge branch 'dev' into feat/lmcache-operator

eb3a221

fix e2e tests

fa3189a

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

Merge branch 'feat/lmcache-operator' of github.com:royyhuang/LMCache …

b012dd7

…into feat/lmcache-operator

fix e2e tests

450e041

Signed-off-by: royyhuang <roy.y.huang@gmail.com>

KuntaiDu approved these changes Mar 6, 2026

View reviewed changes

Oasis-Git approved these changes Mar 6, 2026

View reviewed changes

Merge branch 'dev' into feat/lmcache-operator

294ed7a

royyhuang enabled auto-merge (squash) March 6, 2026 21:24

github-actions Bot added the full Run comprehensive tests on this PR label Mar 6, 2026

Merge branch 'dev' into feat/lmcache-operator

2ca4527

royyhuang merged commit f636089 into LMCache:dev Mar 7, 2026
34 of 46 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feat] LMCache MP mode k8s operator#2701

[Feat] LMCache MP mode k8s operator#2701
royyhuang merged 13 commits intoLMCache:devfrom
royyhuang:feat/lmcache-operator

royyhuang commented Mar 6, 2026

Uh oh!

gemini-code-assist Bot commented Mar 6, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Uh oh!

KuntaiDu left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	// +kubebuilder:default="lmcache/vllm-openai"
	// +kubebuilder:default="lmcache/standalone"

	imgRepo := "lmcache/vllm-openai"
	imgRepo := "lmcache/standalone"

Conversation

royyhuang commented Mar 6, 2026

Uh oh!

gemini-code-assist Bot commented Mar 6, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

KuntaiDu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants