[MP][UX][L2] Support configuring L2 store/prefetch policy via command line by ApostaC · Pull Request #2773 · LMCache/LMCache

ApostaC · 2026-03-14T00:44:50Z

What this PR does / why we need it:

Adds --l2-store-policy and --l2-prefetch-policy CLI flags so users can select L2 store and prefetch policies by name. This prepares for upcoming new policy implementations — adding a new policy only requires creating a single file in storage_controllers/ with a register_store_policy() or register_prefetch_policy() call; auto-discovery handles the rest.

Changes:

Add name-based policy registries to store_policy.py and prefetch_policy.py (register_*_policy, create_*_policy)
Add pkgutil.iter_modules() auto-discovery to storage_controllers/__init__.py
Add store_policy / prefetch_policy fields to StorageManagerConfig and corresponding CLI args
Replace hardcoded DefaultStorePolicy() / DefaultPrefetchPolicy() in StorageManager with registry-based creation

Special notes for your reviewers:

Default behavior is unchanged — both flags default to "default", matching the existing DefaultStorePolicy and DefaultPrefetchPolicy.

If applicable:

this PR contains user facing changes - docs added
this PR contains unit tests

Signed-off-by: ApostaC <yihua98@uchicago.edu>

gemini-code-assist · 2026-03-14T00:45:12Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the flexibility and extensibility of L2 cache management by introducing a configurable policy system. Instead of relying on hardcoded default behaviors, users can now specify L2 store and prefetch policies via command-line arguments. The core change involves setting up a robust, name-based registry with automatic module discovery, which simplifies the process of adding new policy implementations in the future. This change empowers developers to easily experiment with and deploy custom L2 caching strategies, improving overall system adaptability.

Highlights

Configurable L2 Policies: Introduced --l2-store-policy and --l2-prefetch-policy command-line flags, allowing users to select L2 store and prefetch policies by name.
Extensible Policy System: Implemented a name-based registry and auto-discovery mechanism for L2 policies. New policies can be added by simply creating a new file in storage_controllers/ and registering them, without modifying existing core files.
Dynamic Policy Instantiation: Updated the StorageManager to dynamically create L2 store and prefetch policy instances based on the configured policy names, replacing hardcoded defaults.
Documentation Updates: Added comprehensive documentation for the new L2 policy configuration options and the process for implementing new policies.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

docs/source/mp/configuration.rst
- Added a new 'L2 Policies' section to the configuration documentation, detailing the --l2-store-policy and --l2-prefetch-policy CLI arguments.
- Included the new L2 policy CLI arguments in the full example configuration.
docs/source/mp/l2_storage.rst
- Added a new section 'Store and Prefetch Policies' explaining their purpose and CLI selection.
- Provided a table of built-in policies and highlighted the extensibility of the policy system.
lmcache/v1/distributed/config.py
- Imported get_registered_prefetch_policies and get_registered_store_policies for policy lookup.
- Added store_policy and prefetch_policy string fields to StorageManagerConfig with 'default' values.
- Introduced --l2-store-policy and --l2-prefetch-policy arguments to add_storage_manager_args, using registered policies for choices.
- Passed the selected l2_store_policy and l2_prefetch_policy from arguments to StorageManagerConfig in parse_args_to_config.
lmcache/v1/distributed/l2_adapters/design_docs/overall.md
- Updated the design document to describe the name-based selection and self-registration of store and prefetch policies.
- Added a new section 'Implementing a New Store or Prefetch Policy' with detailed instructions and code examples for policy extensibility.
lmcache/v1/distributed/storage_controllers/init.py
- Added a docstring explaining the automatic module discovery for storage controllers.
- Implemented pkgutil.iter_modules to auto-discover and import all modules within the package, ensuring policy self-registration calls are executed.
lmcache/v1/distributed/storage_controllers/prefetch_policy.py
- Introduced _PREFETCH_POLICY_REGISTRY to store prefetch policy classes.
- Defined register_prefetch_policy to add policy classes to the registry.
- Defined get_registered_prefetch_policies to retrieve a list of registered policy names.
- Defined create_prefetch_policy to instantiate a prefetch policy by its registered name.
- Registered DefaultPrefetchPolicy under the name 'default'.
lmcache/v1/distributed/storage_controllers/store_policy.py
- Introduced _STORE_POLICY_REGISTRY to store store policy classes.
- Defined register_store_policy to add policy classes to the registry.
- Defined get_registered_store_policies to retrieve a list of registered policy names.
- Defined create_store_policy to instantiate a store policy by its registered name.
- Registered DefaultStorePolicy under the name 'default'.
lmcache/v1/distributed/storage_manager.py
- Updated imports to use create_prefetch_policy and create_store_policy instead of direct DefaultPrefetchPolicy and DefaultStorePolicy.
- Modified StorageManager.__init__ to instantiate L2 store and prefetch policies dynamically using create_store_policy(config.store_policy) and create_prefetch_policy(config.prefetch_policy).

Activity

No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a well-designed and flexible mechanism for configuring L2 store and prefetch policies via command-line flags. The use of a registry pattern with auto-discovery is an excellent choice for extensibility. The code is clean, and the documentation updates are thorough for both users and developers. I've identified one high-severity issue where the policy auto-discovery may not run in time for command-line argument parsing, which would prevent custom policies from being used. A code suggestion is provided to resolve this. The rest of the implementation is solid.

Signed-off-by: ApostaC <yihua98@uchicago.edu>

maobaolong

Thanks for this refactor and make l2 store/prefetch policy configurable. It LGTM.

sammshen

LGTM!

maobaolong · 2026-03-14T05:39:17Z

@ApostaC The failed ut seems related to this pr

E ImportError: cannot import name 'EvictionConfig' from partially initialized module 'lmcache.v1.distributed.config' (most likely due to a circular import) (/var/lib/buildkite-agent/builds/lmcache-unit-test-build-1/lmcache/unit-tests/lmcache/v1/distributed/config.py)

Signed-off-by: ApostaC <yihua98@uchicago.edu>

… line (LMCache#2773)

… line (LMCache#2773) Signed-off-by: Aaron Wu <aaron.wu@dell.com>

… line (LMCache#2773)

add cmdline for store and prefetch policy

f6e3c3a

Signed-off-by: ApostaC <yihua98@uchicago.edu>

gemini-code-assist Bot reviewed Mar 14, 2026

View reviewed changes

Comment thread lmcache/v1/distributed/config.py Outdated

force import to trigger the registration

5f83e85

Signed-off-by: ApostaC <yihua98@uchicago.edu>

ApostaC mentioned this pull request Mar 14, 2026

[MP] Support buffer only mode for MP mode #2760

Merged

2 tasks

maobaolong approved these changes Mar 14, 2026

View reviewed changes

sammshen approved these changes Mar 14, 2026

View reviewed changes

maobaolong added the full Run comprehensive tests on this PR label Mar 14, 2026

ApostaC enabled auto-merge (squash) March 14, 2026 02:53

fix import issue

c6fa6e4

Signed-off-by: ApostaC <yihua98@uchicago.edu>

ApostaC merged commit ef978a8 into LMCache:dev Mar 15, 2026
27 of 28 checks passed

hyunyul-XCENA pushed a commit to xcena-dev/LMCache that referenced this pull request Mar 20, 2026

[MP][UX][L2] Support configuring L2 store/prefetch policy via command…

175cfeb

… line (LMCache#2773)

realAaronWu pushed a commit to realAaronWu/LMCache that referenced this pull request Mar 20, 2026

[MP][UX][L2] Support configuring L2 store/prefetch policy via command…

429e9c6

… line (LMCache#2773) Signed-off-by: Aaron Wu <aaron.wu@dell.com>

deng451e pushed a commit to deng451e/LMCache that referenced this pull request Mar 25, 2026

[MP][UX][L2] Support configuring L2 store/prefetch policy via command…

9729ea5

… line (LMCache#2773)

deng451e pushed a commit to deng451e/LMCache that referenced this pull request Mar 27, 2026

[MP][UX][L2] Support configuring L2 store/prefetch policy via command…

4293470

… line (LMCache#2773)

jooho-XCENA pushed a commit to xcena-dev/LMCache that referenced this pull request Apr 2, 2026

[MP][UX][L2] Support configuring L2 store/prefetch policy via command…

ed5a034

… line (LMCache#2773)

jooho-XCENA pushed a commit to xcena-dev/LMCache that referenced this pull request Apr 2, 2026

[MP][UX][L2] Support configuring L2 store/prefetch policy via command…

643e590

… line (LMCache#2773)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MP][UX][L2] Support configuring L2 store/prefetch policy via command line#2773

[MP][UX][L2] Support configuring L2 store/prefetch policy via command line#2773
ApostaC merged 3 commits intoLMCache:devfrom
ApostaC:local-dev/l2-policy-cli

ApostaC commented Mar 14, 2026

Uh oh!

gemini-code-assist Bot commented Mar 14, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

maobaolong left a comment

Uh oh!

sammshen left a comment

Uh oh!

maobaolong commented Mar 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ApostaC commented Mar 14, 2026

Uh oh!

gemini-code-assist Bot commented Mar 14, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

maobaolong left a comment

Choose a reason for hiding this comment

Uh oh!

sammshen left a comment

Choose a reason for hiding this comment

Uh oh!

maobaolong commented Mar 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants