Skip to content

[2/4] L2Adapter interface and implementation of MockL2Adapter#2569

Merged
KuntaiDu merged 5 commits intoLMCache:devfrom
ApostaC:local-dev/l2-adapter-2
Feb 26, 2026
Merged

[2/4] L2Adapter interface and implementation of MockL2Adapter#2569
KuntaiDu merged 5 commits intoLMCache:devfrom
ApostaC:local-dev/l2-adapter-2

Conversation

@ApostaC
Copy link
Copy Markdown
Contributor

@ApostaC ApostaC commented Feb 9, 2026

What this PR does / why we need it:

This PR is part of #2562
It defines the interface of the L2 adapter and provides the implementation of a MockL2Adapter using CPU ram (main objective is for testing).
It also has configuration and cmdline interface-related codes.

This PR needs to be rebased after #2563 is merged

Special notes for your reviewers:

If applicable:

  • this PR contains user facing changes
  • this PR contains unit tests

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello @ApostaC, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request lays the groundwork for L2 storage integration by defining a standardized interface for L2 adapters and providing a mock implementation for development and testing. It also introduces a low-level bitmap utility for efficient status tracking and extends the system's configuration capabilities to support these new L2 components. The changes enable future development of diverse L2 storage solutions while ensuring a consistent interaction model.

Highlights

  • L2 Adapter Interface: Introduced a new abstract base class, L2AdapterInterface, which defines the contract for L2 I/O adapters, including non-blocking primitives for store, lookup/lock, load, and unlock operations, along with event file descriptors for signaling task completion.
  • MockL2Adapter Implementation: Provided a MockL2Adapter implementation, primarily for testing purposes. This adapter simulates L2 storage using CPU RAM, featuring configurable maximum capacity and mock bandwidth to simulate transfer delays. It includes a FIFO eviction policy and object locking mechanisms.
  • Bitmap Utility Class: Added a Bitmap C++ class for efficient tracking of bit states, exposed to Python via pybind11. This class supports operations like setting/clearing bits, testing, counting set bits (popcount), counting leading zeros (clz), counting leading ones (clo), and bitwise AND, crucial for handling success/failure bitmaps in L2 operations.
  • Configuration Integration: Integrated L2 adapter configuration into the existing storage manager setup. This allows L2 adapters to be configured dynamically via command-line arguments using a new L2AdaptersConfig and associated parsing utilities.
  • Comprehensive Testing: Added extensive unit tests for both the new Bitmap class and the MockL2Adapter to ensure their correctness, robustness, and adherence to the defined interfaces, covering various scenarios including edge cases and bandwidth simulation.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • csrc/storage_manager/bitmap.cpp
    • Added implementation for a Bitmap class, providing core bit manipulation functionalities like set, clear, test, popcount, clz, clo, bitwise AND, and string conversion.
  • csrc/storage_manager/bitmap.h
    • Added the header file for the Bitmap class, defining its public interface and documentation.
  • csrc/storage_manager/pybind.cpp
    • Modified to include and expose the new Bitmap class to Python via pybind11, making its methods accessible from Python.
  • lmcache/native_storage_ops.pyi
    • Updated type hints to include the newly exposed Bitmap class and its methods.
  • lmcache/v1/distributed/config.py
    • Imported L2 adapter configuration utilities.
    • Added l2_adapter_config to StorageManagerConfig.
    • Integrated add_l2_adapters_args and parse_args_to_l2_adapters_config for command-line argument parsing related to L2 adapters.
  • lmcache/v1/distributed/l2_adapters/base.py
    • Added a new file defining the L2AdapterInterface abstract base class, outlining the required methods for L2 storage interaction.
  • lmcache/v1/distributed/l2_adapters/config.py
    • Added a new file for L2 adapter configuration, including a registry for adapter types, a base config class, and a specific MockL2AdapterConfig.
    • Provided utilities to add and parse L2 adapter arguments from the command line.
  • lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py
    • Added a new file implementing MockL2Adapter, a CPU RAM-based L2 adapter for testing, complete with simulated bandwidth, FIFO eviction, and object locking.
  • setup.py
    • Modified to include csrc/storage_manager/bitmap.cpp in the storage_manager_sources for both CUDA and ROCm extensions, ensuring proper compilation.
  • tests/v1/distributed/init.py
    • Added an empty __init__.py file to establish tests/v1/distributed as a Python package.
  • tests/v1/distributed/test_mock_l2_adapter.py
    • Added comprehensive unit tests for the MockL2Adapter, covering its event FD interfaces, store, lookup/lock, unlock, load operations, end-to-end workflows, close functionality, and bandwidth simulation.
  • tests/v1/native_storage_ops/test_bitmap.py
    • Added unit tests for the Bitmap class, verifying its core functionalities and edge cases, especially for non-multiple-of-8 sizes.
Activity
  • The pull request introduces the L2AdapterInterface to standardize how L2 storage interacts with the system.
  • A MockL2Adapter is implemented to facilitate testing of L2 storage mechanisms without requiring actual hardware.
  • A Bitmap C++ utility, exposed to Python, is added to efficiently manage and track success/failure states for batched L2 operations.
  • Configuration for L2 adapters is integrated, allowing them to be specified and parsed via command-line arguments.
  • New unit tests have been added for both the Bitmap class and the MockL2Adapter, ensuring the reliability and correctness of these new components.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a well-defined L2 adapter interface, a mock implementation for testing purposes, and a new Bitmap utility. The overall structure is robust and extensible, particularly the configuration handling for L2 adapters. The C++ Bitmap implementation and its Python bindings are clean and well-tested. I've identified a few areas for improvement in the MockL2Adapter implementation, including a potential infinite loop during eviction and an issue with tensor cloning. My review includes specific suggestions to address these points.

Comment thread lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py
Comment thread lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py Outdated
Comment thread lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py Outdated
…pter

Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: ApostaC <yihua98@uchicago.edu>
@ApostaC ApostaC force-pushed the local-dev/l2-adapter-2 branch from 6fa74a3 to 0cc638f Compare February 26, 2026 18:51
@ApostaC
Copy link
Copy Markdown
Contributor Author

ApostaC commented Feb 26, 2026

/gemini review

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a well-defined L2AdapterInterface and a MockL2Adapter for testing, which is a great step towards a multi-tiered caching system. The configuration handling via a registry pattern is robust and extensible, and the use of a background asyncio loop in the mock adapter is a solid design choice for simulating asynchronous I/O. The tests are comprehensive and effectively validate the interface contract. I've included a few suggestions to enhance data isolation in the mock adapter and improve robustness against mismatched inputs.

Comment thread lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py Outdated
Comment thread lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py
Comment thread tests/v1/distributed/test_mock_l2_adapter.py
@ApostaC ApostaC mentioned this pull request Feb 26, 2026
8 tasks
Signed-off-by: ApostaC <yihua98@uchicago.edu>
@ApostaC ApostaC force-pushed the local-dev/l2-adapter-2 branch from 744c939 to 2377018 Compare February 26, 2026 19:44
@ApostaC ApostaC added full Run comprehensive tests on this PR mp Buildkite trigger for multi-processing mode test and removed full Run comprehensive tests on this PR labels Feb 26, 2026
Copy link
Copy Markdown
Contributor

@KuntaiDu KuntaiDu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Copy Markdown
Contributor

@sammshen sammshen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@KuntaiDu KuntaiDu merged commit 7705750 into LMCache:dev Feb 26, 2026
32 of 33 checks passed
sammshen pushed a commit to sammshen/LMCache that referenced this pull request Mar 1, 2026
…e#2569)

* [Add] interface for L2 adapter and the implementation for mock l2 adapter

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* add default empty l2 adapter config

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* fix gemini issue

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* update mock l2 connector to use async

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* Update with gemini's suggestion

Signed-off-by: ApostaC <yihua98@uchicago.edu>

---------

Signed-off-by: ApostaC <yihua98@uchicago.edu>
hlin99 pushed a commit to hlin99/LMCache that referenced this pull request Mar 2, 2026
…e#2569)

* [Add] interface for L2 adapter and the implementation for mock l2 adapter

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* add default empty l2 adapter config

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* fix gemini issue

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* update mock l2 connector to use async

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* Update with gemini's suggestion

Signed-off-by: ApostaC <yihua98@uchicago.edu>

---------

Signed-off-by: ApostaC <yihua98@uchicago.edu>
oferki pushed a commit to oferki/LMCache that referenced this pull request Mar 3, 2026
…e#2569)

* [Add] interface for L2 adapter and the implementation for mock l2 adapter

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* add default empty l2 adapter config

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* fix gemini issue

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* update mock l2 connector to use async

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* Update with gemini's suggestion

Signed-off-by: ApostaC <yihua98@uchicago.edu>

---------

Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io>
oferki pushed a commit to oferki/LMCache that referenced this pull request Mar 3, 2026
…e#2569)

* [Add] interface for L2 adapter and the implementation for mock l2 adapter

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* add default empty l2 adapter config

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* fix gemini issue

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* update mock l2 connector to use async

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* Update with gemini's suggestion

Signed-off-by: ApostaC <yihua98@uchicago.edu>

---------

Signed-off-by: ApostaC <yihua98@uchicago.edu>
mauryaavinash95 pushed a commit to mauryaavinash95/LMCache that referenced this pull request Mar 7, 2026
…e#2569)

* [Add] interface for L2 adapter and the implementation for mock l2 adapter

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* add default empty l2 adapter config

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* fix gemini issue

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* update mock l2 connector to use async

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* Update with gemini's suggestion

Signed-off-by: ApostaC <yihua98@uchicago.edu>

---------

Signed-off-by: ApostaC <yihua98@uchicago.edu>
shaoxiawjc pushed a commit to shaoxiawjc/LMCache that referenced this pull request Mar 11, 2026
…e#2569)

* [Add] interface for L2 adapter and the implementation for mock l2 adapter

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* add default empty l2 adapter config

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* fix gemini issue

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* update mock l2 connector to use async

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* Update with gemini's suggestion

Signed-off-by: ApostaC <yihua98@uchicago.edu>

---------

Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: shaoxiawjc <wjc2800@163.com>
realAaronWu pushed a commit to realAaronWu/LMCache that referenced this pull request Mar 20, 2026
…e#2569)

* [Add] interface for L2 adapter and the implementation for mock l2 adapter

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* add default empty l2 adapter config

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* fix gemini issue

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* update mock l2 connector to use async

Signed-off-by: ApostaC <yihua98@uchicago.edu>

* Update with gemini's suggestion

Signed-off-by: ApostaC <yihua98@uchicago.edu>

---------

Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Aaron Wu <aaron.wu@dell.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mp Buildkite trigger for multi-processing mode test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants