[refactor]: Replace use_cufile with use_gds/gds_backend config flags by glimchb · Pull Request #2858 · LMCache/LMCache

glimchb · 2026-03-23T21:52:13Z

Follow up work post #2799 (AMD hipFile AIS support).

Refactor the GDS backend configuration to be backend-agnostic:

Replace extra_config["use_cufile"] with top-level use_gds (bool) config
Add gds_backend config field ("cufile" default) to select GDS library
Rename cufile_buffer_size to gds_buffer_size
Rename internal attributes: self.cufile -> self.gds_module, self._cufile_driver -> self._gds_driver, self.cufile_base_pointer -> self.gds_base_pointer
Update tests and documentation accordingly

New env vars: LMCACHE_USE_GDS, LMCACHE_GDS_BACKEND, LMCACHE_GDS_BUFFER_SIZE

Generated with Devin

What this PR does / why we need it:

Special notes for your reviewers:

If applicable:

this PR contains user facing changes - docs added
this PR contains unit tests

Note

Medium Risk
Touches the GdsBackend initialization and allocator selection paths, which are performance/IO-critical and could change runtime behavior (e.g., auto-disabling GDS on certain filesystems or selecting the wrong backend). Overall scope is contained to configuration and GDS backend wiring, with tests/docs updated accordingly.

Overview
Refactors GDS configuration to be backend-agnostic: replaces extra_config flags (use_cufile/use_hipfile) with top-level use_gds plus a gds_backend selector (cufile/hipfile), and renames cufile_buffer_size to gds_buffer_size (including new env vars like LMCACHE_GDS_BUFFER_SIZE, LMCACHE_USE_GDS, LMCACHE_GDS_BACKEND).

Updates GdsBackend to route imports/driver setup through the chosen backend module, standardizes internal naming (gds_module, _gds_driver, gds_base_pointer), and tweaks filesystem-based auto-disable logic to only override defaults when use_gds wasn’t explicitly set. Tests, sample configs, Buildkite config, and docs are updated to the new keys and behavior.

^{Reviewed by Cursor Bugbot for commit 217b6b9. Bugbot is set up for automated code reviews on this repo. Configure here.}

gemini-code-assist · 2026-03-23T21:52:37Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors the GPU Direct Storage (GDS) backend configuration to be more generic and extensible. The changes introduce new top-level configuration flags and rename existing ones, moving away from NVIDIA cuFile-specific terminology to a more backend-agnostic approach. This prepares the system for future support of alternative GDS implementations, such as AMD hipFile AIS, by centralizing GDS control and selection.

Highlights

Configuration Refactor: Introduced new top-level configuration flags use_gds (boolean) and gds_backend (string, default "cufile") to replace the extra_config["use_cufile"] setting, enabling backend-agnostic GDS configuration.
Parameter Renaming: Renamed the configuration field cufile_buffer_size to gds_buffer_size across the codebase and documentation for broader applicability.
Internal Attribute Updates: Updated internal GDS backend attributes from self.cufile, self._cufile_driver, and self.cufile_base_pointer to self.gds_module, self._gds_driver, and self.gds_base_pointer respectively, to align with the new backend-agnostic design.
Documentation and Test Alignment: Modified documentation and test cases to reflect the new configuration flags and renamed parameters, ensuring consistency and correctness.
New Environment Variables: Added new environment variables LMCACHE_USE_GDS, LMCACHE_GDS_BACKEND, and LMCACHE_GDS_BUFFER_SIZE for configuring GDS.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request refactors the GDS backend configuration to be backend-agnostic, replacing use_cufile with use_gds and introducing a gds_backend configuration flag. The changes are applied throughout the configuration, documentation, and tests. The review identified a redundant code check and a documentation typo that could be addressed to improve code quality.

DongDongJu

Thanks for this refactoring. I left one question.

sammshen

LGTM! please address @DongDongJu's comments

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 4 total unresolved issues (including 3 from previous reviews).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit b584020. Configure here.}

Follow up work post LMCache#2799 (AMD hipFile AIS support). Refactor the GDS backend configuration to be backend-agnostic: - Replace extra_config["use_cufile"] with top-level use_gds (bool) config - Add gds_backend config field ("cufile" default) to select GDS library - Rename cufile_buffer_size to gds_buffer_size - Rename internal attributes: self.cufile -> self.gds_module, self._cufile_driver -> self._gds_driver, self.cufile_base_pointer -> self.gds_base_pointer - Update tests and documentation accordingly New env vars: LMCACHE_USE_GDS, LMCACHE_GDS_BACKEND, LMCACHE_GDS_BUFFER_SIZE Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>

glimchb mentioned this pull request Mar 23, 2026

Add hipFile support for AIS (AMD Infinity Storage) storage #2799

Merged

2 tasks

gemini-code-assist Bot reviewed Mar 23, 2026

View reviewed changes

Comment thread lmcache/v1/storage_backend/gds_backend.py

Comment thread docs/source/kv_cache/storage_backends/gds.rst

glimchb marked this pull request as draft March 23, 2026 21:57

glimchb force-pushed the gds-config-changes branch 6 times, most recently from 95eb201 to ab54625 Compare March 30, 2026 13:31

glimchb marked this pull request as ready for review March 30, 2026 13:31

DongDongJu reviewed Mar 30, 2026

View reviewed changes

Comment thread lmcache/v1/storage_backend/gds_backend.py

sammshen approved these changes Mar 30, 2026

View reviewed changes

DongDongJu approved these changes Mar 31, 2026

View reviewed changes

Comment thread lmcache/v1/storage_backend/gds_backend.py

DongDongJu enabled auto-merge (squash) March 31, 2026 00:09

github-actions Bot added the full Run comprehensive tests on this PR label Mar 31, 2026

auto-merge was automatically disabled March 31, 2026 01:03
Head branch was pushed to by a user without write access

glimchb force-pushed the gds-config-changes branch from ab54625 to 1710819 Compare March 31, 2026 01:03

github-actions Bot removed the full Run comprehensive tests on this PR label Mar 31, 2026

glimchb force-pushed the gds-config-changes branch from 1710819 to ecfb252 Compare March 31, 2026 14:05

sammshen added the full Run comprehensive tests on this PR label Apr 1, 2026

DongDongJu enabled auto-merge (squash) April 1, 2026 19:57

auto-merge was automatically disabled April 6, 2026 14:53
Head branch was pushed to by a user without write access

glimchb force-pushed the gds-config-changes branch from ecfb252 to 1a121ee Compare April 6, 2026 14:53

glimchb requested review from ApostaC, YaoJiayi, chunxiaozheng, deng451e, hickeyma and maobaolong as code owners April 6, 2026 14:53

github-actions Bot removed the full Run comprehensive tests on this PR label Apr 6, 2026

cursor Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread lmcache/v1/config.py

Comment thread lmcache/v1/storage_backend/gds_backend.py

glimchb force-pushed the gds-config-changes branch from 1a121ee to f1241da Compare April 8, 2026 16:01

cursor Bot reviewed Apr 8, 2026

View reviewed changes

Comment thread lmcache/v1/cache_engine.py

glimchb force-pushed the gds-config-changes branch from f1241da to b584020 Compare April 13, 2026 13:36

cursor Bot reviewed Apr 13, 2026

View reviewed changes

Comment thread tests/v1/data/gds.yaml

glimchb force-pushed the gds-config-changes branch 4 times, most recently from d126740 to 6d552f5 Compare April 17, 2026 03:16

glimchb force-pushed the gds-config-changes branch from 6d552f5 to 217b6b9 Compare April 17, 2026 10:34

sammshen added the full Run comprehensive tests on this PR label Apr 20, 2026

DongDongJu enabled auto-merge (squash) April 20, 2026 14:34

DongDongJu merged commit 40adcc9 into LMCache:dev Apr 20, 2026
39 checks passed

glimchb deleted the gds-config-changes branch April 20, 2026 18:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[refactor]: Replace use_cufile with use_gds/gds_backend config flags#2858

[refactor]: Replace use_cufile with use_gds/gds_backend config flags#2858
DongDongJu merged 1 commit intoLMCache:devfrom
glimchb:gds-config-changes

glimchb commented Mar 23, 2026 •

edited by cursor Bot

Loading

Uh oh!

gemini-code-assist Bot commented Mar 23, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

DongDongJu left a comment

Uh oh!

Uh oh!

sammshen left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

glimchb commented Mar 23, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot commented Mar 23, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

DongDongJu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sammshen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

glimchb commented Mar 23, 2026 •

edited by cursor Bot

Loading