[Perf] [GDS] Performance improvements to GDS backend by oferki · Pull Request #2637 · LMCache/LMCache

oferki · 2026-02-25T01:02:06Z

What this PR does / why we need it:
Performance improvements to gds_backend in its critical paths:

Replaced python open with os.open
Added the O_NOATIME flag to not update file access time. Improves performance. Added error handling if this mode is not available.
Removed assertions access to memory_obj.tensor - a property that performs validity checks, crop of raw data, access to views. Added a call to is_valid instead
Moved the assertions under a special flag, that can be enabled for debug (unlikely that everyone runs python -O on their setup to avoid them themselves)

Special notes for your reviewers:
Please review error handling, in general there should not be a change to the control flow of the class.

… and O_NOATIME, remove accesses to memory_obj.tensor Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io>

gemini-code-assist · 2026-02-25T01:02:23Z

Summary of Changes

Hello @oferki, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the performance of the GDS backend by optimizing critical file I/O operations and streamlining assertion checks. It introduces direct operating system calls for metadata reading, leverages the O_NOATIME flag to reduce disk overhead, and refactors assertion logic to avoid unnecessary computations, all while providing a mechanism to enable detailed debugging when needed.

Highlights

File I/O Optimization: Replaced Python's high-level open() with os.open() and os.read() for metadata file access, bypassing buffering for small files to improve read performance.
O_NOATIME Flag Implementation: Integrated the O_NOATIME flag with os.open() to prevent updating file access times, further enhancing performance. Includes robust error handling to fall back to normal os.open() if O_NOATIME is unsupported.
Assertion Refinement: Modified assertion logic to use memory_obj.is_valid() instead of directly accessing memory_obj.tensor, avoiding expensive property checks and validity operations in critical paths.
Debug Assertion Control: Encapsulated performance-impacting assertions within a new _debug_asserts flag, allowing them to be enabled for debugging purposes without affecting production performance.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

lmcache/v1/storage_backend/gds_backend.py
- Added _debug_asserts and _use_noatime instance flags to control assertion behavior and O_NOATIME usage.
- Introduced a new _read_metadata_info method that uses os.open and os.read for faster metadata retrieval, incorporating O_NOATIME with a fallback mechanism.
- Updated _read_metadata to call the new _read_metadata_info method.
- Wrapped existing assert statements in _load_bytes_from_disk_with_allocation and _load_bytes_from_disk_with_memory methods under the _debug_asserts flag.
- Replaced memory_obj.tensor is None check with not memory_obj.is_valid() in _load_bytes_from_disk_with_memory.

Activity

No specific activity (comments, reviews, progress updates) has been recorded for this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces several valuable performance optimizations to the GdsBackend's critical paths. The changes, including using os.open with O_NOATIME for faster metadata reads and moving debug assertions behind a feature flag, are well-reasoned. My review includes a suggestion to improve error handling for cross-platform compatibility and points out a minor code cleanup to remove unreachable code. Overall, these are solid enhancements that should improve performance.

Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io>

sammshen · 2026-02-26T00:20:33Z

@ekaynar could you take a look?

sammshen

LGTM

ekaynar · 2026-03-04T15:40:51Z

LGTM

DongDongJu

LGTM

* [Perf] [GDS] performance improvements to GDS backend: use OS file ops and O_NOATIME, remove accesses to memory_obj.tensor Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io> * Fixes to Gemini comments Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io> --------- Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io> Signed-off-by: Aaron Wu <aaron.wu@dell.com>

* [Perf] [GDS] performance improvements to GDS backend: use OS file ops and O_NOATIME, remove accesses to memory_obj.tensor Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io> * Fixes to Gemini comments Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io> --------- Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io>

[Perf] [GDS] performance improvements to GDS backend: use OS file ops…

9ebcc8a

… and O_NOATIME, remove accesses to memory_obj.tensor Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io>

gemini-code-assist Bot reviewed Feb 25, 2026

View reviewed changes

Comment thread lmcache/v1/storage_backend/gds_backend.py

Comment thread lmcache/v1/storage_backend/gds_backend.py Outdated

Fixes to Gemini comments

6372a38

Signed-off-by: Ofer Kiselov Nahman <ofer.kiselovnahman@weka.io>

oferki added 4 commits February 26, 2026 14:44

Merge branch 'dev' into gds_backend_perf

6e6f634

Merge branch 'dev' into gds_backend_perf

fc96d82

Merge branch 'dev' into gds_backend_perf

c2c7ac4

Merge branch 'dev' into gds_backend_perf

ddb6a64

sammshen requested a review from deng451e March 4, 2026 02:50

sammshen approved these changes Mar 4, 2026

View reviewed changes

Merge branch 'dev' into gds_backend_perf

6697fd9

DongDongJu approved these changes Mar 4, 2026

View reviewed changes

DongDongJu enabled auto-merge (squash) March 4, 2026 18:13

github-actions Bot added the full Run comprehensive tests on this PR label Mar 4, 2026

deng451e approved these changes Mar 4, 2026

View reviewed changes

oferki added 7 commits March 4, 2026 20:33

Merge branch 'dev' into gds_backend_perf

1b3e923

Merge branch 'dev' into gds_backend_perf

6a62744

Merge branch 'dev' into gds_backend_perf

d0547f4

Merge branch 'dev' into gds_backend_perf

01faee7

Merge branch 'dev' into gds_backend_perf

c585f3a

Merge branch 'dev' into gds_backend_perf

7d1fd1f

Merge branch 'dev' into gds_backend_perf

61f2a37

DongDongJu merged commit 81c9472 into LMCache:dev Mar 12, 2026
27 of 28 checks passed

oferki deleted the gds_backend_perf branch March 12, 2026 05:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Perf] [GDS] Performance improvements to GDS backend#2637

[Perf] [GDS] Performance improvements to GDS backend#2637
DongDongJu merged 14 commits intoLMCache:devfrom
oferki:gds_backend_perf

oferki commented Feb 25, 2026

Uh oh!

gemini-code-assist Bot commented Feb 25, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

sammshen commented Feb 26, 2026

Uh oh!

sammshen left a comment

Uh oh!

ekaynar commented Mar 4, 2026

Uh oh!

DongDongJu left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

oferki commented Feb 25, 2026

Uh oh!

gemini-code-assist Bot commented Feb 25, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

sammshen commented Feb 26, 2026

Uh oh!

sammshen left a comment

Choose a reason for hiding this comment

Uh oh!

ekaynar commented Mar 4, 2026

Uh oh!

DongDongJu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants