feat(gds): enable parallel I/O thread pool for all cuFile filesystems by glimchb · Pull Request #2802 · LMCache/LMCache

glimchb · 2026-03-17T19:53:55Z

Previously the thread pool for batched_get_blocking was only created on WekaFS, so cuFile on ext4, xfs, or other filesystems fell back to serial reads.

Move use_thread_pool out of the WekaFS branch so it is set whenever cuFile is active, regardless of filesystem type. Also rename the thread-name prefix from "weka-gds-io" to "gds-io" to reflect the broader scope.

What this PR does / why we need it:

Special notes for your reviewers:

If applicable:

this PR contains user facing changes - docs added
this PR contains unit tests

gemini-code-assist · 2026-03-17T19:54:14Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the performance of I/O operations by extending the use of a parallel I/O thread pool to all filesystems that utilize cuFile. Previously, this optimization was restricted to WekaFS. The change ensures that cuFile operations benefit from parallel processing across a wider range of storage configurations, leading to more efficient data handling.

Highlights

Parallel I/O for all cuFile filesystems: Enabled the parallel I/O thread pool for all filesystems where cuFile is active, removing the previous limitation to WekaFS only.
Thread pool activation logic: Refactored the logic to activate the use_thread_pool flag whenever cuFile is in use, ensuring broader applicability.
Thread name prefix: Renamed the thread pool's name prefix from 'weka-gds-io' to 'gds-io' to reflect its expanded scope beyond WekaFS.

Changelog

lmcache/v1/storage_backend/gds_backend.py
- Removed the initial explicit setting of self.use_thread_pool = False.
- Modified the assignment of self.use_thread_pool to be dependent on self.use_cufile globally, rather than only within the WekaFS-specific block.
- Updated the thread_name_prefix for the ThreadPoolExecutor from 'weka-gds-io' to 'gds-io'.

Activity

No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request enables the parallel I/O thread pool for all cuFile-enabled filesystems, not just WekaFS. This is achieved by tying the use_thread_pool flag to use_cufile. The thread name prefix is also made more generic. The changes are logical and correct. However, enabling this feature more broadly reveals a potential issue in the thread pool implementation (_batched_get_blocking_by_thread_pool_impl) where the memory format is not correctly handled during allocation. I've added a comment with details on this.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7b7336239a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

sammshen

LGTM

DongDongJu

LGTM. please revise two things.

deng451e

LGTM

DongDongJu

Thanks for the work. LGTM

Previously the thread pool for batched_get_blocking was only created on WekaFS, so cuFile on ext4, xfs, or other filesystems fell back to serial reads. Move `use_thread_pool` out of the WekaFS branch so it is set whenever cuFile is active, regardless of filesystem type. Also rename the thread-name prefix from "weka-gds-io" to "gds-io" to reflect the broader scope. Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>

The thread pool implementation of batched_get_blocking was not passing the fmt parameter to allocate(), causing all allocations to use the default memory format regardless of what was stored in the cache. Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>

…LMCache#2802) * feat(gds): enable parallel I/O thread pool for all cuFile filesystems Previously the thread pool for batched_get_blocking was only created on WekaFS, so cuFile on ext4, xfs, or other filesystems fell back to serial reads. Move `use_thread_pool` out of the WekaFS branch so it is set whenever cuFile is active, regardless of filesystem type. Also rename the thread-name prefix from "weka-gds-io" to "gds-io" to reflect the broader scope. Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com> * fix(gds): pass memory format in batched_get_blocking thread pool path The thread pool implementation of batched_get_blocking was not passing the fmt parameter to allocate(), causing all allocations to use the default memory format regardless of what was stored in the cache. Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com> --------- Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>

gemini-code-assist Bot reviewed Mar 17, 2026

View reviewed changes

Comment thread lmcache/v1/storage_backend/gds_backend.py Outdated

chatgpt-codex-connector Bot reviewed Mar 17, 2026

View reviewed changes

Comment thread lmcache/v1/storage_backend/gds_backend.py Outdated

sammshen approved these changes Mar 20, 2026

View reviewed changes

sammshen requested a review from deng451e March 20, 2026 01:55

DongDongJu requested changes Mar 25, 2026

View reviewed changes

Comment thread lmcache/v1/storage_backend/gds_backend.py Outdated

Comment thread lmcache/v1/storage_backend/gds_backend.py Outdated

glimchb force-pushed the gds-thread branch from 7b73362 to 77df7b7 Compare March 27, 2026 12:05

glimchb requested a review from DongDongJu March 27, 2026 12:06

deng451e approved these changes Mar 27, 2026

View reviewed changes

DongDongJu approved these changes Mar 27, 2026

View reviewed changes

DongDongJu enabled auto-merge (squash) March 27, 2026 20:57

github-actions Bot added the full Run comprehensive tests on this PR label Mar 27, 2026

auto-merge was automatically disabled March 28, 2026 00:30
Head branch was pushed to by a user without write access

glimchb force-pushed the gds-thread branch from 77df7b7 to f9bcfb3 Compare March 28, 2026 00:30

github-actions Bot removed the full Run comprehensive tests on this PR label Mar 28, 2026

glimchb and others added 2 commits March 27, 2026 23:17

glimchb force-pushed the gds-thread branch from f9bcfb3 to 909abec Compare March 28, 2026 03:17

deng451e enabled auto-merge (squash) March 28, 2026 05:52

github-actions Bot added the full Run comprehensive tests on this PR label Mar 28, 2026

deng451e merged commit f6a9919 into LMCache:dev Mar 28, 2026
34 checks passed

glimchb deleted the gds-thread branch March 28, 2026 10:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(gds): enable parallel I/O thread pool for all cuFile filesystems#2802

feat(gds): enable parallel I/O thread pool for all cuFile filesystems#2802
deng451e merged 2 commits intoLMCache:devfrom
glimchb:gds-thread

glimchb commented Mar 17, 2026

Uh oh!

gemini-code-assist Bot commented Mar 17, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

sammshen left a comment

Uh oh!

DongDongJu left a comment

Uh oh!

Uh oh!

Uh oh!

deng451e left a comment

Uh oh!

DongDongJu left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

glimchb commented Mar 17, 2026

Uh oh!

gemini-code-assist Bot commented Mar 17, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

sammshen left a comment

Choose a reason for hiding this comment

Uh oh!

DongDongJu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

deng451e left a comment

Choose a reason for hiding this comment

Uh oh!

DongDongJu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants