[2/N][Feat] Add zero-copy aligned buffer odirect#2573
Merged
DongDongJu merged 50 commits intoLMCache:devfrom Feb 24, 2026
Merged
[2/N][Feat] Add zero-copy aligned buffer odirect#2573DongDongJu merged 50 commits intoLMCache:devfrom
DongDongJu merged 50 commits intoLMCache:devfrom
Conversation
This commit introduces a simplified Rust Raw Block Backend implementation that focuses on core read/write functionality without complex scheduling or multi-queue features. Key features: - Direct block device I/O via Rust extension (RawBlockDevice) - O_DIRECT support for bypassing page cache - Manifest persistence for crash recovery - LRU eviction policy - Single device support (no multi-FD pools) - Async write operations using asyncio.to_thread - Blocking read operations via pread_into The implementation is simplified compared to previous versions: - No RawBlockScheduler or RawBlockDevicePool - No prefetch implementation (batched_get_non_blocking returns empty list) - Single device handle per backend instance - Minimal configuration (no io_fds, io_workers parameters) This backend currently supports TP=1 (single GPU) deployments only. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com>
Add warning when storage_plugins is configured but extra_config is empty, providing clearer guidance to users about required configuration. This helps prevent silent failures when storage plugins are specified but not properly configured. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com>
This commit adds the core Rust extension module for direct block device I/O operations used by the Rust Raw Block Backend. Key components: - RawBlockDevice: Single-FD synchronous block device interface - Direct pwrite/pread operations with O_DIRECT support - Zero-copy buffer protocol integration - Aligned buffer allocation for O_DIRECT I/O - RawBlockDevicePool: Multi-FD pool for parallel I/O - Per-CPU FD selection for parallelism - Multiple file descriptors per device - RawBlockScheduler: Async I/O scheduler with worker threads - Priority-based task queues (High/Medium/Low) - Worker threads with CPU pinning - Completion queue with GIL-safe processing The Rust extension provides: - Direct block device access via libc pwrite/pread - O_DIRECT support with automatic alignment handling - Python buffer protocol integration for zero-copy operations - Thread-safe I/O operations with proper GIL management - Error handling with proper Python exception propagation This is the core I/O layer that the Python backend uses for all block device operations. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com>
This commit adds comprehensive tests for the Rust Raw Block Backend implementation. Test coverage includes: - Basic store and retrieve operations - Manifest persistence and recovery - LRU eviction behavior - O_DIRECT alignment handling - Error handling for invalid configurations - Edge cases (empty writes, large chunks) The tests verify both functional correctness and integration with the Rust extension module. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com>
- Add try-except blocks for I/O operations with proper error logging - Fix test code to use correct API (torch.Size and pwrite_from_buffer) Signed-off-by: DongDongJu <commisori28@gmail.com>
- Remove RawBlockDevicePool (multi-FD pool) - Remove RawBlockScheduler (async I/O scheduler) - Remove batch operations and priority queue features - Keep only RawBlockDevice with basic pwrite_from_buffer and pread_into This simplifies the codebase to only include features actually used by the Python backend. Signed-off-by: DongDongJu <commisori28@gmail.com>
- Fix misleading 'zero-copy' comment (actually uses bounce buffers for O_DIRECT) - Remove unused _decode_header method - Remove unimplemented batched_get_non_blocking method - Update remove() method comment to reflect actual behavior - Improve docstrings for accuracy Signed-off-by: DongDongJu <commisori28@gmail.com>
- Fix ruff E501 line length errors in docstrings - Format Rust code with cargo fmt - Fix clippy warnings (unused mut, dead code, manual_div_ceil) - Add Rust fmt and clippy checks to pre-commit hooks Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
Signed-off-by: DongDongJu <commisori28@gmail.com>
hickeyma
approved these changes
Feb 23, 2026
Collaborator
Author
|
Hello @Shaoting-Feng, IDK what happened for buildkite/vllm-integration-tests. Could you help me to debug this? |
sammshen
pushed a commit
to sammshen/LMCache
that referenced
this pull request
Mar 1, 2026
* feat: Add simplified Rust Raw Block Backend This commit introduces a simplified Rust Raw Block Backend implementation that focuses on core read/write functionality without complex scheduling or multi-queue features. Key features: - Direct block device I/O via Rust extension (RawBlockDevice) - O_DIRECT support for bypassing page cache - Manifest persistence for crash recovery - LRU eviction policy - Single device support (no multi-FD pools) - Async write operations using asyncio.to_thread - Blocking read operations via pread_into The implementation is simplified compared to previous versions: - No RawBlockScheduler or RawBlockDevicePool - No prefetch implementation (batched_get_non_blocking returns empty list) - Single device handle per backend instance - Minimal configuration (no io_fds, io_workers parameters) This backend currently supports TP=1 (single GPU) deployments only. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * fix: Improve storage plugin launcher error handling Add warning when storage_plugins is configured but extra_config is empty, providing clearer guidance to users about required configuration. This helps prevent silent failures when storage plugins are specified but not properly configured. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * feat: Add Rust raw block I/O extension This commit adds the core Rust extension module for direct block device I/O operations used by the Rust Raw Block Backend. Key components: - RawBlockDevice: Single-FD synchronous block device interface - Direct pwrite/pread operations with O_DIRECT support - Zero-copy buffer protocol integration - Aligned buffer allocation for O_DIRECT I/O - RawBlockDevicePool: Multi-FD pool for parallel I/O - Per-CPU FD selection for parallelism - Multiple file descriptors per device - RawBlockScheduler: Async I/O scheduler with worker threads - Priority-based task queues (High/Medium/Low) - Worker threads with CPU pinning - Completion queue with GIL-safe processing The Rust extension provides: - Direct block device access via libc pwrite/pread - O_DIRECT support with automatic alignment handling - Python buffer protocol integration for zero-copy operations - Thread-safe I/O operations with proper GIL management - Error handling with proper Python exception propagation This is the core I/O layer that the Python backend uses for all block device operations. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * test: Add tests for Rust Raw Block Backend This commit adds comprehensive tests for the Rust Raw Block Backend implementation. Test coverage includes: - Basic store and retrieve operations - Manifest persistence and recovery - LRU eviction behavior - O_DIRECT alignment handling - Error handling for invalid configurations - Edge cases (empty writes, large chunks) The tests verify both functional correctness and integration with the Rust extension module. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * Improve error handling in Rust Raw Block Backend - Add try-except blocks for I/O operations with proper error logging - Fix test code to use correct API (torch.Size and pwrite_from_buffer) Signed-off-by: DongDongJu <commisori28@gmail.com> * Simplify Rust raw block I/O: remove unused features - Remove RawBlockDevicePool (multi-FD pool) - Remove RawBlockScheduler (async I/O scheduler) - Remove batch operations and priority queue features - Keep only RawBlockDevice with basic pwrite_from_buffer and pread_into This simplifies the codebase to only include features actually used by the Python backend. Signed-off-by: DongDongJu <commisori28@gmail.com> * Clean up comments and remove unused code in RustRawBlockBackend - Fix misleading 'zero-copy' comment (actually uses bounce buffers for O_DIRECT) - Remove unused _decode_header method - Remove unimplemented batched_get_non_blocking method - Update remove() method comment to reflect actual behavior - Improve docstrings for accuracy Signed-off-by: DongDongJu <commisori28@gmail.com> * Fix style issues and add Rust style checks - Fix ruff E501 line length errors in docstrings - Format Rust code with cargo fmt - Fix clippy warnings (unused mut, dead code, manual_div_ceil) - Add Rust fmt and clippy checks to pre-commit hooks Signed-off-by: DongDongJu <commisori28@gmail.com> * [CI] Allow crates.io egress for Rust Clippy Signed-off-by: DongDongJu <commisori28@gmail.com> * [Fix] Defer inflight slot reuse to avoid corruption Signed-off-by: DongDongJu <commisori28@gmail.com> * fix style error Signed-off-by: DongDongJu <commisori28@gmail.com> * [Test] Update raw block tests for new metadata Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Add Rust raw block I/O guide Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Fix rustfmt alignment Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Update raw block README usage Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Clarify raw block I/O behavior Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Add storage backend I/O benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Add sample results for I/O benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Allow output directory for JSON results Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Document output-dir sanity run Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Use same filesystem for raw block fallback Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Fix ruff line length in benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Wrap benchmark description string Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Align buffers for O_DIRECT comparison Signed-off-by: DongDongJu <commisori28@gmail.com> * Enable O_DIRECT zero-copy fast path for rust raw backend - add direct aligned O_DIRECT read/write path in Rust extension with bounce fallback - wire LocalCPUBackend allocator alignment to rust_raw_block O_DIRECT settings - add allocator alignment tests and update raw_block README Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * Fix mypy typing issues in allocator alignment changes - fix MixedMemoryAllocator paging alignment typing - avoid mixed-type kwargs path for MixedMemoryAllocator construction - keep tests formatted after isort/pre-commit Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * local_cpu: remove experimental lazy allocator path and simplify alignment Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * bench: harden storage backend benchmark execution Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * docs: update benchmark results and raw block zero-copy notes Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust/raw_block: add reviewer-focused comments for io paths Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust_raw_block: add async batch submission mode Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust_raw_block: remove async batch submission mode Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * docs: remove fixed benchmark numbers and keep reproducible NVMe comparison steps Signed-off-by: DongDongJu <commisori28@gmail.com> * bench: scale local_disk timeout with num_ops for long runs Signed-off-by: DongDongJu <commisori28@gmail.com> --------- Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com>
hlin99
pushed a commit
to hlin99/LMCache
that referenced
this pull request
Mar 2, 2026
* feat: Add simplified Rust Raw Block Backend This commit introduces a simplified Rust Raw Block Backend implementation that focuses on core read/write functionality without complex scheduling or multi-queue features. Key features: - Direct block device I/O via Rust extension (RawBlockDevice) - O_DIRECT support for bypassing page cache - Manifest persistence for crash recovery - LRU eviction policy - Single device support (no multi-FD pools) - Async write operations using asyncio.to_thread - Blocking read operations via pread_into The implementation is simplified compared to previous versions: - No RawBlockScheduler or RawBlockDevicePool - No prefetch implementation (batched_get_non_blocking returns empty list) - Single device handle per backend instance - Minimal configuration (no io_fds, io_workers parameters) This backend currently supports TP=1 (single GPU) deployments only. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * fix: Improve storage plugin launcher error handling Add warning when storage_plugins is configured but extra_config is empty, providing clearer guidance to users about required configuration. This helps prevent silent failures when storage plugins are specified but not properly configured. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * feat: Add Rust raw block I/O extension This commit adds the core Rust extension module for direct block device I/O operations used by the Rust Raw Block Backend. Key components: - RawBlockDevice: Single-FD synchronous block device interface - Direct pwrite/pread operations with O_DIRECT support - Zero-copy buffer protocol integration - Aligned buffer allocation for O_DIRECT I/O - RawBlockDevicePool: Multi-FD pool for parallel I/O - Per-CPU FD selection for parallelism - Multiple file descriptors per device - RawBlockScheduler: Async I/O scheduler with worker threads - Priority-based task queues (High/Medium/Low) - Worker threads with CPU pinning - Completion queue with GIL-safe processing The Rust extension provides: - Direct block device access via libc pwrite/pread - O_DIRECT support with automatic alignment handling - Python buffer protocol integration for zero-copy operations - Thread-safe I/O operations with proper GIL management - Error handling with proper Python exception propagation This is the core I/O layer that the Python backend uses for all block device operations. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * test: Add tests for Rust Raw Block Backend This commit adds comprehensive tests for the Rust Raw Block Backend implementation. Test coverage includes: - Basic store and retrieve operations - Manifest persistence and recovery - LRU eviction behavior - O_DIRECT alignment handling - Error handling for invalid configurations - Edge cases (empty writes, large chunks) The tests verify both functional correctness and integration with the Rust extension module. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * Improve error handling in Rust Raw Block Backend - Add try-except blocks for I/O operations with proper error logging - Fix test code to use correct API (torch.Size and pwrite_from_buffer) Signed-off-by: DongDongJu <commisori28@gmail.com> * Simplify Rust raw block I/O: remove unused features - Remove RawBlockDevicePool (multi-FD pool) - Remove RawBlockScheduler (async I/O scheduler) - Remove batch operations and priority queue features - Keep only RawBlockDevice with basic pwrite_from_buffer and pread_into This simplifies the codebase to only include features actually used by the Python backend. Signed-off-by: DongDongJu <commisori28@gmail.com> * Clean up comments and remove unused code in RustRawBlockBackend - Fix misleading 'zero-copy' comment (actually uses bounce buffers for O_DIRECT) - Remove unused _decode_header method - Remove unimplemented batched_get_non_blocking method - Update remove() method comment to reflect actual behavior - Improve docstrings for accuracy Signed-off-by: DongDongJu <commisori28@gmail.com> * Fix style issues and add Rust style checks - Fix ruff E501 line length errors in docstrings - Format Rust code with cargo fmt - Fix clippy warnings (unused mut, dead code, manual_div_ceil) - Add Rust fmt and clippy checks to pre-commit hooks Signed-off-by: DongDongJu <commisori28@gmail.com> * [CI] Allow crates.io egress for Rust Clippy Signed-off-by: DongDongJu <commisori28@gmail.com> * [Fix] Defer inflight slot reuse to avoid corruption Signed-off-by: DongDongJu <commisori28@gmail.com> * fix style error Signed-off-by: DongDongJu <commisori28@gmail.com> * [Test] Update raw block tests for new metadata Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Add Rust raw block I/O guide Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Fix rustfmt alignment Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Update raw block README usage Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Clarify raw block I/O behavior Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Add storage backend I/O benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Add sample results for I/O benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Allow output directory for JSON results Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Document output-dir sanity run Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Use same filesystem for raw block fallback Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Fix ruff line length in benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Wrap benchmark description string Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Align buffers for O_DIRECT comparison Signed-off-by: DongDongJu <commisori28@gmail.com> * Enable O_DIRECT zero-copy fast path for rust raw backend - add direct aligned O_DIRECT read/write path in Rust extension with bounce fallback - wire LocalCPUBackend allocator alignment to rust_raw_block O_DIRECT settings - add allocator alignment tests and update raw_block README Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * Fix mypy typing issues in allocator alignment changes - fix MixedMemoryAllocator paging alignment typing - avoid mixed-type kwargs path for MixedMemoryAllocator construction - keep tests formatted after isort/pre-commit Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * local_cpu: remove experimental lazy allocator path and simplify alignment Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * bench: harden storage backend benchmark execution Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * docs: update benchmark results and raw block zero-copy notes Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust/raw_block: add reviewer-focused comments for io paths Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust_raw_block: add async batch submission mode Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust_raw_block: remove async batch submission mode Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * docs: remove fixed benchmark numbers and keep reproducible NVMe comparison steps Signed-off-by: DongDongJu <commisori28@gmail.com> * bench: scale local_disk timeout with num_ops for long runs Signed-off-by: DongDongJu <commisori28@gmail.com> --------- Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com>
mauryaavinash95
pushed a commit
to mauryaavinash95/LMCache
that referenced
this pull request
Mar 7, 2026
* feat: Add simplified Rust Raw Block Backend This commit introduces a simplified Rust Raw Block Backend implementation that focuses on core read/write functionality without complex scheduling or multi-queue features. Key features: - Direct block device I/O via Rust extension (RawBlockDevice) - O_DIRECT support for bypassing page cache - Manifest persistence for crash recovery - LRU eviction policy - Single device support (no multi-FD pools) - Async write operations using asyncio.to_thread - Blocking read operations via pread_into The implementation is simplified compared to previous versions: - No RawBlockScheduler or RawBlockDevicePool - No prefetch implementation (batched_get_non_blocking returns empty list) - Single device handle per backend instance - Minimal configuration (no io_fds, io_workers parameters) This backend currently supports TP=1 (single GPU) deployments only. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * fix: Improve storage plugin launcher error handling Add warning when storage_plugins is configured but extra_config is empty, providing clearer guidance to users about required configuration. This helps prevent silent failures when storage plugins are specified but not properly configured. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * feat: Add Rust raw block I/O extension This commit adds the core Rust extension module for direct block device I/O operations used by the Rust Raw Block Backend. Key components: - RawBlockDevice: Single-FD synchronous block device interface - Direct pwrite/pread operations with O_DIRECT support - Zero-copy buffer protocol integration - Aligned buffer allocation for O_DIRECT I/O - RawBlockDevicePool: Multi-FD pool for parallel I/O - Per-CPU FD selection for parallelism - Multiple file descriptors per device - RawBlockScheduler: Async I/O scheduler with worker threads - Priority-based task queues (High/Medium/Low) - Worker threads with CPU pinning - Completion queue with GIL-safe processing The Rust extension provides: - Direct block device access via libc pwrite/pread - O_DIRECT support with automatic alignment handling - Python buffer protocol integration for zero-copy operations - Thread-safe I/O operations with proper GIL management - Error handling with proper Python exception propagation This is the core I/O layer that the Python backend uses for all block device operations. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * test: Add tests for Rust Raw Block Backend This commit adds comprehensive tests for the Rust Raw Block Backend implementation. Test coverage includes: - Basic store and retrieve operations - Manifest persistence and recovery - LRU eviction behavior - O_DIRECT alignment handling - Error handling for invalid configurations - Edge cases (empty writes, large chunks) The tests verify both functional correctness and integration with the Rust extension module. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * Improve error handling in Rust Raw Block Backend - Add try-except blocks for I/O operations with proper error logging - Fix test code to use correct API (torch.Size and pwrite_from_buffer) Signed-off-by: DongDongJu <commisori28@gmail.com> * Simplify Rust raw block I/O: remove unused features - Remove RawBlockDevicePool (multi-FD pool) - Remove RawBlockScheduler (async I/O scheduler) - Remove batch operations and priority queue features - Keep only RawBlockDevice with basic pwrite_from_buffer and pread_into This simplifies the codebase to only include features actually used by the Python backend. Signed-off-by: DongDongJu <commisori28@gmail.com> * Clean up comments and remove unused code in RustRawBlockBackend - Fix misleading 'zero-copy' comment (actually uses bounce buffers for O_DIRECT) - Remove unused _decode_header method - Remove unimplemented batched_get_non_blocking method - Update remove() method comment to reflect actual behavior - Improve docstrings for accuracy Signed-off-by: DongDongJu <commisori28@gmail.com> * Fix style issues and add Rust style checks - Fix ruff E501 line length errors in docstrings - Format Rust code with cargo fmt - Fix clippy warnings (unused mut, dead code, manual_div_ceil) - Add Rust fmt and clippy checks to pre-commit hooks Signed-off-by: DongDongJu <commisori28@gmail.com> * [CI] Allow crates.io egress for Rust Clippy Signed-off-by: DongDongJu <commisori28@gmail.com> * [Fix] Defer inflight slot reuse to avoid corruption Signed-off-by: DongDongJu <commisori28@gmail.com> * fix style error Signed-off-by: DongDongJu <commisori28@gmail.com> * [Test] Update raw block tests for new metadata Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Add Rust raw block I/O guide Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Fix rustfmt alignment Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Update raw block README usage Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Clarify raw block I/O behavior Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Add storage backend I/O benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Add sample results for I/O benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Allow output directory for JSON results Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Document output-dir sanity run Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Use same filesystem for raw block fallback Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Fix ruff line length in benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Wrap benchmark description string Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Align buffers for O_DIRECT comparison Signed-off-by: DongDongJu <commisori28@gmail.com> * Enable O_DIRECT zero-copy fast path for rust raw backend - add direct aligned O_DIRECT read/write path in Rust extension with bounce fallback - wire LocalCPUBackend allocator alignment to rust_raw_block O_DIRECT settings - add allocator alignment tests and update raw_block README Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * Fix mypy typing issues in allocator alignment changes - fix MixedMemoryAllocator paging alignment typing - avoid mixed-type kwargs path for MixedMemoryAllocator construction - keep tests formatted after isort/pre-commit Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * local_cpu: remove experimental lazy allocator path and simplify alignment Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * bench: harden storage backend benchmark execution Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * docs: update benchmark results and raw block zero-copy notes Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust/raw_block: add reviewer-focused comments for io paths Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust_raw_block: add async batch submission mode Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust_raw_block: remove async batch submission mode Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * docs: remove fixed benchmark numbers and keep reproducible NVMe comparison steps Signed-off-by: DongDongJu <commisori28@gmail.com> * bench: scale local_disk timeout with num_ops for long runs Signed-off-by: DongDongJu <commisori28@gmail.com> --------- Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com>
shaoxiawjc
pushed a commit
to shaoxiawjc/LMCache
that referenced
this pull request
Mar 11, 2026
* feat: Add simplified Rust Raw Block Backend This commit introduces a simplified Rust Raw Block Backend implementation that focuses on core read/write functionality without complex scheduling or multi-queue features. Key features: - Direct block device I/O via Rust extension (RawBlockDevice) - O_DIRECT support for bypassing page cache - Manifest persistence for crash recovery - LRU eviction policy - Single device support (no multi-FD pools) - Async write operations using asyncio.to_thread - Blocking read operations via pread_into The implementation is simplified compared to previous versions: - No RawBlockScheduler or RawBlockDevicePool - No prefetch implementation (batched_get_non_blocking returns empty list) - Single device handle per backend instance - Minimal configuration (no io_fds, io_workers parameters) This backend currently supports TP=1 (single GPU) deployments only. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * fix: Improve storage plugin launcher error handling Add warning when storage_plugins is configured but extra_config is empty, providing clearer guidance to users about required configuration. This helps prevent silent failures when storage plugins are specified but not properly configured. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * feat: Add Rust raw block I/O extension This commit adds the core Rust extension module for direct block device I/O operations used by the Rust Raw Block Backend. Key components: - RawBlockDevice: Single-FD synchronous block device interface - Direct pwrite/pread operations with O_DIRECT support - Zero-copy buffer protocol integration - Aligned buffer allocation for O_DIRECT I/O - RawBlockDevicePool: Multi-FD pool for parallel I/O - Per-CPU FD selection for parallelism - Multiple file descriptors per device - RawBlockScheduler: Async I/O scheduler with worker threads - Priority-based task queues (High/Medium/Low) - Worker threads with CPU pinning - Completion queue with GIL-safe processing The Rust extension provides: - Direct block device access via libc pwrite/pread - O_DIRECT support with automatic alignment handling - Python buffer protocol integration for zero-copy operations - Thread-safe I/O operations with proper GIL management - Error handling with proper Python exception propagation This is the core I/O layer that the Python backend uses for all block device operations. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * test: Add tests for Rust Raw Block Backend This commit adds comprehensive tests for the Rust Raw Block Backend implementation. Test coverage includes: - Basic store and retrieve operations - Manifest persistence and recovery - LRU eviction behavior - O_DIRECT alignment handling - Error handling for invalid configurations - Edge cases (empty writes, large chunks) The tests verify both functional correctness and integration with the Rust extension module. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * Improve error handling in Rust Raw Block Backend - Add try-except blocks for I/O operations with proper error logging - Fix test code to use correct API (torch.Size and pwrite_from_buffer) Signed-off-by: DongDongJu <commisori28@gmail.com> * Simplify Rust raw block I/O: remove unused features - Remove RawBlockDevicePool (multi-FD pool) - Remove RawBlockScheduler (async I/O scheduler) - Remove batch operations and priority queue features - Keep only RawBlockDevice with basic pwrite_from_buffer and pread_into This simplifies the codebase to only include features actually used by the Python backend. Signed-off-by: DongDongJu <commisori28@gmail.com> * Clean up comments and remove unused code in RustRawBlockBackend - Fix misleading 'zero-copy' comment (actually uses bounce buffers for O_DIRECT) - Remove unused _decode_header method - Remove unimplemented batched_get_non_blocking method - Update remove() method comment to reflect actual behavior - Improve docstrings for accuracy Signed-off-by: DongDongJu <commisori28@gmail.com> * Fix style issues and add Rust style checks - Fix ruff E501 line length errors in docstrings - Format Rust code with cargo fmt - Fix clippy warnings (unused mut, dead code, manual_div_ceil) - Add Rust fmt and clippy checks to pre-commit hooks Signed-off-by: DongDongJu <commisori28@gmail.com> * [CI] Allow crates.io egress for Rust Clippy Signed-off-by: DongDongJu <commisori28@gmail.com> * [Fix] Defer inflight slot reuse to avoid corruption Signed-off-by: DongDongJu <commisori28@gmail.com> * fix style error Signed-off-by: DongDongJu <commisori28@gmail.com> * [Test] Update raw block tests for new metadata Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Add Rust raw block I/O guide Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Fix rustfmt alignment Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Update raw block README usage Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Clarify raw block I/O behavior Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Add storage backend I/O benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Add sample results for I/O benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Allow output directory for JSON results Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Document output-dir sanity run Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Use same filesystem for raw block fallback Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Fix ruff line length in benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Wrap benchmark description string Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Align buffers for O_DIRECT comparison Signed-off-by: DongDongJu <commisori28@gmail.com> * Enable O_DIRECT zero-copy fast path for rust raw backend - add direct aligned O_DIRECT read/write path in Rust extension with bounce fallback - wire LocalCPUBackend allocator alignment to rust_raw_block O_DIRECT settings - add allocator alignment tests and update raw_block README Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * Fix mypy typing issues in allocator alignment changes - fix MixedMemoryAllocator paging alignment typing - avoid mixed-type kwargs path for MixedMemoryAllocator construction - keep tests formatted after isort/pre-commit Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * local_cpu: remove experimental lazy allocator path and simplify alignment Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * bench: harden storage backend benchmark execution Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * docs: update benchmark results and raw block zero-copy notes Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust/raw_block: add reviewer-focused comments for io paths Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust_raw_block: add async batch submission mode Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust_raw_block: remove async batch submission mode Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * docs: remove fixed benchmark numbers and keep reproducible NVMe comparison steps Signed-off-by: DongDongJu <commisori28@gmail.com> * bench: scale local_disk timeout with num_ops for long runs Signed-off-by: DongDongJu <commisori28@gmail.com> --------- Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> Signed-off-by: shaoxiawjc <wjc2800@163.com>
realAaronWu
pushed a commit
to realAaronWu/LMCache
that referenced
this pull request
Mar 20, 2026
* feat: Add simplified Rust Raw Block Backend This commit introduces a simplified Rust Raw Block Backend implementation that focuses on core read/write functionality without complex scheduling or multi-queue features. Key features: - Direct block device I/O via Rust extension (RawBlockDevice) - O_DIRECT support for bypassing page cache - Manifest persistence for crash recovery - LRU eviction policy - Single device support (no multi-FD pools) - Async write operations using asyncio.to_thread - Blocking read operations via pread_into The implementation is simplified compared to previous versions: - No RawBlockScheduler or RawBlockDevicePool - No prefetch implementation (batched_get_non_blocking returns empty list) - Single device handle per backend instance - Minimal configuration (no io_fds, io_workers parameters) This backend currently supports TP=1 (single GPU) deployments only. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * fix: Improve storage plugin launcher error handling Add warning when storage_plugins is configured but extra_config is empty, providing clearer guidance to users about required configuration. This helps prevent silent failures when storage plugins are specified but not properly configured. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * feat: Add Rust raw block I/O extension This commit adds the core Rust extension module for direct block device I/O operations used by the Rust Raw Block Backend. Key components: - RawBlockDevice: Single-FD synchronous block device interface - Direct pwrite/pread operations with O_DIRECT support - Zero-copy buffer protocol integration - Aligned buffer allocation for O_DIRECT I/O - RawBlockDevicePool: Multi-FD pool for parallel I/O - Per-CPU FD selection for parallelism - Multiple file descriptors per device - RawBlockScheduler: Async I/O scheduler with worker threads - Priority-based task queues (High/Medium/Low) - Worker threads with CPU pinning - Completion queue with GIL-safe processing The Rust extension provides: - Direct block device access via libc pwrite/pread - O_DIRECT support with automatic alignment handling - Python buffer protocol integration for zero-copy operations - Thread-safe I/O operations with proper GIL management - Error handling with proper Python exception propagation This is the core I/O layer that the Python backend uses for all block device operations. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * test: Add tests for Rust Raw Block Backend This commit adds comprehensive tests for the Rust Raw Block Backend implementation. Test coverage includes: - Basic store and retrieve operations - Manifest persistence and recovery - LRU eviction behavior - O_DIRECT alignment handling - Error handling for invalid configurations - Edge cases (empty writes, large chunks) The tests verify both functional correctness and integration with the Rust extension module. Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * Improve error handling in Rust Raw Block Backend - Add try-except blocks for I/O operations with proper error logging - Fix test code to use correct API (torch.Size and pwrite_from_buffer) Signed-off-by: DongDongJu <commisori28@gmail.com> * Simplify Rust raw block I/O: remove unused features - Remove RawBlockDevicePool (multi-FD pool) - Remove RawBlockScheduler (async I/O scheduler) - Remove batch operations and priority queue features - Keep only RawBlockDevice with basic pwrite_from_buffer and pread_into This simplifies the codebase to only include features actually used by the Python backend. Signed-off-by: DongDongJu <commisori28@gmail.com> * Clean up comments and remove unused code in RustRawBlockBackend - Fix misleading 'zero-copy' comment (actually uses bounce buffers for O_DIRECT) - Remove unused _decode_header method - Remove unimplemented batched_get_non_blocking method - Update remove() method comment to reflect actual behavior - Improve docstrings for accuracy Signed-off-by: DongDongJu <commisori28@gmail.com> * Fix style issues and add Rust style checks - Fix ruff E501 line length errors in docstrings - Format Rust code with cargo fmt - Fix clippy warnings (unused mut, dead code, manual_div_ceil) - Add Rust fmt and clippy checks to pre-commit hooks Signed-off-by: DongDongJu <commisori28@gmail.com> * [CI] Allow crates.io egress for Rust Clippy Signed-off-by: DongDongJu <commisori28@gmail.com> * [Fix] Defer inflight slot reuse to avoid corruption Signed-off-by: DongDongJu <commisori28@gmail.com> * fix style error Signed-off-by: DongDongJu <commisori28@gmail.com> * [Test] Update raw block tests for new metadata Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Add Rust raw block I/O guide Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Fix rustfmt alignment Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Update raw block README usage Signed-off-by: DongDongJu <commisori28@gmail.com> * [Docs] Clarify raw block I/O behavior Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Add storage backend I/O benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Add sample results for I/O benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Allow output directory for JSON results Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Document output-dir sanity run Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Use same filesystem for raw block fallback Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Fix ruff line length in benchmark Signed-off-by: DongDongJu <commisori28@gmail.com> * [Style] Wrap benchmark description string Signed-off-by: DongDongJu <commisori28@gmail.com> * [Bench] Align buffers for O_DIRECT comparison Signed-off-by: DongDongJu <commisori28@gmail.com> * Enable O_DIRECT zero-copy fast path for rust raw backend - add direct aligned O_DIRECT read/write path in Rust extension with bounce fallback - wire LocalCPUBackend allocator alignment to rust_raw_block O_DIRECT settings - add allocator alignment tests and update raw_block README Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * Fix mypy typing issues in allocator alignment changes - fix MixedMemoryAllocator paging alignment typing - avoid mixed-type kwargs path for MixedMemoryAllocator construction - keep tests formatted after isort/pre-commit Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * local_cpu: remove experimental lazy allocator path and simplify alignment Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * bench: harden storage backend benchmark execution Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * docs: update benchmark results and raw block zero-copy notes Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust/raw_block: add reviewer-focused comments for io paths Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust_raw_block: add async batch submission mode Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * rust_raw_block: remove async batch submission mode Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> * docs: remove fixed benchmark numbers and keep reproducible NVMe comparison steps Signed-off-by: DongDongJu <commisori28@gmail.com> * bench: scale local_disk timeout with num_ops for long runs Signed-off-by: DongDongJu <commisori28@gmail.com> --------- Signed-off-by: DongDongJu <commisori28@gmail.com> Signed-off-by: Dongjoo Seo <dongjoo.seo1@samsung.com> Signed-off-by: Aaron Wu <aaron.wu@dell.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
This PR improves the Rust raw block backend O_DIRECT path and keeps scope focused on zero-copy behavior and correctness.
Main changes:
LocalCPUBackenduses rust raw block alignment knobs (rust_raw_block.block_align,rust_raw_block.align_local_cpu_allocator)local_cpu.pinned_align_bytestruncate()for real block device pathSpecial notes for your reviewers:
io_uringIf applicable: