gc optimize#3843
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.OpenSSF Scorecard
Scanned Files
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR implements garbage collection optimizations for the Rooch blockchain pruner. The main focus is on improving performance through batched I/O operations and parallel processing with work-stealing. The changes introduce a new AtomicBloomFilterMarker for thread-safe node marking, refactor the reachability analysis to use batch reads, and add a parallel implementation with work-stealing for multi-core systems.
Key changes:
- Added
multi_getbatch read method toNodeDBStorefor efficient I/O - Implemented
AtomicBloomFilterMarkerusing atomic operations for lock-free parallel marking - Refactored single-threaded reachability to use batch I/O processing
- Added parallel reachability builder with work-stealing using crossbeam-deque
- Updated GC integration to choose between parallel and single-threaded modes
- Added comprehensive test coverage for parallel execution scenarios
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| moveos/moveos-store/src/state_store/mod.rs | Added multi_get method for batch node retrieval to optimize I/O |
| crates/rooch-pruner/src/marker.rs | Implemented AtomicBloomFilterMarker with lock-free atomic operations for parallel GC |
| crates/rooch-pruner/src/reachability.rs | Refactored reachability analysis with batch I/O and added parallel work-stealing implementation |
| crates/rooch-pruner/src/garbage_collector.rs | Updated to conditionally use parallel or single-threaded reachability based on worker count |
| crates/rooch-pruner/src/tests/reachability_correctness.rs | Added extensive tests for parallel execution, work stealing, and consistency validation |
| crates/rooch-pruner/src/tests/gc_integration_test.rs | Updated expected marker type to reflect AtomicBloomFilter usage |
| crates/rooch-pruner/Cargo.toml | Added crossbeam-deque dependency for work-stealing queues |
| Cargo.lock | Updated lock file with crossbeam-deque 0.8 dependency |
|
Docker images for this PR are available:
Pull commands:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Summary about this PR