[MOD-14096] FFI bindings for GC on NumericRangeTree#8401
[MOD-14096] FFI bindings for GC on NumericRangeTree#8401LukeMathWalker merged 2 commits intomasterfrom
Conversation
src/redisearch_rs/c_entrypoint/numeric_range_tree_ffi/src/gc.rs
Outdated
Show resolved
Hide resolved
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #8401 +/- ##
==========================================
+ Coverage 82.51% 82.59% +0.07%
==========================================
Files 421 421
Lines 60938 60948 +10
Branches 18808 18818 +10
==========================================
+ Hits 50284 50339 +55
+ Misses 10463 10418 -45
Partials 191 191
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
b72444e to
5c42040
Compare
c3a70ab to
75f4c36
Compare
5c42040 to
ba1411a
Compare
75f4c36 to
a5870de
Compare
src/redisearch_rs/c_entrypoint/numeric_range_tree_ffi/src/gc.rs
Outdated
Show resolved
Hide resolved
a5870de to
74a9e4b
Compare
ba1411a to
42deea6
Compare
74a9e4b to
0671484
Compare
42deea6 to
9cb1e97
Compare
82552e2 to
7a124f4
Compare
9cb1e97 to
be0e1f9
Compare
7a124f4 to
6d059cf
Compare
src/redisearch_rs/c_entrypoint/numeric_range_tree_ffi/src/lib.rs
Outdated
Show resolved
Hide resolved
6d059cf to
f4b5d96
Compare
be0e1f9 to
f0c7711
Compare
f4b5d96 to
32b2f99
Compare
f0c7711 to
5fc6411
Compare
32b2f99 to
4fd3d40
Compare
|
❌ The last analysis has failed. |
4fd3d40 to
77f8d14
Compare
5fc6411 to
55b3c0b
Compare
77f8d14 to
2170f63
Compare
1411491 to
68977c6
Compare
4e1ca51 to
fa9da32
Compare
12963d4 to
dc9ae47
Compare
fa9da32 to
91d9770
Compare
91d9770 to
7559046
Compare
🛡️ Jit Security Scan Results✅ No security findings were detected in this PR
Security scan by Jit
|
bb7ae68 to
387e96f
Compare
7b89449 to
2f85461
Compare
| #[repr(C)] | ||
| pub struct NumericGcNodeEntry { | ||
| /// The node's slab position. | ||
| pub node_position: u32, |
There was a problem hiding this comment.
sohuldn't the type of this be NodeIndex?
There was a problem hiding this comment.
In #8407 and #8408, we forked slab, the crate that underpins our node arena, to add support for generational indices: each index now tracks the position into the memory vector (one u32) as well as a monotonically increasing counter (another u32), which gets bumped every time that slot in the memory vector is reused.
This prevents us from hitting the ABA problem:
- You allocate a new node in slot 5, getting index 5.
- You remove the node in slot 5, it goes back on the free list.
- You allocate a new node, slot 5 gets reused for a new, unrelated node.
- Some other part of our code still holds the old index 5, thinking it points to the original item.
It reads or writes through that stale index, silently accessing the wrong node.
I improved the docs here to explain how they relate to the index, and the documentation of the node index itself.
Without generational indices, we would have to abort GCing whenever the revision id changes, but that would be way too aggressive.
| /// The node's slab position. | ||
| pub node_position: u32, | ||
| /// The node's slab generation. | ||
| pub node_generation: u32, |
There was a problem hiding this comment.
remind me what node_generation is?
Consider documenting
2115229 to
32b6108
Compare
|



Describe the changes in the pull request
FFI bindings for GC on NumericRangeTree.
Stacked on top of #8460.
Which additional issues this PR fixes
Main objects this PR modified
Mark if applicable
Release Notes
If a release note is required (bug fix / new feature / enhancement), describe the user impact of this PR in the title.
Note
Medium Risk
Adds new C/Rust FFI entrypoints and a custom msgpack wire format for streaming numeric index GC deltas; mistakes here could cause crashes or incorrect GC when integrating with the C-side process/pipes.
Overview
Adds new FFI bindings to run garbage collection on
NumericRangeTreenumeric inverted indexes from C, including streaming per-node GC scanning and applying those deltas back to the tree.Introduces a
NumericGcScannerthat iterates nodes and emits a msgpack-serializedGcScanDeltaplus HLL registers, plusNumericRangeTree_ApplyGcEntryto validate/deserialize that payload, apply it to a specific node (by position+generation), and return an explicit status (Ok/NodeNotFound/DeserializationError). Also exposes tree cleanup helpers (NumericRangeTree_TrimEmptyLeaves,NumericRangeTree_CompactIfSparse) and updates dependencies/headers accordingly.Written by Cursor Bugbot for commit 32b6108. This will update automatically on new commits. Configure here.