Skip to content

[MOD-14096] FFI bindings for GC on NumericRangeTree#8401

Merged
LukeMathWalker merged 2 commits intomasterfrom
numeric-range-tree-2b-gc-ffi
Feb 25, 2026
Merged

[MOD-14096] FFI bindings for GC on NumericRangeTree#8401
LukeMathWalker merged 2 commits intomasterfrom
numeric-range-tree-2b-gc-ffi

Conversation

@LukeMathWalker
Copy link
Copy Markdown
Collaborator

@LukeMathWalker LukeMathWalker commented Feb 17, 2026

Describe the changes in the pull request

FFI bindings for GC on NumericRangeTree.
Stacked on top of #8460.

Which additional issues this PR fixes

  1. MOD-...
  2. #...

Main objects this PR modified

  1. ...

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Release Notes

  • This PR requires release notes
  • This PR does not require release notes

If a release note is required (bug fix / new feature / enhancement), describe the user impact of this PR in the title.


Note

Medium Risk
Adds new C/Rust FFI entrypoints and a custom msgpack wire format for streaming numeric index GC deltas; mistakes here could cause crashes or incorrect GC when integrating with the C-side process/pipes.

Overview
Adds new FFI bindings to run garbage collection on NumericRangeTree numeric inverted indexes from C, including streaming per-node GC scanning and applying those deltas back to the tree.

Introduces a NumericGcScanner that iterates nodes and emits a msgpack-serialized GcScanDelta plus HLL registers, plus NumericRangeTree_ApplyGcEntry to validate/deserialize that payload, apply it to a specific node (by position+generation), and return an explicit status (Ok/NodeNotFound/DeserializationError). Also exposes tree cleanup helpers (NumericRangeTree_TrimEmptyLeaves, NumericRangeTree_CompactIfSparse) and updates dependencies/headers accordingly.

Written by Cursor Bugbot for commit 32b6108. This will update automatically on new commits. Configure here.

@LukeMathWalker LukeMathWalker changed the title FFI bindings for GC on NumericRangeTree [MOD-14096] FFI bindings for GC on NumericRangeTree Feb 17, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Feb 17, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.59%. Comparing base (011a3a7) to head (2115229).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8401      +/-   ##
==========================================
+ Coverage   82.51%   82.59%   +0.07%     
==========================================
  Files         421      421              
  Lines       60938    60948      +10     
  Branches    18808    18818      +10     
==========================================
+ Hits        50284    50339      +55     
+ Misses      10463    10418      -45     
  Partials      191      191              
Flag Coverage Δ
flow 84.06% <ø> (+0.07%) ⬆️
unit 51.84% <100.00%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2-ffi branch from b72444e to 5c42040 Compare February 17, 2026 09:02
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from c3a70ab to 75f4c36 Compare February 17, 2026 09:02
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2-ffi branch from 5c42040 to ba1411a Compare February 17, 2026 09:50
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from 75f4c36 to a5870de Compare February 17, 2026 09:50
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from a5870de to 74a9e4b Compare February 17, 2026 10:40
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2-ffi branch from ba1411a to 42deea6 Compare February 17, 2026 10:40
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from 74a9e4b to 0671484 Compare February 18, 2026 07:25
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2-ffi branch from 42deea6 to 9cb1e97 Compare February 18, 2026 07:25
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch 2 times, most recently from 82552e2 to 7a124f4 Compare February 18, 2026 07:51
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2-ffi branch from 9cb1e97 to be0e1f9 Compare February 18, 2026 10:41
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from 7a124f4 to 6d059cf Compare February 18, 2026 10:41
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from 6d059cf to f4b5d96 Compare February 18, 2026 12:18
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2-ffi branch from be0e1f9 to f0c7711 Compare February 18, 2026 16:18
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from f4b5d96 to 32b2f99 Compare February 18, 2026 16:21
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2-ffi branch from f0c7711 to 5fc6411 Compare February 18, 2026 16:31
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from 32b2f99 to 4fd3d40 Compare February 18, 2026 16:31
@sonarqubecloud
Copy link
Copy Markdown

❌ The last analysis has failed.

See analysis details on SonarQube Cloud

@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from 4fd3d40 to 77f8d14 Compare February 18, 2026 17:22
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2-ffi branch from 5fc6411 to 55b3c0b Compare February 18, 2026 19:02
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from 77f8d14 to 2170f63 Compare February 18, 2026 19:02
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from 1411491 to 68977c6 Compare February 23, 2026 09:52
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 3 potential issues.

@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch 2 times, most recently from 4e1ca51 to fa9da32 Compare February 23, 2026 11:04
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2a-gc-ffi branch 2 times, most recently from 12963d4 to dc9ae47 Compare February 23, 2026 11:05
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from fa9da32 to 91d9770 Compare February 23, 2026 11:05
Base automatically changed from numeric-range-tree-2a-gc-ffi to master February 23, 2026 18:06
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch from 91d9770 to 7559046 Compare February 24, 2026 08:28
@jit-ci
Copy link
Copy Markdown

jit-ci bot commented Feb 24, 2026

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR


Security scan by Jit

@LukeMathWalker LukeMathWalker changed the base branch from master to numeric-range-tree-2c-debug-empty February 24, 2026 08:34
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2b-gc-ffi branch 2 times, most recently from bb7ae68 to 387e96f Compare February 24, 2026 16:42
@LukeMathWalker LukeMathWalker force-pushed the numeric-range-tree-2c-debug-empty branch from 7b89449 to 2f85461 Compare February 24, 2026 16:42
#[repr(C)]
pub struct NumericGcNodeEntry {
/// The node's slab position.
pub node_position: u32,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sohuldn't the type of this be NodeIndex?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In #8407 and #8408, we forked slab, the crate that underpins our node arena, to add support for generational indices: each index now tracks the position into the memory vector (one u32) as well as a monotonically increasing counter (another u32), which gets bumped every time that slot in the memory vector is reused.
This prevents us from hitting the ABA problem:

  • You allocate a new node in slot 5, getting index 5.
  • You remove the node in slot 5, it goes back on the free list.
  • You allocate a new node, slot 5 gets reused for a new, unrelated node.
  • Some other part of our code still holds the old index 5, thinking it points to the original item.
    It reads or writes through that stale index, silently accessing the wrong node.

I improved the docs here to explain how they relate to the index, and the documentation of the node index itself.
Without generational indices, we would have to abort GCing whenever the revision id changes, but that would be way too aggressive.

/// The node's slab position.
pub node_position: u32,
/// The node's slab generation.
pub node_generation: u32,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remind me what node_generation is?
Consider documenting

Base automatically changed from numeric-range-tree-2c-debug-empty to master February 25, 2026 08:18
meiravgri
meiravgri previously approved these changes Feb 25, 2026
@sonarqubecloud
Copy link
Copy Markdown

@LukeMathWalker LukeMathWalker added this pull request to the merge queue Feb 25, 2026
Merged via the queue into master with commit 78ad399 Feb 25, 2026
48 checks passed
@LukeMathWalker LukeMathWalker deleted the numeric-range-tree-2b-gc-ffi branch February 25, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants