Skip to content

db/recsplit: enable sharded FuseFilter by default#21793

Merged
AskAlexSharov merged 1 commit into
mainfrom
jklondon/enable-sharded-fusefilter
Jun 13, 2026
Merged

db/recsplit: enable sharded FuseFilter by default#21793
AskAlexSharov merged 1 commit into
mainfrom
jklondon/enable-sharded-fusefilter

Conversation

@JkLondon

Copy link
Copy Markdown
Member

Summary

Re-enable the sharded FuseFilter existence-filter format by setting ExistenceFilterVersion back from 1 to 2 in db/recsplit/recsplit.go. This is the switch that #21164 turned off after #20644 introduced the format. New index files (.kvi/.efi) will again embed the sharded BinaryFuse filter (256 shards by keyHash>>56), which cuts existence-filter build-time RAM ~256× (the motivation was bloatnet 3B-key .efi builds).

Why it's safe

  • Read path is version-aware. db/recsplit/index.go reads the embedded dataStructureVersion per file and dispatches to v0/v1/v2 readers, so existing v1 files stay readable and new v2 files are read by the matching case 2 path.
  • Accessor/existence files are built locally, not seeded. SeedableV3Extensions() publishes only .kv/.v/.ef/.ap; .kvi/.efi/.kvei are rebuilt locally via BuildMissedAccessors, so this does not affect the shared snapshot set.

Test plan

  • CI green

@JkLondon JkLondon self-assigned this Jun 13, 2026
@AskAlexSharov AskAlexSharov added this pull request to the merge queue Jun 13, 2026
Merged via the queue into main with commit f2cfc7b Jun 13, 2026
92 checks passed
@AskAlexSharov AskAlexSharov deleted the jklondon/enable-sharded-fusefilter branch June 13, 2026 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants