Skip to content

db/recsplit: set ExistenceFilterVersion to 1#21164

Merged
AskAlexSharov merged 1 commit into
mainfrom
jklondon/fusefilter_v1_35
May 13, 2026
Merged

db/recsplit: set ExistenceFilterVersion to 1#21164
AskAlexSharov merged 1 commit into
mainfrom
jklondon/fusefilter_v1_35

Conversation

@JkLondon

Copy link
Copy Markdown
Member

Summary

  • Change ExistenceFilterVersion from 2 to 1 in db/recsplit/recsplit.go. New index files will use the monolithic FuseFilter format instead of the sharded one.

Test plan

  • CI green

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@AskAlexSharov AskAlexSharov enabled auto-merge May 13, 2026 08:20
@AskAlexSharov AskAlexSharov added this pull request to the merge queue May 13, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 13, 2026
@AskAlexSharov AskAlexSharov added this pull request to the merge queue May 13, 2026
Merged via the queue into main with commit ee507ce May 13, 2026
59 checks passed
@AskAlexSharov AskAlexSharov deleted the jklondon/fusefilter_v1_35 branch May 13, 2026 12:29
sudeepdino008 added a commit that referenced this pull request Jun 3, 2026
- Merge `origin/main` (up to #21546) into the `performance` branch.
- Conflicts resolved by taking main's finalized form where the perf
branch was behind (`ExistenceFilterVersion`→1 per #21164,
`mdbx-go`→v0.40.1, `merge.go` `findMergeRangeInFiles` refactor, dropped
`erigon-snapshot` module dep, fusefilter deferred-close refactor,
`Versions.MustSupport`, atomic per-key prune throttle).
- Adopted main's collation-at-tip design (`CollateAndPrune` in the FCU
path, #21398/#21415) and removed the perf branch's older
`frozenBlocks`-gating (`SetFrozenBlocksProvider`/`MaxCollatableTxNum`,
`db/services/snapshot_progress.go`, its gating tests, and callers).
- Verified: `make erigon integration` build, `make lint` (clean), `make
test-short` (green).
pull Bot pushed a commit to Dustin4444/erigon that referenced this pull request Jun 13, 2026
## Summary

Re-enable the sharded FuseFilter existence-filter format by setting
`ExistenceFilterVersion` back from `1` to `2` in
`db/recsplit/recsplit.go`. This is the switch that erigontech#21164 turned off
after erigontech#20644 introduced the format. New index files (`.kvi`/`.efi`) will
again embed the sharded BinaryFuse filter (256 shards by `keyHash>>56`),
which cuts existence-filter build-time RAM ~256× (the motivation was
bloatnet 3B-key `.efi` builds).

## Why it's safe

- **Read path is version-aware.** `db/recsplit/index.go` reads the
embedded `dataStructureVersion` per file and dispatches to v0/v1/v2
readers, so existing v1 files stay readable and new v2 files are read by
the matching `case 2` path.
- **Accessor/existence files are built locally, not seeded.**
`SeedableV3Extensions()` publishes only `.kv/.v/.ef/.ap`;
`.kvi/.efi/.kvei` are rebuilt locally via `BuildMissedAccessors`, so
this does not affect the shared snapshot set.

## Test plan
- [ ] CI green

Co-authored-by: JkLondon <me@ilyamikheev.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants