cp related PRs for "don't query in db, what is in files" #20518
Merged
Conversation
Closes #20356 - Adjust `fromTxNum` for the DB iterator in `iterateChangedRecent` to `max(fromTxNum, files.EndTxNum())`, matching the existing pattern in `HistoryKeyTxNumRange` - Prevents reading prunable DB entries that are already covered by segment files - Add `TestHistory_IterateChangedRecent_SkipsFileRange`: builds files, prunes DB entries within the file range, verifies `HistoryRange` spanning both files and DB still returns correct results
Closes #20358 - Adjust `recentIterateRange` DB query bounds to skip txNums already covered by frozen files - For ascending: raise the inclusive lower bound (`from`) to `filesEndTxNum` - For descending: raise the exclusive lower bound (`to`) to `filesEndTxNum-1` (so `value > filesEndTxNum-1` ≡ `value >= filesEndTxNum`) - Add `TestInvertedIndex_IdxRange_SkipsFileRange`: builds files for 2 steps, prunes DB entries in the file range, verifies IdxRange still returns correct results for both asc and desc ## Test plan - [x] New test `TestInvertedIndex_IdxRange_SkipsFileRange` passes - [x] All existing `TestInvIndex*` / `TestInvertedIndex*` tests pass - [x] `make lint` clean - [x] `make erigon integration` builds
- In `RangeAsOf`, the DB iterator (`HistoryRangeAsOfDB`) now starts from `max(startTxNum, files.EndTxNum())` instead of `startTxNum` directly - This avoids reading DB entries within the file range, which are redundant (served by `HistoryRangeAsOfFiles`) and prunable - Added `TestRangeAsOf_DBIteratorSkipsFileRange` to verify correctness after pruning DB entries in the file range Closes #20357 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
…st (#20444) Fixes #20355 - Skip DB entries whose step falls within the file range (`step.ToTxNum < files.EndTxNum()`) in both the initial DB cursor push and the advance loop — they are never loaded into the heap - Matches the "don't query DB for data in files" pattern from #20360, - Add `TestDomain_IteratePrefix_PrefersFilesOverDB` that simulates partial lexicographic prune and verifies file values win Follow-up: #20474 applies the same fix to `DomainLatestIterFile.initCursorMDBX()` and `advanceInFiles()` - [x] New test passes with fix, fails without it - [x] `make lint` clean - [x] `make erigon integration` builds
- Remove unused `step` parameter from the `IteratePrefix` callback signature (interface, implementations, and all call sites) - Remove `step` field from `CursorItem` struct (only used by `debugIteratePrefixLatest`) - Remove `step` field from `DomainDelPrefix` tombs tuple (was stored but never read) - Collapse step decode in DB cursor advancement to a single `endTxNum` assignment
93de384 to
609c08a
Compare
AskAlexSharov
approved these changes
Apr 13, 2026
AskAlexSharov
added a commit
that referenced
this pull request
Apr 18, 2026
we need it because we're already doing out-of-order prune of domains. So need this protection. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Apr 20, 2026
Cherry-pick from `release/3.4` to `main`: - #20635 db/state: fix MaxStep using MinTxNum in DomainRoTx.prune - #20639 dbg.SaveHeapProfileNearOOMPeriodically() call in all sub-commands - #20571 blk-range: wrong logs - #20565 merge: set merge workers to 1 - #20576 simplify user-visible logs - #20518 cp related PRs for "don't query in db, what is in files" - #20517 teach: Unwind beyond data in snapshots not allowed - #20368 rpc: use synctest for TestClientCancelHTTP to fix CI flakiness - #20329 logs: move some Info logs to Debug level - #20262 stageloop, execmodule: add ERIGON_IN_MEM_HISTORY env flag for in-mem history reads --------- Co-authored-by: bendertherobert <bendertherobert@gmail.com> Co-authored-by: moskud <sudeepdino008@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: info@weblogix.biz <admin@10gbps.weblogix.it>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
we need it because we're already doing out-of-order prune of domains. So need this protection.