Parent issue
Part of #16239 — avoiding inconsistency due to lexicographic prune.
Problem
In db/state/history.go, the iterateChangedRecent function creates a HistoryChangesIterDB with startTxKey set from fromTxNum:
if fromTxNum >= 0 {
binary.BigEndian.PutUint64(s.startTxKey[:], uint64(fromTxNum))
}
When toTxNum extends beyond the file range (or is -1/unlimited), the DB is queried starting from fromTxNum, which may be within the file range. This means DB entries for txNums already covered by files are read and returned. If those DB entries are pruned (in any order), the results would be incomplete.
The rangeIsInFiles early return only handles the case where the entire range is in files. When the range spans both file and DB ranges, the DB portion starts too early.
Fix
Adjust fromTxNum for the DB iterator to max(fromTxNum, files.EndTxNum()):
dbFrom := fromTxNum
if len(ht.iit.files) > 0 {
dbFrom = max(fromTxNum, int(ht.iit.files.EndTxNum()))
}
if dbFrom >= 0 {
binary.BigEndian.PutUint64(s.startTxKey[:], uint64(dbFrom))
}
This follows the same pattern already used in HistoryKeyTxNumRange (~line 1503):
dbFrom = max(fromTxNum, int(ht.iit.files.EndTxNum()))
The caller HistoryRange unions the file iterator (iterateKeyTxNumFrozen) with this DB iterator. The file iterator handles [fromTxNum, files.EndTxNum()) and the DB iterator handles [files.EndTxNum(), toTxNum).
Note
PR #17559 contains this fix but has not been merged.
Test needed
Add TestHistory_IterateChangedRecent_SkipsFileRange: write history data across multiple steps, build files, call HistoryRange spanning both file and DB ranges, prune DB entries within the file range, verify results are still correct.
Parent issue
Part of #16239 — avoiding inconsistency due to lexicographic prune.
Problem
In
db/state/history.go, theiterateChangedRecentfunction creates aHistoryChangesIterDBwithstartTxKeyset fromfromTxNum:When
toTxNumextends beyond the file range (or is -1/unlimited), the DB is queried starting fromfromTxNum, which may be within the file range. This means DB entries for txNums already covered by files are read and returned. If those DB entries are pruned (in any order), the results would be incomplete.The
rangeIsInFilesearly return only handles the case where the entire range is in files. When the range spans both file and DB ranges, the DB portion starts too early.Fix
Adjust
fromTxNumfor the DB iterator tomax(fromTxNum, files.EndTxNum()):This follows the same pattern already used in
HistoryKeyTxNumRange(~line 1503):The caller
HistoryRangeunions the file iterator (iterateKeyTxNumFrozen) with this DB iterator. The file iterator handles[fromTxNum, files.EndTxNum())and the DB iterator handles[files.EndTxNum(), toTxNum).Note
PR #17559 contains this fix but has not been merged.
Test needed
Add
TestHistory_IterateChangedRecent_SkipsFileRange: write history data across multiple steps, build files, callHistoryRangespanning both file and DB ranges, prune DB entries within the file range, verify results are still correct.