Parent issue
Part of #16239 — avoiding inconsistency due to lexicographic prune.
Problem
In db/state/history.go, the RangeAsOf function creates a HistoryRangeAsOfDB with startTxNum and startTxKey set directly from the caller's startTxNum:
dbit := &HistoryRangeAsOfDB{
...
startTxNum: startTxNum,
...
}
binary.BigEndian.PutUint64(dbit.startTxKey[:], startTxNum)
When startTxNum is within the file range, the DB iterator seeks to startTxKey for each key, returning history entries that are already covered by the file iterator (HistoryRangeAsOfFiles). The stream.UnionKV merge (files first) would pick the file value for duplicate keys, but the DB read is unnecessary and depends on data that should be prunable.
If DB entries within the file range are pruned (in any order), the DB iterator may return incomplete results for those keys. Since UnionKV picks files first, this doesn't cause incorrect results today, but it means the DB data is being read when it shouldn't be.
Fix
Adjust startTxNum for the DB iterator:
dbStartTxNum := max(startTxNum, ht.iit.files.EndTxNum())
dbit := &HistoryRangeAsOfDB{
...
startTxNum: dbStartTxNum,
...
}
binary.BigEndian.PutUint64(dbit.startTxKey[:], dbStartTxNum)
For keys whose relevant history entry is within the file range, the file iterator handles them. For keys with entries only beyond the file range, the DB iterator handles them.
Note
PR #17559 contains this fix but has not been merged. Consider merging that PR or cherry-picking the relevant changes.
Test needed
Add a test verifying RangeAsOf returns correct results after pruning DB entries within the file range.
Parent issue
Part of #16239 — avoiding inconsistency due to lexicographic prune.
Problem
In
db/state/history.go, theRangeAsOffunction creates aHistoryRangeAsOfDBwithstartTxNumandstartTxKeyset directly from the caller'sstartTxNum:When
startTxNumis within the file range, the DB iterator seeks tostartTxKeyfor each key, returning history entries that are already covered by the file iterator (HistoryRangeAsOfFiles). Thestream.UnionKVmerge (files first) would pick the file value for duplicate keys, but the DB read is unnecessary and depends on data that should be prunable.If DB entries within the file range are pruned (in any order), the DB iterator may return incomplete results for those keys. Since
UnionKVpicks files first, this doesn't cause incorrect results today, but it means the DB data is being read when it shouldn't be.Fix
Adjust
startTxNumfor the DB iterator:For keys whose relevant history entry is within the file range, the file iterator handles them. For keys with entries only beyond the file range, the DB iterator handles them.
Note
PR #17559 contains this fix but has not been merged. Consider merging that PR or cherry-picking the relevant changes.
Test needed
Add a test verifying
RangeAsOfreturns correct results after pruning DB entries within the file range.