db/state, db/integrity: FilesOnlyStateReader for commitment rebuild and CommitmentRoot#21026
Merged
Conversation
…ment rebuild and CommitmentRoot Replace LimitedHistoryStateReader (which fell back to GetLatest on miss and silently leaked post-limit values) with a strict files-only reader for the two callers that need a fixed boundary snapshot: - RebuildCommitmentFiles uses FilesOnlyStateReader(rwTx, lastTxnumInShard-1) so commitment.kv is rebuilt only from the matching accounts/storage/code .kv files at the shard boundary. - CheckCommitmentRoot's checkCommitmentRootViaSd uses FilesOnlyStateReader at maxTxNum, and checkCommitmentRootViaRecompute switches from an account-domain-only touchHistoricalKeys to sd.TouchChangedKeysFromHistory (accounts + storage; code is covered transitively via account.codeHash). getLatestFromFiles walkback is also fixed: the previous bound skipped files ending before maxTxNum, which broke the walkback semantics and made files-only reads return nil for any key whose latest write lives in an older .kv. Now only files starting strictly after maxTxNum are skipped.
AskAlexSharov
approved these changes
May 7, 2026
3 tasks
AskAlexSharov
pushed a commit
that referenced
this pull request
May 7, 2026
Sahil-4555
pushed a commit
to Sahil-4555/erigon
that referenced
this pull request
May 8, 2026
…h#21029) ## Summary - Filter `.kv` files before slicing in `CheckCommitmentRoot`, so `onlyCheckLastFile` picks the latest `.kv` instead of whatever `aggTx.Files` returned last (which was often the `.ef`). - Default `CHECK_COMMITMENT_ROOT_ONLY_LAST_FILE` and `CHECK_COMMITMENT_ROOT_ONLY_LAST_FILE_RECOMPUTE` to `false` so the check now covers all `.kv` files by default. - Add a discovery log line and a clearer warning when no `.kv` files are present. - with erigontech#21026 -- recompute check works fine now; and it is fast enough to do for all kv files ## Test plan - [ ] `make lint` - [ ] `make erigon integration` - [ ] Run `erigon seg integrity --check=CommitmentRoot` against a datadir and confirm all `.kv` files get checked.
awskii
added a commit
that referenced
this pull request
May 13, 2026
…build and CommitmentRoot (#21143) Cherry-pick of #21026 to `release/3.4`. Conflicts resolved: - `db/state/squeeze.go` — kept the `SetStateReader(NewFilesOnlyStateReader(...))` swap; dropped the unrelated `if concurrent { EnableParaTrieDB }` context lines that don't exist on `release/3.4`. - `execution/commitment/commitmentdb/commitment_context.go` — removed `SetLimitedHistoryStateReader` as in the original PR; did not introduce `SetCustomHistoryStateReader` since it isn't part of this PR's diff and isn't present on `release/3.4`. Co-authored-by: moskud <sudeepdino008@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LimitedHistoryStateReader -> FilesOnlyStateReader
CommitmentRoothas a subcheckcheckCommitmentRootViaRecompute: it touches/replays content of the final block in commitment.kv file and doesComputeCommitment. The resulting root hash should remain unchanged.RebuildCommitment: it creates shard of size 16 and ultimately merged.Both need contrainted/limited query of state data (i.e. only return value for key K from before step X)
both suffer from 2 problems today:
getLatestFromFiles(maxTxNum): it effectively searches only a single kv file (the one that contains maxTxNum). The files "left and right" of it are ignored.LimitedHistoryStateReader: ifgetLatestFromFiles(maxTxNum)returns nil, it fallbacks to GetLatest.This creates problem like:
getLatestFromFileslogic)LimitedHistoryStateReaderwould fallback on GetLatest in this case.FilesOnlyStateReader: it's essentially LimitedHistoryStateReader, without the fallback on
GetLatest.getLatestFromFiles walkback
the problem is mentioned above. The fix simply walks back from "maxTxNum containing snapshot" to the first snapshot.
The change only impacts when
maxTxNum != uint64.maxwhich is used inFilesOnlyStateReader,checkCommitmentRootViaFileDataandcommitment printcommand.checkCommitmentRootViaRecompute doing TouchKey only on accounts domain.
checkCommitmentRootViaRecompute— switch from account-domain-onlytouchHistoricalKeystosd.TouchChangedKeysFromHistory(accounts + storage; code is covered transitively viaaccount.codeHash). This is the same helper used byrpc/rpchelper/commitment.goand receipts generation.