db/state: restore empty tombstones on unwind (cherry-pick #20627 to release/3.4)#20636
Merged
Merged
Conversation
Fix the DB-level unwind path so first-time-write reversals leave a proper
empty-value tombstone in the domain values table, rather than deleting the
current-step entry and silently falling back to files (which have no
concept of deletions and therefore return stale pre-unwind data).
`DomainEntryDiff.Value` carries three distinct shapes that must be handled
differently during unwind (per the comment in db/kv/helpers.go):
- nil — "different step": prev value lives at another step, skip
restore (only produced by legacy V0 changesets where
valueLen==0 deserializes as nil)
- []byte{} — "no previous value": key was absent before this step; write
an empty tombstone at the unwind step so subsequent reads
see the key as absent again
- non-empty — restore the actual previous value
The old guard `if len(value) > 0` conflated `nil` and `[]byte{}` and
skipped both, leaving no tombstone after unwinding a first-time write.
Corrected to `if value != nil`, matching the diff's documented contract.
The written tombstone lands at max(step, currentFilesEndStep), which by
construction is beyond the frozen file range, so the existing step-age
guard in getLatestFromDb already accepts it without needing to change the
read path.
This is a deliberately narrow re-land of #20483 (reverted by #20509). The
PR's other two pieces — the `len(v)==0` empty-authority bypass in
getLatestFromDb and the LargeValues cross-check in getLatest — are NOT
included here. Regressions surfaced in kurtosis/assertoor_glamsterdam
(which runs with --experimental.bal and the storagerefundtx spammer,
generating a deluge of tombstones) suggest the empty-authority bypass
has downstream interactions with the BAL / post-Amsterdam paths that
need separate narrowing. The interrupted-prune LargeValues case is the
original regression suspect from #20509 and remains out-of-scope.
Tests in db/state/domain_test.go:
- TestDomain_UnwindRestoresDeletionMarker (DupSort + LargeValues) — write
a key, delete it, re-write within the same step, build files, then unwind
the re-write. Fails on pre-fix code (getLatest returns the stale post-
unwind value from the frozen file); passes with this fix. Directly
exercises the write-side bug.
- TestDomain_DeletedKeyNotResurrectedByFiles (DupSort + LargeValues) —
retained as a forward regression guard for the read-side contract,
even though it passes on both pre-fix and post-fix code on current main
(file-build + prune semantics have evolved since #20483 such that the
original scenario no longer triggers the read-side bug on its own).
Tested: go test -short ./db/state/... all pass; make lint 0 issues;
make erigon builds.
f63977b to
d72790e
Compare
Remove 3 test helpers/tests that depend on main-only APIs (beginForTests, _testBuildAccessorHook). Replace beginForTests() with BeginFilesRo() in the two tests that are part of this cherry-pick.
Contributor
There was a problem hiding this comment.
LGTM. Correct fix: unwind was treating empty byte slices like "nothing to restore", but here nil and empty have different meanings. nil = previous value lives at another step / skip restore; empty slice = restore empty tombstone so frozen files do not resurrect stale values. Changing the checks from len(value) > 0 to value != nil is the right semantic fix. The two added regression tests are strong and cover both DupSort and LargeValues paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick of #20627 to release/3.4.