db/kv, db/migrations, cmd: drop pre-E3 plain/hashed state and history tables#21280
Merged
Conversation
… tables
E3 execution stores state in kv.AccountsDomain / kv.StorageDomain /
kv.CommitmentDomain, which are incarnation-free. The following pre-E3
("E2-era") tables are dormant in production — they are not read by
the canonical execution / commitment path:
- PlainState (account + storage rows; storage keys
baked the per-account incarnation
counter into [addr]+[inc]+[key])
- HashedAccount / HashedStorage (legacy MPT inputs; storage keys
used [hash(addr)]+[inc]+[hash(key)])
- AccountChangeSet / StorageChangeSet (per-block reverse-diff
storage with [blockNum]+[addr]
(+inc) keys)
- AccountHistory / StorageHistory (roaring-bitmap E2 history indices,
superseded by the AccountHistory*
/ StorageHistory* domain tables)
Schema
- All seven constants move from kv.ChaindataTables to
kv.ChaindataDeprecatedTables. The mdbx migrator opens the buckets
read-only if they exist (DBAccede) and drops the empty ones on the
next exclusive open; the live schema no longer recreates them on
fresh DBs.
- DupSort flags for PlainState / HashedStorage / AccountChangeSet /
StorageChangeSet are preserved in ChaindataTablesCfg so existing E2
databases can still be opened with the correct bucket flags by the
migration before it clears them.
Migration
- New drop_legacy_e2_tables migration (db/migrations/) clears all
seven tables on existing E2-migrated databases. The migration is
idempotent — ClearTable is a no-op on already-empty tables.
Caller cleanups
- t8ntool's CalculateStateRoot dropped the PlainState →
HashedStorage/HashedAccounts hashing loop; ComputeCommitment reads
from kv.CommitmentDomain directly and the loop's writes were never
consumed.
- kvcache.AssertCheckValues (an assert-tag-only debug check that
compared cache rows against kv.PlainState) is stubbed to a no-op —
the active state for cache validation now lives in
kv.AccountsDomain / kv.StorageDomain and the read-key format no
longer matches what the cache stores.
- cmd/integration/commands/refetence_db.go and cmd/pics/state.go drop
the deprecated tables from their debug whitelists.
- cmd/rpctest's account_range_verify used kv.E2AccountsHistory purely
as a bucket name to JSON-dump eth_accountRange results into a fresh
temp MDBX env for erigon/geth comparison (nothing to do with the
actual E2 history index semantics). Switched the tool to a local
"AccountDump" bucket via mdbx.WithTableCfg so the temp DB is fully
decoupled from the chain schema.
Test adjustments
- TestManagedTx (db/kv/mdbx) previously picked kv.ChaindataTables[0]
and [1] alphabetically and relied on them having matching DupSort
flags. After the deprecated tables move, that invariant doesn't
hold (AccountHistory (Flags=0) and AccountHistoryKeys (DupSort)
now lead the list). Switched the test to explicit TblAccountVals /
TblStorageVals so it does not depend on sort order.
- TestEthCallToPrunedBlock (rpc/jsonrpc) called PruneTableDupSort on
kv.StorageChangeSetDeprecated, which the schema no longer
materialises in ChaindataTables — cursor open panics with
MDBX_BAD_DBI. Dropped that prune; the active prune on TblAccountVals
exercises the same code path.
This is the schema-cleanup half of #12440 (incarnation removal) —
the in-memory IncarnationPath removal and the storage-key /
legacy-MPT encoding changes ship separately on top of this PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR removes legacy pre-E3 (“E2-era”) state/history buckets from the active chain DB schema, adds a migration to clear existing instances of those buckets on upgraded databases, and updates tools/tests that previously referenced these deprecated tables.
Changes:
- Move seven legacy E2 buckets (PlainState / hashed state / changesets / history indices) out of
kv.ChaindataTablesintokv.ChaindataDeprecatedTables, preventing new DBs from creating them. - Add a
drop_legacy_e2_tablesmigration to clear these legacy buckets on existing databases. - Remove remaining reads/writes and debug tooling references to the deprecated buckets (t8ntool state root calc, rpctest temp DB dump bucket, integration/pics whitelists, tests).
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| rpc/jsonrpc/eth_call_test.go | Removes pruning of deprecated changeset bucket; updates test comments. |
| db/migrations/migrations.go | Registers the new drop_legacy_e2_tables migration for ChainDB. |
| db/migrations/drop_legacy_e2_tables.go | New migration to clear legacy E2 buckets. |
| db/kv/tables.go | Moves legacy buckets to ChaindataDeprecatedTables and preserves needed bucket flags in table config. |
| db/kv/mdbx/kv_abstract_test.go | Makes TestManagedTx pick stable DupSort tables instead of relying on ChaindataTables ordering. |
| db/kv/kvcache/cache.go | Stubs AssertCheckValues to a no-op since PlainState is no longer part of the active schema. |
| cmd/rpctest/rpctest/account_range_verify.go | Uses a local-only AccountDump bucket via WithTableCfg instead of reusing E2 history bucket name. |
| cmd/pics/state.go | Removes legacy buckets from debug label map. |
| cmd/integration/commands/refetence_db.go | Removes legacy buckets from debug whitelist. |
| cmd/evm/internal/t8ntool/transition.go | Removes legacy PlainState→hashed-state hashing loop; uses commitment domain directly. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…clear Address Copilot review on #21280: - drop_legacy_e2_tables migration was calling tx.ClearTable, which only empties the bucket (MDBX Drop with del=false) and leaves the bucket metadata in place. The doc comments on both the migration and the matching ChaindataDeprecatedTables / ChaindataTablesCfg entries said "drop" or "clears + drops" — overstated relative to the implementation. Switch to tx.DropTable: it requires IsDeprecated=true (which these tables now have), reclaims the bucket metadata, and is idempotent — dropEvenIfBucketIsNotDeprecated handles the NonExistingDBI case and returns nil if a re-open with mdbx.IsNotFound fails. - eth_call_test.go's doPrune was leaking the time.NewTicker(20s) it created for the prune progress log; nothing called Stop() so the underlying ticker goroutine survived each test run. Add the missing defer. - Tighten the comment in doPrune to say StorageChangeSetDeprecated is no longer "part of the active schema" rather than "materialised in the active schema". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
yperbasis
added a commit
that referenced
this pull request
May 19, 2026
…clear Address Copilot review on #21280: - drop_legacy_e2_tables migration was calling tx.ClearTable, which only empties the bucket (MDBX Drop with del=false) and leaves the bucket metadata in place. The doc comments on both the migration and the matching ChaindataDeprecatedTables / ChaindataTablesCfg entries said "drop" or "clears + drops" — overstated relative to the implementation. Switch to tx.DropTable: it requires IsDeprecated=true (which these tables now have), reclaims the bucket metadata, and is idempotent — dropEvenIfBucketIsNotDeprecated handles the NonExistingDBI case and returns nil if a re-open with mdbx.IsNotFound fails. - eth_call_test.go's doPrune was leaking the time.NewTicker(20s) it created for the prune progress log; nothing called Stop() so the underlying ticker goroutine survived each test run. Add the missing defer. - Tighten the comment in doPrune to say StorageChangeSetDeprecated is no longer "part of the active schema" rather than "materialised in the active schema". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ad PlainState test puts AssertCheckValues was the E2 sanity check that compared cache rows against kv.PlainState. With PlainState moved to the deprecated-tables list it could no longer do its job; the previous commit left a no-op stub so the assert-tag-only call site in txnprovider/txpool/pool.go would still compile. Just remove both — the stub had no callers outside this repo and the txpool assertion never observed a real divergence in practice. Also drop the `tx.Put(kv.PlainState, ...)` calls in TestEviction: they silently failed via `_ = tx.Put(...)` and TestEviction's assertions key off c.OnNewBlock populating the cache directly, not those puts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two mdbx envs opened by CompareAccountRange were never closed, so running the debug tool repeatedly in-process would leak file descriptors and keep MDBX locks held. Pre-existing in the function but flagged on the touched lines by the PR review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AskAlexSharov
approved these changes
May 22, 2026
yperbasis
added a commit
that referenced
this pull request
May 22, 2026
Five conflicts: 1) cmd/rpctest/rpctest/account_range_verify.go — main added defer Close() on the two MDBX handles. Trivial: kept both. 2) db/migrations/migrations.go — main landed #21280 (the legacy E2 tables drop spinoff originally cut from this branch), which adds dropLegacyE2Tables. Moksha has both dropLegacyE2Tables and dropAccountIncarnation registered. Resolution: keep both in the ChainDB sequence. 3) db/kv/kvcache/cache.go — main removed AssertCheckValues entirely (the txpool stub call site went away with #21280's caller cleanup). Moksha had it as a no-op stub. Take main's deletion; the only reference left was a commented-out test line. 4) execution/stagedsync/exec3_finalize_test.go — PR #21211 deleted ~240 lines of test scenarios (TestFinalizeTx_SimpleTransfer, _London, _AllScenarios, coinbaseIsRecipientScenario, selfTransferScenario, hasCoinbaseDelta, adjustForTransferDelta) because the finalizeWithIBS / finalizeTx (delta-args) code paths they exercised were dead. Take main's deletion. 5) execution/state/intra_block_state.go — three blocks: a) Same-block-revival check in CreateAccount. Main (#21319) simplified it to `!account.Empty()` on the version-map-refreshed record. Moksha had a path-by-path LatestTxIndex scan (BalancePath/NoncePath/CodeHashPath > destructTxIndex). Take main's cleaner check — semantically equivalent on the same input (the refreshed `account` reflects exactly those higher- txIndex writes). b) prevInc / IncarnationPath bookkeeping for CreateAccount on main. Entirely incarnation-dependent; moksha has no IncarnationPath and no Account.Incarnation. Take HEAD's empty resolution. c) Synthetic versionRead(IncarnationPath, ...) in CreateAccount on main. Same reason. Take HEAD's empty resolution. Plus two non-conflicting touch-ups dropping IncarnationPath references that the merge silently brought in via main-side changes: - execution/state/versionmap.go MVReadResultNone/MapRead recursive- cross-check whitelist dropped IncarnationPath from the path list. - execution/state/versionedio.go SD short-circuit zero-value-fallback whitelist dropped IncarnationPath from the path list. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
yperbasis
added a commit
that referenced
this pull request
Jun 4, 2026
PR #21280 dropped pre-E3 plain/hashed state and history tables (a data migration), so bump DBSchemaVersion and KvServiceAPIVersion to 7.1.0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Schema-cleanup half of #12440 (incarnation removal). Seven pre-E3 ("E2-era") tables are dormant in production — they are not read by the canonical E3 execution / commitment path, which works off
kv.AccountsDomain/kv.StorageDomain/kv.CommitmentDomain:PlainState— account + storage rows; storage keys baked the per-account incarnation counter into[addr]+[inc]+[key]HashedAccount/HashedStorage— legacy MPT inputs; storage keys used[hash(addr)]+[inc]+[hash(key)]AccountChangeSet/StorageChangeSet— per-block reverse-diff storage with[blockNum]+[addr](+inc)keysAccountHistory/StorageHistory— roaring-bitmap E2 history indices, superseded by theAccountHistory*/StorageHistory*domain tablesThis PR moves all seven from
kv.ChaindataTablesintokv.ChaindataDeprecatedTablesso the live schema no longer recreates them on fresh DBs. A newdrop_legacy_e2_tablesmigration drops the buckets (both data and metadata) on existing E2-migrated databases; the migration is idempotent —DropTableis a no-op for tables that were never materialised, so fresh-DB upgrades are a pass-through.Knock-on cleanups required because the now-deprecated tables would otherwise still be read/written:
t8ntool.CalculateStateRoot— dropped the deadPlainState → HashedStorage/HashedAccountshashing loop;ComputeCommitmentreads fromkv.CommitmentDomaindirectly.kvcache.AssertCheckValues(assert-tag-only debug check that compared cache rows againstkv.PlainState) — removed, along with its sole call site intxnprovider/txpool/pool.go.cmd/integration/commands/refetence_db.goandcmd/pics/state.go— dropped deprecated tables from their debug whitelists.cmd/rpctest/account_range_verifyusedkv.E2AccountsHistorypurely as a bucket name to JSON-dumpeth_accountRangeresults into a temp MDBX env for erigon/geth comparison; switched to a localAccountDumpbucket viamdbx.WithTableCfgso the temp DB is fully decoupled from the chain schema.TestManagedTx(db/kv/mdbx) — was picking the first two entries of the alphabetically-sortedChaindataTableslist and relying on them having matching DupSort flags; switched to explicitTblAccountVals/TblStorageVals.TestEthCallToPrunedBlock(rpc/jsonrpc) — dropped the now-impossiblePruneTableDupSort(kv.StorageChangeSetDeprecated, ...)call.TestEviction(db/kv/kvcache) — dropped deadtx.Put(kv.PlainState, ...)calls that the test's assertions never depended on.This spinoff is independent of and lands ahead of the in-memory
IncarnationPathremoval (already onmain) and the storage-key + legacy-MPT-path incarnation removal (#21256), so reviewers focused on schema vs trie/state-write changes can evaluate the two in isolation.Test plan
make erigon integration— cleango build ./...— clean (default and-tags assert)make lint—0 issuesgo test -short -failfast ./rpc/jsonrpc/... ./db/kv/... ./db/migrations/... ./cmd/evm/... ./cmd/rpctest/... ./txnprovider/txpool/...— all green, including the previously-failingTestManagedTxandTestEthCallToPrunedBlock🤖 Generated with Claude Code