db: full mode now prunes block data to EIP-8252 window#21342
Merged
Conversation
…blocks) DefaultPruneDistance is bumped from 100_000 to 262_144 — EIP-8252's REORG_RETENTION_WINDOW, ~36.4 days, the inactivity-leak-bounded non-finality window across which an EL must be able to reconstruct state to handle reorgs without external sync. Full and blocks modes pick up the new value; full additionally moves Blocks from DefaultBlocksPruneMode to Distance(DefaultPruneDistance) so it now bounds block-data retention by distance. A separate MinimalPruneDistance (100_000) keeps minimal mode at the pre-EIP-8252 floor for operators trading state retention for disk. EnsureNotChanged grows a generic retention-window compat path: any finite↔finite Distance change on History or Blocks is accepted (warned and rewritten to the DB) rather than rejected as a config tamper. Sentinel↔finite transitions still error — silently adopting them would delete substantial block data on existing full nodes. Existing full-mode datadirs therefore do NOT auto-migrate: persisted Blocks=DefaultBlocksPruneMode against parsed Blocks=Distance(262_144) is sentinel→finite and trips the guardrail. Operators re-sync or pass --prune.distance.blocks=18446744073709551615 to keep the pre-bump behavior on the existing datadir. The seg-du estimator/detector now uses per-mode distances; QMTree's prune-policy doc reflects the full/minimal split. Refs #20447 See ethereum/EIPs#11601 (EIP-8252) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…n compat shim Existing full-mode datadirs persist Blocks=DefaultBlocksPruneMode (the chain history-expiry sentinel). After the EIP-8252 default bump, the parsed FullMode.Blocks is Distance(DefaultPruneDistance), and the strict deep-equal guard would reject startup. Extend isRetentionWindowChange to accept the one-way DefaultBlocksPruneMode → Distance(N) transition on Blocks so the upgrade is silent — no operator intervention, no explicit --prune.distance.blocks override. Frozen .seg file deletion is gated by #21306; existing on-disk segments persist until that lands. The shim records the config-level transition so future prune passes will use the new cutoff once the deletion path exists. The reverse direction (Distance(N) → DefaultBlocksPruneMode) and any KeepAllBlocksPruneMode transition remain rejected — those are mode-shape changes that need explicit operator action. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Update sync-modes.md to describe full mode's new retention semantics (state + block data within the last 262,144 blocks, ~36.4 days, EIP-8252 window) instead of the previous EIP-4444 pre-merge framing. Refresh the mode-comparison table for all four prune modes. Add a [3.5.0] ChangeLog entry covering the constant bump, the per-mode retention split (full/blocks at 262_144, minimal at 100_000), the automatic-upgrade story for existing datadirs, and the #21306 caveat about physical .seg file deletion. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The PR carries the actual rationale and migration story; the issue had little discussion. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates Erigon’s pruning defaults and compatibility logic to align full/blocks modes with EIP-8252’s reorg retention window (262,144 blocks) while keeping minimal intentionally smaller (100,000 blocks) for disk-constrained operators. It also updates the seg du estimator and documentation to reflect the new retention windows and upgrade behavior.
Changes:
- Increase
DefaultPruneDistanceto262_144and introduceMinimalPruneDistance = 100_000, wiringfull/blocksto the former andminimalto the latter. - Change
FullMode.Blocksfrom the history-expiry sentinel to a finite distance, and generalizeEnsureNotChangedto accept retention-window changes (with a warning + DB rewrite). - Update DU estimation logic and docs/changelog to reflect the new defaults and expected retention behavior.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| execution/commitment/qmtree/PRUNE_POLICY.md | Updates QMTree pruning policy documentation to reflect new full/minimal distances and block-based wording. |
| docs/site/docs/fundamentals/sync-modes.md | Updates sync mode descriptions to EIP-8252 window for full/blocks and 100k window for minimal. |
| db/kv/prune/storage_mode.go | Implements new defaults for prune modes and adds retention-window-change compatibility logic + DB rewrite. |
| db/kv/prune/storage_mode_test.go | Adds unit/integration tests for retention-window change detection and EnsureNotChanged upgrade paths. |
| db/config3/config3.go | Bumps DefaultPruneDistance and adds MinimalPruneDistance with rationale comments. |
| cmd/utils/app/snapshots_cmd.go | Revises seg du estimation logic to model full vs minimal as different distance windows (removes merge-height special casing). |
| cmd/utils/app/snapshots_cmd_test.go | Updates DU tests for the new window math and introduces boundary fixtures to validate cutoff behavior. |
| ChangeLog.md | Documents the full-mode retention policy change, migration behavior, and related caveats. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The previous shim accepted DefaultBlocksPruneMode→finite (the EIP-8252 upgrade path) but rejected the reverse. That left operators stuck: once the binary auto-rewrote persisted Blocks to a finite Distance on first start, the documented override flag (--prune.distance.blocks= 18446744073709551615 to restore the chain-history-expiry sentinel) would no longer work without manual DB intervention or a re-sync. Generalize isRetentionWindowChange's Blocks rule via a single helper: both finite Distance values and DefaultBlocksPruneMode now qualify as "retention policy" values that the shim lets operators move between in either direction. KeepAllBlocksPruneMode remains excluded — narrowing from "keep all" is destructive enough to require explicit operator action from a fresh datadir. ChangeLog updated to reflect the bidirectional acceptance. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
docs/site/docs/fundamentals/sync-modes.md:37
- The full-node description states that nodes “maintain state and block data within” the EIP-8252 window and that “older blocks… are pruned.” In practice, full mode’s new distance-based
Blockssetting mainly impacts pruning/downloading of transaction segments; other block artifacts (e.g., headers/bodies) are governed by different policies. Consider rephrasing to explicitly distinguish which parts of block history are window-bounded vs always retained, to avoid implying that all block bodies/headers beyond the window are unavailable.
The default configuration in Erigon 3 is a Full Node. This setup is designed to offer significantly **faster sync times and reduced resource consumption** for daily operations compared to other clients. It maintains state and block data within the **EIP-8252 reorg-retention window** — the last 262,144 blocks (~36.4 days), the inactivity-leak-bounded non-finality window across which an execution-layer client must be able to reconstruct state to handle any reorg without external sync. Older blocks, receipts, and state history are pruned. See [EIP-8252](https://github.com/ethereum/EIPs/pull/11601) for the rationale behind the constant.
The download path had two parallel filters for transaction segments: buildBlackListForPruning handled distance-based modes (Blocks finite) and isTransactionsSegmentExpired handled the chain-history-expiry sentinel (Blocks=DefaultBlocksPruneMode). Each ran independently at the call site, and each was unaware of the other's scope. Move the pre-merge check into buildBlackListForPruning so a single function knows how to expire tx segments under any mode: - finite Distance → res.To <= blockPrune - DefaultBlocksPruneMode + MergeHeight → res.From < mergeHeight - KeepAllBlocksPruneMode → never blacklist Signature changes accordingly (pruneMode bool → prune.Mode, plus a *chain.Config for IsPreMerge). isTransactionsSegmentExpired and its inline call site are removed. Behavior on the existing test case (distance-based) is unchanged; a new TestBlackListForPruning_ChainHistoryExpiry covers the absorbed sentinel path against mainnet preverified. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Distance.Enabled() returns false only for math.MaxUint64 (DefaultBlocksPruneMode); it returns true for KeepAllBlocksPruneMode (math.MaxUint64 - 1). The previous blocksByDistance variable relied on .Enabled() and only happened to behave correctly because KeepAllBlocksPruneMode.PruneTo() returns 0 (so no segment ever matched blockPrune >= res.To). Renamed to blocksByFiniteDistance and used an explicit two-sentinel exclusion so the control flow matches the variable name. Added TestBlackListForPruning_BlocksModeKeepsAllTransactions to lock down that --prune.mode=blocks never blacklists transaction segments regardless of stepPrune. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The persistReceipts branch in Capabilities assumed receipts/logs are
available from genesis when the receipt cache is on, but that only holds
when block bodies and log indexes are also kept indefinitely
(DefaultBlocksPruneMode / KeepAllBlocksPruneMode). With this PR making
FullMode use a finite Distance for Blocks, eth_getBlockReceipts and
getLogsV3 silently return empty for blocks older than head-pruneDistance
— so eth_capabilities was advertising data the node cannot serve.
Reuse avail() in both branches so receipts/logs carry the same
DeleteStrategy{window, N} as blocks when the bound is finite. Update the
minimal_persist_receipts subtest, which previously pinned the buggy
oldest=0 / DeleteStrategy=nil behaviour, and add full_eip8252_no_persist
+ full_eip8252_persist_receipts subtests that exercise the new
production FullMode shape (both History and Blocks finite).
buildBlackListForPruning internally clamps blockPrune via adjustBlockPrune when Blocks is a finite distance — with blockPrune=25M and minBlockToDownload=20M, the effective tx cutoff is 20M, not 25M. The test asserted `info.To > blockPrune (25M)`, which the function can never violate no matter how the clamp behaves, so a regression that over-blacklists segments between 20M and 25M would slip through. Compute effectiveCutoff via adjustBlockPrune in the test and assert against that. Flagged by Copilot in #21342 (comment). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
duDetectNodeType gated the "blocks" branch on hasGenesisTxSegment && maxBlock > MinimalPruneDistance (100_000), but full mode doesn't start pruning tx segments until the chain crosses DefaultPruneDistance (262_144). On a chain in the [100k, 262k] band, a real full-mode datadir still has the genesis tx segment because pruning hasn't kicked in — and the detector misclassified it as "blocks". Gate on DefaultPruneDistance instead. In the ambiguous [100k, 262k] band full and blocks look identical on disk, and the fall-through defaults to full (the more common mode). Added a regression test at maxBlock=200_000 that pins the new behaviour. Flagged by Copilot in #21342 (comment). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The prune-mode mapping block and the proof-capability table both skipped --prune.mode=blocks, which currently reads as if qmtree has no defined behaviour under that mode. Blocks mode's History distance is DefaultPruneDistance, same as full, so qmtree entry retention matches full one-for-one. Add the missing rows to both the code block and the table. Flagged by Copilot in #21342 (comment). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two coupled cleanups from the unresolved Copilot thread at #21342 (comment): 1. db/snapshotsync: buildBlackListForPruning was computing preMergeCutoff via blocksRetentionCutoff(pruneMode, cc, 0), with a four-line comment apologising for the head=0 trick (which suppressed the finite-distance branch to extract just the merge-height value). Replace with an explicit applyChainHistoryExpiry flag and call cc.IsPreMerge(res.From) for the per-segment check — matches the docstring verbatim and drops the indirection. blocksRetentionCutoff is still used by isReceiptsSegmentPruned. 2. db/kv/prune: rename DefaultBlocksPruneMode → KeepPostMergeBlocksPruneMode. The "Default" name dates from when this sentinel was the actual default for FullMode. After this PR FullMode.Blocks is a finite Distance and the sentinel is no longer any named mode's default. The new name parallels KeepAllBlocksPruneMode and describes the behaviour: on chains with MergeHeight set, pre-merge tx is dropped; post-merge data is kept. The symbol rename ripples through db/snapshotsync, rpc/jsonrpc and the ChangeLog entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…cate After the FullMode shape change in this PR, the test fixture named fullMode / testFullMode (Blocks=KeepPostMergeBlocksPruneMode, History finite) no longer matches the production FullMode — it matches the legacy pre-#21342 shape. Rename to legacyFull / testLegacyFull so the variable name doesn't actively mislead. testFullModeEIP8252 (added earlier in this PR for the new finite-finite shape) is structurally identical to testMinimalMode at the test's testPruneDistance=10. Drop the duplicate and reuse testMinimalMode; the existing per-subtest comments still document which production scenario each subtest models, and the variable's docstring is widened to note that the shape stands in for both production modes at test scale. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6 tasks
wmitsuda
added a commit
that referenced
this pull request
May 29, 2026
Replaces the minimal 5-entry fixture with the full mainnet.toml copied from erigon-snapshot v1.3.1-0.20260402120223-7bb412bc89cd — the version pinned on main when the Go-module dep was dropped. Needed because new tests added on main (TestBlackListForPruning* from #21342) consult snapcfg.KnownCfg(mainnet) and assert against state-history step ranges and 20M+ block tx segments that the minimal fixture doesn't cover. Updates TestMain doc to explain the prior implicit coupling to the embedded module and why we froze a copy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements EIP-8252's
REORG_RETENTION_WINDOW = 262_144blocks (~36.4 days) as the new default for full and blocks prune modes.Mode definitions
archiveKeepAllBlocksPruneModefull(default)Distance(262_144)(wasDefaultBlocksPruneMode)Distance(262_144)(was100_000)blocksKeepAllBlocksPruneModeDistance(262_144)(was100_000)minimalDistance(100_000)Distance(100_000)MinimalPruneDistanceconstant, deliberately sub-EIP-8252The
DefaultBlocksPruneModesentinel is also renamed in this PR toKeepPostMergeBlocksPruneModeto reflect what it actually does (drop pre-merge data, keep post-merge) now that it's no longer the default for any named mode.Fresh-sync impact
New full nodes download substantially fewer transaction segments than 3.4's full mode.
buildBlackListForPruningactivates for distance-basedBlocksand blacklists tx segments withTo <= frozenBlocks - 262_144before the download loop. On mainnet today (head ~22M, merge ~15.5M) that's keeping ~262k blocks of tx instead of ~6.5M post-merge blocks — roughly a 25× reduction in transaction-segment disk footprint. Headers and bodies (excluding tx) follow their existing rules.Existing-datadir upgrades
EnsureNotChangedgot a generic retention-window compat path:Distancechanges onHistoryorBlocksin either direction (covers the EIP-8252 default bump and any operator-initiated--prune.distancechange).DistanceandKeepPostMergeBlocksPruneMode(chain-history-expiry sentinel) onBlocks— so legacy full datadirs upgrade silently, and operators can revert with--prune.distance.blocks=18446744073709551615even after the auto-upgrade has rewritten the persisted value.KeepAllBlocksPruneMode— narrowing from "keep all blocks" is the one truly destructive direction and stays explicit.When a compatible transition is accepted, the persisted value is rewritten so subsequent restarts don't re-fire the warning. The pre-existing archive-default-bump compat branch is preserved for the orthogonal
{KeepPostMergeBlocksPruneMode, KeepPostMergeBlocksPruneMode} → ArchiveModelegacy-archive case.eth_capabilitiesclampPreviously
eth_capabilitiesadvertisedreceipts.oldestBlock=0andlogs.oldestBlock=0whenever--persist.receiptswas set, regardless of whether the node could actually serve them. That was correct under the oldFullModeshape (Blocks=DefaultBlocksPruneModekept block bodies indefinitely), but not under the new finite-Distance shape —eth_getBlockReceiptswalks block bodies andgetLogsV3reads log indexes, both of which followprune.Blocks. The RPC now reportsblocksOldestfor receipts/logs with the sameDeleteStrategy{window, N}as blocks, so routing layers can correctly determine which node to ask.Caveat: frozen
.segfile deletion is gated by #21306Physical deletion of already-on-disk frozen transaction segments is not yet implemented (#21306). Upgraded full datadirs will record the new cutoff but keep the old
.segfiles on disk until that lands; freshly-synced full nodes use the blacklist at download time and avoid the data in the first place. So an upgraded full node will have more on-disk data than a freshly-synced one until #21306 ships.Other changes wrapped up in this PR
Distance.Enabled()corrected to return false for both sentinel values (KeepPostMergeBlocksPruneModeandKeepAllBlocksPruneMode). Previously only the former was excluded; the snapshotsync blacklist needed a local workaround that's now gone.isTransactionsSegmentExpiredremoved, absorbed intobuildBlackListForPruning. The download path now has one filter for tx segments instead of two parallel ones covering different mode shapes.buildBlackListForPruningusescc.IsPreMerge(res.From)directly for the chain-history-expiry case rather than indirecting throughblocksRetentionCutoffwith ahead=0trick.blocksRetentionCutoffextracted and used byisReceiptsSegmentPrunedso the per-mode cutoff logic lives in one place.downloadFilteringAppliesextracted to gate the slowgetMinimumBlocksToDownloadcall with the same predicatebuildBlackListForPruninguses internally; previously the gate was narrower than the function's own activation logic, which caused the--prune.mode=archive --prune.distance.blocks=18446744073709551615hybrid to silently skip pre-merge tx filtering.Mode.String()recognises legacy shapes rather than rendering them as misleading "archive ..." strings:{KeepPostMergeBlocksPruneMode, finite}→"full(legacy) --prune.distance=N";{KeepAllBlocksPruneMode, finite}→"blocks --prune.distance=N". The fallback rendering still mirrorsFromCliinput so operator-supplied--prune.mode=archive --prune.distance=…round-trips visibly.duDetectNodeTypeblocks-mode gate usesDefaultPruneDistancefor the maturity check: a genesis tx segment is evidence of blocks mode only on a chain past the point where full would have pruned it. On chains in the[MinimalPruneDistance, DefaultPruneDistance]band, full and blocks are indistinguishable on disk; the fall-through defaults the ambiguous case to full, the more common mode.duComputeEstimatesreturns a 4th row for blocks mode — mirrors full's state-history pruning but keeps all tx segments and receipt-related state, matching the runtime behaviour.setIfNotExistandoverwriteStoredModeshare awriteBlockAmounthelper that owns the on-disk serialisation.docs/site/docs/fundamentals/sync-modes.md,execution/commitment/qmtree/PRUNE_POLICY.md, ChangeLog entry, regeneratedllms-full.txtartifacts.Refs EIP-8252 (informational): ethereum/EIPs#11601
Closes #20447
Test plan
make lintclean across multiple runs (non-deterministic linter)go build ./...cleango test ./db/kv/prune/...: 11-caseisRetentionWindowChangetable + 9EnsureNotChangedintegration tests (legacy minimal no-op, blocks-mode History bump with DB rewrite, full sentinel→finite acceptance, finite→KeepPostMergeBlocksPruneModerevert acceptance,KeepAllBlocksPruneModenarrowing rejected, archive identity, arbitrary-distance change, pre-existing archive-default-bump compat) +TestModeString_LegacyShapesgo test ./db/snapshotsync/...:TestBlackListForPruning(asserts against the adjusted-cutoff value the function actually uses, so a regression inadjustBlockPrunewould fail loudly),TestBlackListForPruning_BlocksModeKeepsAllTransactions(locks down theDistance.Enabled()fix),TestBlackListForPruning_ChainHistoryExpiry(covers the absorbed sentinel path against mainnet preverified),TestDownloadFilteringApplies(8-case table including the{KeepPostMerge, KeepPostMerge}hybrid)go test ./cmd/utils/app/... -run TestDU: boundary fixtures atToin the gap between minimal's and full's cutoffs validate per-mode pruning math; blocks-mode row in estimates; detector classification including the[MinimalPruneDistance, DefaultPruneDistance]ambiguity-band regression testgo test ./rpc/jsonrpc/... -run TestCapabilities: legacy-full shape (chain-history-expiry sentinel) and finite-finite shape (EIP-8252) covered, including--persist.receiptsinteractions andMergeHeightclamping