execution/execmodule: apply runloop refactor with furious-prune fix#21270
Merged
Conversation
Applies the intent of PR #21245 onto rev_up_check2 (off performance branch, so a direct cherry-pick wasn't clean). RunLoop refactor: - New PruneFn callback alongside CommitCycleFn - CommitCycleFn now takes hasMore so impl can skip BeginTemporalRw on the final iter (PFB no longer needs the post-loop flush+commit) - Dropped BeforeIteration, PruneTimeout, FirstCycle from RunLoopConfig - RunLoop always invokes CommitCycle; caller returns (nil,nil) to skip ProcessFrozenBlocks: PruneFn wraps pe.sync.RunPrune; CommitCycle kicks agg.BuildFilesInBackground after each commit so seg-build progresses alongside PFB. updateForkChoice: - initialCycle = !isSynced (was limitedBigJump) — prune budget tracks actual sync state, not just LoopBlockLimit chunking - Tip case (!initialCycle): PruneFn and CommitCycle both no-op; the post-RunLoop runForkchoiceFlushCommit + runForkchoicePrune handle the single block - Catchup case: PruneFn forces initialCycle=false to runForkchoicePrune so it always uses furious budget regardless of in-loop initialCycle Skipped from upstream PR: aggregator.go's MaxCollationTxNum getter + its optional cap in BuildFilesInBackground — the underlying SetMaxCollationTxNum / maxCollationTxNum atomic field aren't on performance, so BuildFilesInBackground is called uncapped. Functionally neutral; just slightly less collation throttle control.
…short-chain prune skip
taratorio
approved these changes
May 20, 2026
This was referenced May 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Manual reapply of #21245 (runloop refactor: split prune from commit-cycle) onto the
performancebranch, plus follow-up fixes for the post-frozen-blocks prune-budget regression observed on bloatnet.Changes
CommitCycleFnfromPruneFninRunLoopConfig; dropBeforeIteration/PruneTimeout/FirstCycle.ProcessFrozenBlocks: kicksagg.BuildFilesInBackgroundfrom the newCommitCyclecallback so snapshot files advance during PFB (previously they only progressed after normal sync resumed). On the last iter (!hasMore) returns(nil, nil)to skip a wastedBeginTemporalRw. Outerdefer tx.Rollback()is closure-form so it follows the closure'stxreassignments across iterations.updateForkChoice:PruneFnis a no-op at tip (post-RunLoop path handles flush+commit+prune); in catchup it drains pipeline prune viarunForkchoicePrune(true)—initialCycle=truesoPruneExecutionStagegets the catchup budget instead of the 2 s slot budget that can't keep up with 40-block bursts.initialCyclepredicate — block-count delta againstfinishProgressBefore:headNum >guard avoidsuint64underflow when an FCU goes back (e.g. ePBS on Glamsterdam). Threshold 16 engages catchup mode for bloatnet's ~40-block bursts and stays out of the way for steady-state 1-block tip FCUs and test fixtures.runForkchoicePrune: short-chain skip-gate removed (maxTxNum < (stepSize*5)/4). The gate leftChangeSets3un-pruned on disk for short chains, which brokeMaxReorgDepthenforcement in tests (TestFcuReturnsReorgTooDeepCode38006on the upstreammainbranch). Skip removal is also consistent with theexec3/storage-componentdirection (c380b438e7).CommitCyclesafety:defer commitRwTx.Rollback()immediately afterBeginTemporalRwso the Commit-error path doesn't leak the RW tx. Rollback after a successful Commit is idempotent (per Copilot review on execution/execmodule: apply runloop refactor with furious-prune fix #21245).Why the predicate change matters — bloatnet
bloatnet stabilises at ~38 GB on tip only when
initialCycle=falseis held until execution actually reaches tip:--batchSize=100mbsettles on) writes ~1.4 GB into MDBX (lots of commitment trie changes).SecondsPerSlot/3≈ 2 s) removes less than one such batch.The earlier
!isSyncedandwall-clock head-agevariants either flipped too eagerly (Caplin keeps headers/finish aligned during bursts) or never (test fixtures use ancient timestamps, regressing several rpc/jsonrpc + engineapi tests). Block-count delta with threshold 16 is a clean proxy and works for tests, mainnet tip, and bloatnet bursts.Plateau holds across cycles vs. the prior failure mode where the file extended +6 GB/cycle once the predicate flipped early.
Remaining differences vs #21245 (intentional)
Aggregator.MaxCollationTxNum()getter and theBuildFilesInBackgroundcap pattern from execution/execmodule: apply runloop refactor with furious-prune fix #21245 are NOT ported here — the underlyingmaxCollationTxNumfield doesn't exist on theperformancebranch lineage. Followup, requires also adding the field on this branch.runForkchoicePrunebody usese.db.UpdateTemporal(...)directly (matches this branch's storage design —agg.CollateAndPruneIfNeededownership is being moved out of the FCU path on theexec3/storage-componenttrack), whereas execution/execmodule: apply runloop refactor with furious-prune fix #21245 still callsCollateAndPruneIfNeededvia this function.Test plan
initialCycle=falsefor steady-state 1-block FCUs.