clients/erigon: import all per-block files in a single process#1519
Merged
danceratopz merged 4 commits intoJun 3, 2026
Merged
Conversation
The erigon client restarts a full erigon process for every file in
/blocks, whereas clients/go-ethereum/geth.sh imports them all in one
invocation. On block-heavy BlockchainTests (walletReorganizeOwners,
ForkStressTest) the per-process startup overhead (~0.5s per block times
hundreds of blocks) pushes total startup past Hive's container-startup
timeout, causing intermittent "client did not start: timed out waiting
for container startup" failures (e.g. legacy-cancun
walletReorganizeOwners_{Cancun,Istanbul,London,Paris}).
Import all per-block files in one erigon process, matching geth. Requires
erigontech/erigon#21513 (older erigon import only processed the first
file argument).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced May 29, 2026
Sahil-4555
pushed a commit
to Sahil-4555/erigon
that referenced
this pull request
Jun 3, 2026
…#21513) ## Problem The Hive `legacy-cancun` BlockchainTests suite intermittently fails 4 tests with `client did not start: timed out waiting for container startup`: - `walletReorganizeOwners_{Cancun,Istanbul,London,Paris}` (e.g. [this run](https://hive.ethpandaops.io/#/test/generic/1779833679-c7f17208e0b5dbf959b3448bbdaed60e)). They took ~181s, just over Hive's 180s container-startup timeout. The same test on other forks (Shanghai/Berlin) and `ForkStressTest_*` sit at 146–150s — right at the cliff. ## Root cause The `import` command documents multi-file support: ``` USAGE: erigon import [command options] <filename> (<filename 2> ... <filename N>) ``` but `importChain` only processed `cliCtx.Args().First()`, silently ignoring the rest. That forced Hive's erigon entrypoint into a one-process-per-block-file loop. For `walletReorganizeOwners` (235 block files) that is 235 full erigon startups — measured at ~0.5s/process × 261 = 145s of pure startup, while actual block execution is ~40ms each. go-ethereum has no such problem: its entrypoint imports every block in one `geth import` invocation. ## Fix Iterate every file argument in a single process, tolerating per-file failures when several files are given (matching go-ethereum). This lets the hive entrypoint import all blocks in one invocation, collapsing hundreds of process startups into one (~150s → ~10s). It also force-disables the embedded MCP server for the one-shot import (`--mcp.disable`, alongside the existing NAT / downloader / external-consensus disables) — a batch import has no use for it. ## Companion change (required) The hive entrypoint must pass all block files in one invocation: ethereum/hive#1519. Order matters — the hive change depends on this one (on an old erigon, `import /blocks/*` would import only the first block). ## Testing - New unit tests (`import_cmd_test.go`): all-files iteration, per-file failure tolerance, single-file error surfacing. - Real-binary check: `erigon import a.rlp b.rlp` now attempts **both** files in one process (previously stopped after the first). - `make lint` clean; `make erigon integration` builds. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Giulio Rebuffo <111551070+Giulio2002@users.noreply.github.com>
Condense the comment to one general sentence (no per-test names) and remove the duplicate "Loading remaining individual blocks..." echo, matching geth.sh. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The erigon client restarts a full erigon process for every file in
/blocks, whereasclients/go-ethereum/geth.shimports them all in a single invocation:On block-heavy BlockchainTests (
walletReorganizeOwners,ForkStressTest) the per-process startup overhead (~0.5s × hundreds of blocks) pushes total client startup past Hive's 180s container-startup timeout, producing intermittentclient did not start: timed out waiting for container startupfailures — e.g.legacy-cancunwalletReorganizeOwners_{Cancun,Istanbul,London,Paris}(example run, ~181s each; the Shanghai/Berlin variants andForkStressTest_*sit at 146–150s, right at the cliff).Fix
Import all per-block files in one erigon process, matching
geth.sh:This collapses hundreds of process startups into one (~150s → ~10s for
walletReorganizeOwners).Dependency
Requires erigontech/erigon#21513 (merged). Older erigon
importonly processed the first file argument, soimport $(ls | sort -n)would import only the first block. That fix must merge and ship in the erigon image before this lands — hence draft.