Skip to content

cmd/evm: fix regressions in statetest cmd#21111

Merged
taratorio merged 4 commits into
mainfrom
worktree-fix-evm-fuzz-statetest
May 13, 2026
Merged

cmd/evm: fix regressions in statetest cmd#21111
taratorio merged 4 commits into
mainfrom
worktree-fix-evm-fuzz-statetest

Conversation

@taratorio

Copy link
Copy Markdown
Member

for #21080

@yperbasis yperbasis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context: Fixes issue #21080: (a) stateRoot no longer printed on stderr, (b) JSON result reports empty.RootHash instead of the real computed root when there's an error. Both root causes were introduced by #21058.

The core fixes are correct. Three observations worth addressing before merge:


1. enginextest default-workers regression (likely the biggest issue)

cmd/evm/enginexrunner.go drops:

if workers <= 0 {
    // Knee of the wall-time vs. RAM curve on the full EEST engine_x set
    // (~64k tests / ~36k groups): workers=8 reaches plateau speedup at
    // ~4.3GB peak RSS. Higher values barely improve wall time and
    // values >24 risk MDBX virtual-memory exhaustion.
    workers = 8
}

Combined with WorkersFlag.Value = 1, this means evm enginextest <dir> (no flag) now runs at 1 worker instead of 8 — i.e., the typical EEST engine_x sweep gets ~8× slower by default. The PR title says "fix regressions in statetest cmd" but this is a new regression in enginextest. The deleted comment was load-bearing tuning lore for the engine_x workload; the unified default of 1 makes sense for statetest/blocktest but not here.

Options:

  • Keep the safety net in enginexrunner.go so a missing --workers still maps to 8.
  • Or explicitly document that EEST engine_x runs should always pass --workers=8 (preferably with a banner when workers == 1 && len(groups) > some_threshold).

2. stderr emission gate differs from the original

Previously (pre-#21058), the stderr emission was gated on --json (MachineFlag):

if jsonOut {
    fmt.Fprintf(os.Stderr, "{\"stateRoot\": ...}", ...)
}

After this PR (cmd/evm/staterunner.go):

emitStateRoot := ctx.Int(WorkersFlag.Name) <= 1

The new gate is broader: any sequential run now writes {\"stateRoot\":...} lines to stderr, even without --json. This adds a per-test stderr line for users who weren't expecting it (e.g. anyone redirecting 2>err.log). goevmlab always uses --json, so it's unaffected — but the spec of the gate quietly changed. Consider:

emitStateRoot := ctx.Bool(MachineFlag.Name) && ctx.Int(WorkersFlag.Name) <= 1

The added comment correctly justifies the workers <= 1 clause but is silent about why the --json clause was dropped.

3. Silent no-op for --workers=0 or negative values

cmd/evm/staterunner.go and cmd/evm/blockrunner.go both dropped the if workers <= 0 { workers = 1 } safety net. With WorkersFlag.Value = 1, the implicit default is fine, but the user passing --workers=0 explicitly now lands here:

for w := 0; w < workers; w++ {   // never enters
    wg.Add(1)
    go func() { ... }()
}
go func() { wg.Wait(); close(resultCh) }()  // immediate close
for fr := range resultCh { ... }            // empty

→ command silently produces zero results, exit code 0. Either restore the safety net or validate up-front and return an error.


What's right

  • execution/tests/testutil/state_test_util.go: returning the real root instead of empty.RootHash on checkedErr is the correct fix — RunNoVerify always computes commitment after applyErr, so a meaningful root is available even for intrinsic gas too low-type errors (issue #21080's second screenshot).
  • The unused common/empty import is correctly dropped.
  • Flag-default unification (0 → 1) and typo fix in Usage ("defaul") are clean.
  • The gate on workers <= 1 for stderr emission does correctly avoid the interleaving problem in parallel mode — that part of the reasoning is sound; only the --json half is missing.
  • In-Go test path (execution/tests/state_test.go) discards the returned root, so the state_test_util.go change has no effect there.

Suggested fix sketch

// cmd/evm/enginexrunner.go
workers := cliCtx.Int(WorkersFlag.Name)
if workers <= 0 {
    workers = 8 // keep engine_x-specific default, see comment above
}

// cmd/evm/staterunner.go + blockrunner.go
workers := ctx.Int(WorkersFlag.Name)
if workers <= 0 {
    return nil, fmt.Errorf(\"--workers must be > 0, got %d\", workers)
}

// cmd/evm/staterunner.go
emitStateRoot := ctx.Bool(MachineFlag.Name) && ctx.Int(WorkersFlag.Name) <= 1

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses regressions in evm statetest output formatting/contents (per #21080), primarily by ensuring the computed post-state root can still be surfaced for tooling that consumes the stderr JSONL stream.

Changes:

  • Return the computed post-state root from StateTest.Run even when the error classification fails, instead of returning an “empty” root.
  • Re-introduce per-subtest stateRoot emission to stderr (JSONL) for sequential statetest runs.
  • Change --workers default to 1 and simplify worker handling in several runners.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
execution/tests/testutil/state_test_util.go Returns computed root alongside error-checking outcomes to avoid losing stateRoot in error cases.
cmd/evm/staterunner.go Emits stateRoot JSONL on stderr (sequential mode) and simplifies passing --workers into parallel runner.
cmd/evm/main.go Changes --workers default to 1 and updates its help text.
cmd/evm/enginexrunner.go Removes the previous “auto/default workers” behavior for engine-x runs.
cmd/evm/blockrunner.go Simplifies passing --workers into blocktest parallel runner.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cmd/evm/staterunner.go Outdated
Comment thread cmd/evm/blockrunner.go Outdated
Comment thread cmd/evm/enginexrunner.go Outdated
Comment thread cmd/evm/staterunner.go Outdated
@taratorio

taratorio commented May 12, 2026

Copy link
Copy Markdown
Member Author

Context: Fixes issue #21080: (a) stateRoot no longer printed on stderr, (b) JSON result reports empty.RootHash instead of the real computed root when there's an error. Both root causes were introduced by #21058.

The core fixes are correct. Three observations worth addressing before merge:

1. enginextest default-workers regression (likely the biggest issue)

cmd/evm/enginexrunner.go drops:

if workers <= 0 {
    // Knee of the wall-time vs. RAM curve on the full EEST engine_x set
    // (~64k tests / ~36k groups): workers=8 reaches plateau speedup at
    // ~4.3GB peak RSS. Higher values barely improve wall time and
    // values >24 risk MDBX virtual-memory exhaustion.
    workers = 8
}

Combined with WorkersFlag.Value = 1, this means evm enginextest <dir> (no flag) now runs at 1 worker instead of 8 — i.e., the typical EEST engine_x sweep gets ~8× slower by default. The PR title says "fix regressions in statetest cmd" but this is a new regression in enginextest. The deleted comment was load-bearing tuning lore for the engine_x workload; the unified default of 1 makes sense for statetest/blocktest but not here.

Options:

  • Keep the safety net in enginexrunner.go so a missing --workers still maps to 8.
  • Or explicitly document that EEST engine_x runs should always pass --workers=8 (preferably with a banner when workers == 1 && len(groups) > some_threshold).

2. stderr emission gate differs from the original

Previously (pre-#21058), the stderr emission was gated on --json (MachineFlag):

if jsonOut {
    fmt.Fprintf(os.Stderr, "{\"stateRoot\": ...}", ...)
}

After this PR (cmd/evm/staterunner.go):

emitStateRoot := ctx.Int(WorkersFlag.Name) <= 1

The new gate is broader: any sequential run now writes {\"stateRoot\":...} lines to stderr, even without --json. This adds a per-test stderr line for users who weren't expecting it (e.g. anyone redirecting 2>err.log). goevmlab always uses --json, so it's unaffected — but the spec of the gate quietly changed. Consider:

emitStateRoot := ctx.Bool(MachineFlag.Name) && ctx.Int(WorkersFlag.Name) <= 1

The added comment correctly justifies the workers <= 1 clause but is silent about why the --json clause was dropped.

3. Silent no-op for --workers=0 or negative values

cmd/evm/staterunner.go and cmd/evm/blockrunner.go both dropped the if workers <= 0 { workers = 1 } safety net. With WorkersFlag.Value = 1, the implicit default is fine, but the user passing --workers=0 explicitly now lands here:

for w := 0; w < workers; w++ {   // never enters
    wg.Add(1)
    go func() { ... }()
}
go func() { wg.Wait(); close(resultCh) }()  // immediate close
for fr := range resultCh { ... }            // empty

→ command silently produces zero results, exit code 0. Either restore the safety net or validate up-front and return an error.

What's right

  • execution/tests/testutil/state_test_util.go: returning the real root instead of empty.RootHash on checkedErr is the correct fix — RunNoVerify always computes commitment after applyErr, so a meaningful root is available even for intrinsic gas too low-type errors (issue Statetest execution: missing stateroot on stderr #21080's second screenshot).
  • The unused common/empty import is correctly dropped.
  • Flag-default unification (0 → 1) and typo fix in Usage ("defaul") are clean.
  • The gate on workers <= 1 for stderr emission does correctly avoid the interleaving problem in parallel mode — that part of the reasoning is sound; only the --json half is missing.
  • In-Go test path (execution/tests/state_test.go) discards the returned root, so the state_test_util.go change has no effect there.

Suggested fix sketch

// cmd/evm/enginexrunner.go
workers := cliCtx.Int(WorkersFlag.Name)
if workers <= 0 {
    workers = 8 // keep engine_x-specific default, see comment above
}

// cmd/evm/staterunner.go + blockrunner.go
workers := ctx.Int(WorkersFlag.Name)
if workers <= 0 {
    return nil, fmt.Errorf(\"--workers must be > 0, got %d\", workers)
}

// cmd/evm/staterunner.go
emitStateRoot := ctx.Bool(MachineFlag.Name) && ctx.Int(WorkersFlag.Name) <= 1

@yperbasis the first 2 are all intentional and not issues

  1. the default is to have 1 worker for all 3 cmds preserving the logic from before my change. about enginextest being set to 8 workers by default previously - that is ok because it is a newly added cmd - im using it in the east-spec-test sharded CI but there I explicitly set the workers with --workers 8
  2. this diff is also intentional because now whether we print the state root is controlled by worker=1 (sequential -> then yes) vs parallel workers (it doesn't make sense to print it there because it will be intertwined/unclear which run it is for). coincidentally this also is backwards compatible with goevmlab and fixes the issue reported by Martin

@taratorio

Copy link
Copy Markdown
Member Author

@yperbasis point 3 is addressed in 8d327e8. ready for another review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Comment thread cmd/evm/staterunner.go
Comment thread cmd/evm/staterunner.go
Comment thread cmd/evm/blockrunner.go
@taratorio taratorio enabled auto-merge May 12, 2026 12:07
@taratorio taratorio added this pull request to the merge queue May 12, 2026
Merged via the queue into main with commit 92d4a28 May 13, 2026
109 of 111 checks passed
@taratorio taratorio deleted the worktree-fix-evm-fuzz-statetest branch May 13, 2026 00:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants