…3_PARALLEL flakes (erigontech#21136) (erigontech#21153)

Splits the substantive changes out of erigontech#21017 (which is left with just
the
serial-vs-parallel exec-mode CI matrix yamls). With these merged,
erigontech#21017's new
parallel CI legs go green.

## What's here

**Parallel-exec correctness — the SD-of-pre-existing-contract / recreate
/ fork-transition family**
- `execution/stagedsync, execution/state`: parallel
SD-of-pre-existing-contract fixes — `IteratePrefix(StorageDomain, addr)`
to emit the full `StoragePath=0` cascade on self-destruct (covers
genesis-allocated / prior-block storage that `vm.StorageKeys` doesn't
see), drop `StoragePath` writes for SD'd addresses, post-SD
baseline-is-zero handling, EIP-161 empty-removal gated by
SpuriousDragon, etc.
- `execution/state`: don't emit `StoragePath=0` writes from
`IBS.Selfdestruct`.
- `execution/stagedsync`: clear calc `Deleted` on a non-SD account-field
write even when zero (a 0-value transfer that re-creates a
previously-destroyed address as an empty account, pre-EIP-161).
- `execution/stagedsync`: install the per-block changeset accumulator
before any of the block's writes (idempotent
`ensureChangesetAccumulator`), so changeset capture is robust against
out-of-band block scheduling.
- `execution/stagedsync`: clean exit when a single-block fork-validation
batch already covered `maxBlockNum` (avoids a spurious
`ErrLoopExhausted` → "unexpected state step has more work").

**Performance**
- `execution/stagedsync`: O(1) `CollectorWrites` fee-balance update.
After a tx finalizes, `nextResult` re-applies fee-adjusted balances into
`result.CollectorWrites`; `CollectorWrites` is a flat slice, so doing
that per-`addWrites`-entry was O(len(addWrites)·len(CollectorWrites)).
For a block-end IBS reconstruction both are ~one entry per account the
block touched, so a block whose tx pays ~100k accounts
(`execution/tests`' `invalid-receipt-hash-high-mgas` corner) hit ~10^10
comparisons — `TestInvalidReceiptHashHighMgas` ran >11 min under
`EXEC3_PARALLEL` + `-race` and tripped `mock_cl`'s 10-min per-call
deadline (serial: ~10 s). Index `CollectorWrites`' `BalancePath` entries
by address once (first match wins, mirroring the old `SetBalance` linear
scan) and update in O(1); the test now matches serial. Removed the
now-dead `VersionedWrites.SetBalance`.

**Test gating for residual `EXEC3_PARALLEL` flakes — tracked in erigontech#21136**
- `t.Skip`/`SkipLoad` under `dbg.Exec3Parallel` for:
`TestEIP7708BurnLogWhenCoinbaseSelfDestructs`,
`TestLegacyBlockchain/ValidBlocks/bcEIP3675/tipInsideBlock`, and
`TestValidateChainAndUpdateForkChoiceWithSideForksThatGoBackAndForwardInHeight`
(intermittent wrong-trie-root under heavy concurrent test load — the
parallel reorg/commitment path is timing-sensitive). Serial coverage of
all of these is unaffected. (The `eest_blockchain` `SkipLoad`s from the
original gating commit are no longer applicable — that package was
removed by erigontech#21092 — and were dropped accordingly.)

## Notes
- Cleanly cherry-picked onto current `main`. `make lint` clean; `make
test-short` clean; `execution/state` + `execution/stagedsync` +
`execution/execmodule` tests pass serial **and** `EXEC3_PARALLEL=true`;
`TestInvalidReceiptHashHighMgas` passes both modes (~12 s parallel, no
`-race`).
- The serial/parallel CI matrix that exercises the new parallel legs
lives in erigontech#21017 — once this merges, erigontech#21017 will be trimmed to just those
workflow yamls and rebased on top.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: milen <94537774+taratorio@users.noreply.github.com>

Main's #21092 removed both execution-eest-blockchain and
execution-eest-devnet from tools/test-groups when spec tests were
extracted out of make test-all. The merge resolution mistakenly kept
the PR-side branch with the devnet entry plus a stale "Temporarily
disable the blockchain shard" comment. Drop the devnet entry and the
comment; keep only the PR's own execution-eest-zkevm contribution.

Per PR #21629 review (the #21092 pattern): the ~22k-test zkevm
execution-witness conformance suite must not run under `go test` /
`make test-all`. Replace it with a `zkevmtest` cmd/evm subcommand executed as
parallel `eest-spec-tests` shards (incl. a -race variant).

- cmd/evm/zkevmrunner.go: new `zkevmtest` subcommand — imports each EEST zkevm
  blockchain_tests fixture, queries debug_executionWitness (canonical) per
  block, multiset-compares state/codes/headers. Expected-fail patterns for the
  known #21563/#21566 divergences are ported via TestMatcher so the shard
  budget stays 0.
- testutil: add BlockTest.RunWithTesterCLI (nil tb, caller-owned tester) and
  TestMatcher.CheckFailureForName for CLI runners; RunCLI now shares the path.
- shards: add zkevm-witness + zkevm-witness-race-all to eest-spec-shards.yml,
  route them in run-eest-spec-test.sh, cache the eest_zkevm tarball in
  test-eest-spec.yml, and add Makefile targets provisioning test-fixtures-zkevm.
- remove the go-test suite (execution/tests/eest_zkevm_witness), its
  execution-eest-zkevm test-group, the zkevm steps in test-all-erigon-race.yml,
  and the GOTEST_PACKAGES filter (back to ./...).

…21629)

Adds the zkEVM (canonical) conformance runner for
`debug_executionWitness`, the legacy-mode field population, the optional
`mode` request param, and a format spec.

## Changes

- **`zkevmtest` runner** — a `cmd/evm` subcommand (mirroring
`staterunner`/`blockrunner`/`enginexrunner`) that checks
`debug_executionWitness` (canonical mode) against the
`ethereum/execution-spec-tests` zkevm@v0.4.0 corpus (~22.5k tests). Runs
as the `zkevm-witness-race` shard in the `eest-spec-tests` workflow, not
under `go test`. Known divergences (erigontech#21563, erigontech#21566) are absorbed in the
runner so the shard budget stays 0.
- **legacy `keys`/`codes`** — `keys` holds the 20B address / 32B
storage-slot preimages; `codes` holds pre-state bytecode incl. EIP-7702
designators and the empty-code entry.
- **`mode` request param** (`legacy`/`canonical`).
- **Spec** at `docs/plans/witness-legacy-mode-spec.md`, re-verified
against code.

## erigontech#21487

Carries erigontech#21487's suite plus the fixes that make it pass; erigontech#21487 is
redundant once CI is green here. The open question from erigontech#21487 — whether
a fixture-gated suite belongs in the required race-gate — is resolved:
it's now a `zkevmtest` runner shard in the `eest-spec-tests` workflow
(the erigontech#21092 pattern), not a `go test` package.

## Mainnet legacy soak vs reth-legacy

Rolling soak near tip (block ~`25.25M`, ~3.7k blocks checked):

- 0 health failures — no 500s, no root mismatches; every witness passes
`verifyWitnessStateless`.
- 90.6% strict set-equal on `{keys, codes, state}`.
- Divergences are almost all over-inclusion (204 blocks `keys`, 86
`codes`, 62 `state`) — a few extra nodes per block, still re-executing
to the correct root. 3 blocks show minor `keys`/`codes` under-inclusion,
none affecting `verifyWitnessStateless`.

…h#21640)

## Problem

`tools/run-eest-spec-test.sh` invokes the cmd/evm runner with `|| true`
and decides shard pass/fail from the `--jsonout` report alone. That
design is correct for the non-race shards (the runners always exit 0 and
report via JSON; crashes surface as missing/truncated JSON, which the
script catches via the `total == 0` check and `set -euo pipefail` on
jq).

It is wrong for the `*-race-*` shards: the Go race runtime reports
detected races to **stderr** and overrides the process exit code to
**66** (even through `os.Exit(0)`), while the run completes normally and
the JSON stays fully valid with every test passing. The `|| true`
discards that exit code, so a race shard whose binary detected a data
race still went green — the `WARNING: DATA RACE` report only visible in
the job log. The race detector's whole value is catching races that
*don't* happen to corrupt results, so this silently weakened the
guarantee of every race shard:

```
blocktests-stable-race-{pre-cancun,cancun,prague,osaka}-{sequential,parallel}
blocktests-devnet-race-amsterdam
```

(and `zkevm-witness-race-all` once erigontech#21629 lands).

Pre-existing since the shards were introduced in erigontech#21092; surfaced while
reviewing erigontech#21629, which doubles the corpus running under the race
detector.

## Fix

Capture the runner's exit code and fail the shard explicitly on 66 with
a pointer to the race report above it in the log. All other exit-code
tolerance is unchanged.

## Verification (local repro)

Built a minimal `-race` Go binary containing a genuine data race that
emits valid passing JSON (`[{"name":"t1","pass":true}]`) and calls
`os.Exit(0)`, then ran it through the real script as `EVM_BIN` against
`blocktests-stable-race-cancun-sequential`:

- **Before** (unmodified script): binary prints `Found 1 data race(s)`
and exits 66 → script prints `ran 1 tests, 0 failed` and exits **0**
(green despite the race).
- **After**: same run exits **1** with `ERROR: data race detected (race
runtime exit code 66); see WARNING: DATA RACE in the log above`.

Regression matrix with stub binaries through the patched script,
confirming only the race case changes behavior:

| Scenario | Before | After |
|---|---|---|
| race detected (exit 66, valid passing JSON) | exit 0 | **exit 1** |
| clean pass (exit 0) | exit 0 | exit 0 |
| non-zero exit ≠ 66 with valid JSON | exit 0 (tolerated by design) |
exit 0 |
| test failures over budget | exit 1 | exit 1 |

`shellcheck --severity=warning` (the lint-workflow invocation) is clean
on the modified script; `make lint` clean.

The race shards running on this PR exercise the patched path end-to-end
in CI (race-free run → `rc=0` → behavior identical to before).

## Note on TDD

This is a shell-script CI guard with no test harness; the Red → Green
cycle above (reproduce the masked exit code through the unmodified
script, then flip it with the fix) stands in for a checked-in failing
test.

## End-to-end verification with a real race in erigon code

Repeated the verification with the full production pipeline instead of
stub binaries: planted a deliberate data race in
`IntraBlockState.CommitBlock` (two unsynchronized goroutines
incrementing a package var, executed once per imported block — never
committed), built the real `evm.race` via `make evm.race`, and ran a
single real EEST fixture
(`cancun/eip1153_tstore/test_basic_tload_after_store.json`, one
`fork_Cancun` test) through the
`blocktests-stable-race-cancun-sequential` shard:

- **old script** (origin/main): race report correctly attributes
`intra_block_state.go` in `CommitBlock.func1`, binary prints `Found 1
data race(s)` and exits 66 → script prints `ran 1 tests, 0 failed`,
**exit 0** — green despite the race.
- **this PR's script**: same binary, same fixture → **exit 1**, last
line `ERROR: data race detected (race runtime exit code 66); see
WARNING: DATA RACE in the log above`.
- **control**: race reverted, `evm.race` rebuilt clean, same fixture
through this PR's script → 0 race warnings, `ran 1 tests, 0 failed`,
**exit 0** — no false positive.