Skip to content

ci: tolerate consensus spec test download failure#19037

Merged
anacrolix merged 3 commits into
mainfrom
anacrolix/ci-tolerate-spec-download-failure
Feb 10, 2026
Merged

ci: tolerate consensus spec test download failure#19037
anacrolix merged 3 commits into
mainfrom
anacrolix/ci-tolerate-spec-download-failure

Conversation

@anacrolix

@anacrolix anacrolix commented Feb 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Extracts wget into a new download-spec Makefile target in cl/spectest
  • Splits the CI workflow into a download step (continue-on-error: true) and a test step that only runs if the download succeeded
  • Transient GitHub CDN errors (e.g. 502 Bad Gateway) now produce a warning instead of failing the job

Test plan

  • Verify CI passes when the download succeeds (tests run normally)
  • Verify make tests still works locally (it depends on download-spec)

🤖 Generated with Claude Code

The consensus spec CI downloads test fixtures from GitHub releases via
wget. Transient GitHub CDN errors (e.g. 502) cause the entire job to
fail even though the code is fine. Split the wget into a separate
download-spec Makefile target and CI step with continue-on-error, so a
download failure produces a warning instead of blocking PRs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@anacrolix

Copy link
Copy Markdown
Contributor Author

@copilot did this change cause the RPC CI failure in commit 633386c? If not how do I avoid false positive?

Copilot AI commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

@anacrolix I've opened a new pull request, #19064, to work on those changes. Once the pull request is ready, I'll request review from you.

@anacrolix anacrolix merged commit ca69ab5 into main Feb 10, 2026
20 of 21 checks passed
@anacrolix anacrolix deleted the anacrolix/ci-tolerate-spec-download-failure branch February 10, 2026 09:40
Sahil-4555 pushed a commit to Sahil-4555/erigon that referenced this pull request Feb 11, 2026
## Summary
- Extracts `wget` into a new `download-spec` Makefile target in
`cl/spectest`
- Splits the CI workflow into a download step (`continue-on-error:
true`) and a test step that only runs if the download succeeded
- Transient GitHub CDN errors (e.g. 502 Bad Gateway) now produce a
warning instead of failing the job

## Test plan
- [ ] Verify CI passes when the download succeeds (tests run normally)
- [ ] Verify `make tests` still works locally (it depends on
`download-spec`)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sahil-4555 pushed a commit to Sahil-4555/erigon that referenced this pull request Feb 11, 2026
## Summary
- Extracts `wget` into a new `download-spec` Makefile target in
`cl/spectest`
- Splits the CI workflow into a download step (`continue-on-error:
true`) and a test step that only runs if the download succeeded
- Transient GitHub CDN errors (e.g. 502 Bad Gateway) now produce a
warning instead of failing the job

## Test plan
- [ ] Verify CI passes when the download succeeds (tests run normally)
- [ ] Verify `make tests` still works locally (it depends on
`download-spec`)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
anacrolix added a commit that referenced this pull request Mar 12, 2026
anacrolix added a commit that referenced this pull request Mar 12, 2026
The previous approach (#19037) split the step into a download step with
continue-on-error and a test step that only ran if download succeeded.
This was wrong: if the download failed, the test step was skipped
entirely, so mainnet tests never ran either — the job just silently
passed without doing anything useful.

The correct behaviour is to tolerate setup/download failures but still
run mainnet tests regardless, and fail the job if mainnet tests fail.

Use a bash if statement: run make tests, and if it fails emit a
::warning:: annotation and skip mainnet; if it succeeds, run mainnet
normally. This way transient CDN errors produce a visible warning
instead of a silent skip, and actual test regressions still fail the job.

Example of incorrect behaviour: https://github.com/erigontech/erigon/actions/runs/22987591330/job/66741331639#step:5:20
Reverts: #19037
AskAlexSharov pushed a commit that referenced this pull request Mar 16, 2026
Reverts #19037 and replaces the approach.

## What was wrong

#19037 split the step into a separate download step (`continue-on-error:
true`) and a test step. The download step passed as intended, but the
test step ran `make tests` which had `download-spec` as a Make
dependency — so it re-ran the download, which failed.

Example:
https://github.com/erigontech/erigon/actions/runs/22987591330/job/66741331639#step:5:20

## Fix

Revert the Makefile split (curl back inline in `tests`) and use a single
bash `if` statement in the workflow:

- If `make tests` fails → emit a `::warning::` annotation and skip
mainnet. The step exits 0, so the job passes.
- If `make tests` succeeds → run `make mainnet` normally. A failure here
fails the job.

Transient download failures produce a visible warning rather than a
failing job, and actual test regressions still fail the job.
lupin012 pushed a commit that referenced this pull request Mar 17, 2026
Reverts #19037 and replaces the approach.

## What was wrong

#19037 split the step into a separate download step (`continue-on-error:
true`) and a test step. The download step passed as intended, but the
test step ran `make tests` which had `download-spec` as a Make
dependency — so it re-ran the download, which failed.

Example:
https://github.com/erigontech/erigon/actions/runs/22987591330/job/66741331639#step:5:20

## Fix

Revert the Makefile split (curl back inline in `tests`) and use a single
bash `if` statement in the workflow:

- If `make tests` fails → emit a `::warning::` annotation and skip
mainnet. The step exits 0, so the job passes.
- If `make tests` succeeds → run `make mainnet` normally. A failure here
fails the job.

Transient download failures produce a visible warning rather than a
failing job, and actual test regressions still fail the job.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants