Skip to content

ci: harden apt-get installs against transient mirror failures#65

Merged
krazyjakee merged 1 commit into
masterfrom
claude/fix-flaky-ci-setup-Ix1NQ
May 13, 2026
Merged

ci: harden apt-get installs against transient mirror failures#65
krazyjakee merged 1 commit into
masterfrom
claude/fix-flaky-ci-setup-Ix1NQ

Conversation

@krazyjakee

Copy link
Copy Markdown
Contributor

The Windows Export & Smoke Test job has been failing intermittently on
"Install system libraries and Wine" — apt-get hangs (mirror or i386
dependency resolution) and the 10-minute step timeout fires before the
install can complete. The bare apt-get update && apt-get install -qq
pattern has no retries and -qq hides any progress, so transient
network issues become hard failures with no diagnostic output.

Introduces .github/scripts/apt-install.sh, a wrapper that:

  • Configures apt with Acquire::Retries=5 and short connection
    timeouts so apt fails fast on stuck mirrors and retries internally.
  • Retries the full update + install cycle up to 3 times with
    exponential backoff, recovering dpkg/apt state between attempts.
  • Drops -qq so a hang leaves a useful tail in the log.

Routes every apt-get install in ci.yml and release.yml through the
wrapper, and bumps the Wine step's timeout from 10 to 15 minutes to
absorb the slowest healthy cold-cache install.

The Windows Export & Smoke Test job has been failing intermittently on
"Install system libraries and Wine" — apt-get hangs (mirror or i386
dependency resolution) and the 10-minute step timeout fires before the
install can complete. The bare `apt-get update && apt-get install -qq`
pattern has no retries and `-qq` hides any progress, so transient
network issues become hard failures with no diagnostic output.

Introduces `.github/scripts/apt-install.sh`, a wrapper that:
  - Configures apt with Acquire::Retries=5 and short connection
    timeouts so apt fails fast on stuck mirrors and retries internally.
  - Retries the full `update + install` cycle up to 3 times with
    exponential backoff, recovering dpkg/apt state between attempts.
  - Drops `-qq` so a hang leaves a useful tail in the log.

Routes every apt-get install in ci.yml and release.yml through the
wrapper, and bumps the Wine step's timeout from 10 to 15 minutes to
absorb the slowest healthy cold-cache install.
@krazyjakee krazyjakee merged commit d32a082 into master May 13, 2026
6 checks passed
@krazyjakee krazyjakee deleted the claude/fix-flaky-ci-setup-Ix1NQ branch May 13, 2026 12:03
krazyjakee added a commit that referenced this pull request May 13, 2026
…st (#70)

The previous hardening (#65) addressed transient apt-get failures but
the Windows Export & Smoke Test job is still timing out at 15 minutes
in "Install system libraries and Wine". The root cause isn't transient
mirror noise — it's structural:

  - `dpkg --add-architecture i386` roughly doubles the apt manifest
    download and pulls hundreds of MB of i386 dependencies on top of
    wine64 itself.
  - Retrying inside a 15-minute step timeout cannot fix a >15-minute
    install regardless of how many backoff attempts we make.

This commit takes a more complete workaround: Wine is only needed for
the optional "Validate Windows binary starts (Wine)" step. The Windows
export itself, addon-binary verification, and artifact upload — the
real value of this job — all work without Wine. So:

  - Split the install: audio libraries become a separate hard-fail
    step; Wine becomes a soft-fail step with `continue-on-error` and
    a 10-minute timeout.
  - Drop `dpkg --add-architecture i386`. wine64 is a pure 64-bit
    runtime, and Godot's Windows export is 64-bit, so i386 wasn't
    actually needed.
  - Gate the smoke test on the Wine install having succeeded, and
    emit a clear warning + step summary entry when it gets skipped
    so degraded-mirror skips don't look like green-on-broken-export.

Also removes `Acquire::http::No-Cache "true"` from apt-install.sh —
that directive was forcing apt to re-download manifests on every
retry attempt, which is the opposite of what we want when recovering
from a flaky mirror.

Co-authored-by: Claude <noreply@anthropic.com>

https://claude.ai/code/session_01JYWykzyHjAnM24hANLgwyX

Co-authored-by: Claude <noreply@anthropic.com>
krazyjakee added a commit that referenced this pull request Jun 8, 2026
Builds on the #53 update checker toward the reference's auto-update flow:

- Hourly periodic check (in addition to the throttled startup check), gated
  on the auto-update-check setting.
- Pre-release awareness: an update is offered only for stable releases unless
  this build is itself a pre-release (AppRelease now parses `prerelease`).
- Skip vs dismiss: "Skip this version" persists (new `skippedUpdateVersion`
  setting) and permanently suppresses that version; the banner ✕ dismisses
  for the session only. Banner respects both.
- Platform-aware download: AppRelease parses release `assets`; the Updates
  page's button links straight to the asset matching the current OS/arch
  (Android .apk, Windows .exe/.msi/.zip, macOS .dmg/.pkg/.zip, Linux
  .AppImage/.tar.gz/.deb/.rpm), falling back to the release page. Web shows a
  "refresh to update" note.

Deferred (documented in code): true in-place binary self-replacement +
auto-restart (Windows .bat handoff, macOS .app swap, Linux binary replace,
Android installer intent) — it needs untestable per-platform native
machinery; for now updates hand off to the OS via the asset download.

Closes #65.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants