ci(replay-tests): add per-platform headless replay regression tests#125
Merged
Conversation
Adds a new reusable workflow (replay-tests.yml) that runs deterministic headless replay tests after each successful ZH build. Setup per run: - Downloads built binary artifact from the upstream build job - Checks out fbraz3/generalsx-test-files and extracts game assets using the ASSETS_KEY secret (password-protected 7z, no raw assets) - Checks out fbraz3/GeneralsXReplays and installs custom maps + platform-specific replays (linux_*.rep / macos_*.rep) to the standard user data directory Linux specifics: - Installs mesa-vulkan-drivers (lavapipe) for software Vulkan in CI - Sets LD_LIBRARY_PATH to the bundle root (flat lib layout) - Forces VK_ICD_FILENAMES to lavapipe ICD macOS specifics: - Uses MoltenVK bundled inside the .app artifact (no Vulkan SDK install) - Sets VK_ICD_FILENAMES/VK_DRIVER_FILES to bundled MoltenVK_icd.json - Sets CNC_GENERALS_ZH_PATH to the bin/ dir with extracted assets ci.yml changes: - replay-test-linux: runs after build-linux-gzip succeeds (ZH only) - replay-test-macos: runs after build-macos succeeds (ZH only) - ci-summary: updated to report replay test results and fail CI on replay regressions Only ZH is tested (Generals base has no replays yet). Cross-platform replay compatibility is a known future work item.
GitHub Actions checkout path must be under GITHUB_WORKSPACE. Replace absolute /tmp checkout paths with workspace-relative directories for assets and replay repositories. Also install coreutils on macOS and use gtimeout for replay execution on macOS runners.
The replay workflow inferred APP_RESOURCES from a discovered binary path, which could resolve to an invalid location depending on tar layout. Resolve .app first, then set: - APP_RESOURCES=<app>/Contents/Resources - GAME_DIR=<resources>/bin - GAME_BIN=<resources>/bin/GeneralsXZH Also add explicit validation with directory dumps for faster CI diagnosis.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds automated deterministic replay regression tests to CI. After each successful ZH build, the pipeline downloads the built artifact, fetches encrypted game assets and platform-specific replay files, and runs every replay headlessly — failing CI if any replay exits non-zero.
New:
.github/workflows/replay-tests.ymlReusable workflow called per platform with inputs:
platform:linuxormacosgame-artifact: artifact name from the upstream build jobASSETS_KEYsecret: password for the encryptedgeneralszh.7zWhat each run does
linux-generalsxzh-linux64-bundleormacos-generalsxzh-app)fbraz3/generalsx-test-files(via LFS) and extractsgeneralszh.7zwithASSETS_KEYfbraz3/GeneralsXReplaysand installs:Maps/dir (includesMapCache.ini)linux_*.rep/macos_*.rep) → user dataReplays/dir-headless -replay <path>and collects pass/failPlatform specifics
Linux:
mesa-vulkan-drivers(lavapipe) — software Vulkan, no GPU requiredLD_LIBRARY_PATHset to the bundle root (flat.solayout)VK_ICD_FILENAMESforced to lavapipe ICDmacOS:
.appartifact — no Vulkan SDK install neededVK_ICD_FILENAMES/VK_DRIVER_FILESpointed at bundledMoltenVK_icd.jsonCNC_GENERALS_ZH_PATHset to thebin/dir alongside the extracted assetsChanges to
ci.ymlreplay-test-linuxbuild-linux-gzipsucceedslinux_*.repvs Linux binaryreplay-test-macosbuild-macossucceedsmacos_*.repvs macOS binaryci-summaryupdated to report replay results and fail CI on regressions.Notes
GeneralsMD) is tested — Generals base has no replays yetASSETS_KEYsecret must be set in the repository secrets for asset extraction to workfbraz3/generalsx-test-filesandfbraz3/GeneralsXReplaysmust be accessible with the defaultGITHUB_TOKEN(public) or a PAT if private