Skip to content

GHA: add three old (gcc 6, 7, 9) mingw-w64 jobs#13759

Closed
vszakats wants to merge 2 commits intocurl:masterfrom
vszakats:gha-windows-gcc7-on-new-runner
Closed

GHA: add three old (gcc 6, 7, 9) mingw-w64 jobs#13759
vszakats wants to merge 2 commits intocurl:masterfrom
vszakats:gha-windows-gcc7-on-new-runner

Conversation

@vszakats
Copy link
Member

@vszakats vszakats commented May 23, 2024

Re-implement old mingw-w64 jobs in GHA. This allows to use the latest
Windows runners, replacing Windows Server 2012 R2 (gcc 6) and Windows
Server 2016 (gcc 7, 9) with Windows Server 2022.

GHA runners are also significantly faster, and allow running tests in
parallel (-j14). It also offloads 3 more long-running jobs from
AppVeyor CI.

These jobs download (then cache) the mingw-w64 packages from their
original location, which allows flexibility in choosing which versions
and flavours (win32/POSIX, SEH/DWARF, 64/32-bit) we want to test in CI.
The new jobs use these distros:

I matched existing AppVeyor job configs, with these differences:

  • gcc 6.4.0 instead of 6.3.0.
    (same distro as on AppVeyor, but the latest bugfix release)
  • gcc 9.5.0 instead of 9.1.0 and a different (but compatible) binary distro.
    (in AppVeyor this relies on an old MSYS2 pre-installed on the runner)
  • using win32 builds instead of posix for gcc 6.4.0 and 7.3.0.
  • websockets enabled.
  • always build examples.
  • always build tests (this wasn't done for 6.4.0 with AppVeyor CI).

I did not replicate existing test exclusions, and oddly enough the few
failures (so far) were different from MSYS2 jobs and also from their
AppVeyor CI counterparts.

Also:

  • delete redundant (default) -u option from cygpath calls.
  • allow matrix options to override default ones in CMake.
  • detect and use Windows-supplied curl for TFLAGS -ac option.
    (it's available in modern runners.)
  • delete the 3 AppVeyor CI jobs now replicated in GHA.
  • appveyor: prefer SYSTEMROOT over WINDIR.
  • tidy-up quotes.

Job performance:

                                                                 AppVeyor  GHA
                                                                           w/examples
                                                                           w/tests
                                                                 --------  ----------
CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity   1m25s     8m50s
CMake, mingw-w64, gcc 7, Debug, x64, Schannel, Static, Unicode   31m45s     9m39s
CMake, mingw-w64, gcc 9, Debug, x64, Schannel, Static            28m25s    13m38s

Based on these runs:
https://ci.appveyor.com/project/curlorg/curl/builds/49880799
https://github.com/curl/curl/actions/runs/9218292508

Notice that building examples and tests is time consuming.

We can tweak any build parameter as necessary to make them more useful
and/or without clogging the job queue or introducing flakiness.

Closes #13759


  • make caching work. [Right, to make it work, path: must use Windows path notation. Not even ~ works, despite the documentation and apparently all cache examples are meant for *nix runners.]
  • cleanup the PR, maybe move off some changes to a different PR?
  • delete jobs from AppVeyor CI.
  • saving the cache works now to a point, but then complains that: 'Failed to save: Unable to reserve cache with key Windows-mingw-w64-6.4.0-i686, another job may be creating this cache. More details: Cache already exists. Scope: refs/pull/13759/merge, Key: Windows-mingw-w64-6.4.0-i686, Version: 7029f2ef7decc13f34a275491ff299ebeebf54f184f7c19a012a4c41c363a793', then fails to find the cache with: 'Cache not found for input keys: Windows-mingw-w64-6.4.0-i686'. Perhaps this will be gone when merging this to master? Ignoring this for now, for better or worse. → It fixed itself after merging to master.

@vszakats vszakats added Windows Windows-specific CI Continuous Integration labels May 23, 2024
@vszakats vszakats marked this pull request as draft May 23, 2024 12:57
@github-actions github-actions bot added the tests label May 23, 2024
@vszakats vszakats force-pushed the gha-windows-gcc7-on-new-runner branch from 45ef829 to 8c06af0 Compare May 24, 2024 10:43
@vszakats vszakats marked this pull request as ready for review May 24, 2024 10:56
@vszakats vszakats changed the title GHA: try old mingw-w64 on new runner GHA: add three old (gcc 6, 7, 9) mingw-w64 jobs May 24, 2024
@vszakats vszakats force-pushed the gha-windows-gcc7-on-new-runner branch from 12fca29 to 9a3ee70 Compare May 24, 2024 11:59
@vszakats vszakats force-pushed the gha-windows-gcc7-on-new-runner branch from 9a3ee70 to 4813950 Compare May 24, 2024 20:19
@vszakats vszakats mentioned this pull request May 24, 2024
3 tasks
@vszakats vszakats force-pushed the gha-windows-gcc7-on-new-runner branch from 4813950 to ab8d437 Compare May 24, 2024 20:27
@vszakats vszakats closed this in 0914d8a May 27, 2024
@vszakats vszakats deleted the gha-windows-gcc7-on-new-runner branch May 27, 2024 10:17
vszakats added a commit to vszakats/curl that referenced this pull request May 28, 2024
These exclusions came from the AppVeyor CI config, but they do pass now
and they are static tests with no flakiness risk.

Follow-up to 0914d8a curl#13759
Closes #xxxxx
vszakats added a commit that referenced this pull request May 28, 2024
These exclusions came from the AppVeyor CI config, but they do pass now
and they are static tests with no flakiness risk.

Follow-up to 0914d8a #13759
Closes #13817
vszakats added a commit that referenced this pull request Jun 2, 2024
- stop altering the `PATH` via `GITHUB_ENV`. This confused the
  `actions/cache` post-job, which needs to run in the exact same
  environment as its pre-job, to have a consistent cache entry "version"
  hash. Altering the `PATH` via `GITHUB_ENV` spills into the the
  post-job and breaks this hash. GHA doesn't reset the env automatically
  and I have not found a way to do it manually.

- add double-quotes where missing.

- move cache directory under `USERPROFILE` to not rely on absolute
  paths.

- make cache directory flatter and versionless.

Follow-up to 0914d8a #13759
Closes #13856
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Continuous Integration tests Windows Windows-specific

Development

Successfully merging this pull request may close these issues.

1 participant