CI: update toolchains to use latest GitHub runner images and rewrite set_matrix.py without polly#780
Conversation
b23ca88 to
c3e7d55
Compare
|
from #781
|
|
need to update |
Update from the deprecated ubuntu-20.04 images to the currently latest ubuntu-24.04 image. In the process update the `gcc` version from 7, 8, 9 to 12, 13, 14.
f00adc4 to
edbac40
Compare
|
basic functionality restored, Now need to update custom found the following files: $ find cmake/projects/ -name "*.sh" -o -name "*.json" -o -name "*.cmd"
cmake/projects/filament/ci/build.sh
cmake/projects/filament/ci/matrix.json
cmake/projects/Qt/generate.sh
cmake/projects/pthreads-win32/ci/matrix.json
cmake/projects/occt/ci/build.sh
cmake/projects/freetype-gl/ci/build.sh
cmake/projects/freetype-gl/ci/matrix.json
cmake/projects/Boost/scripts/create-boost-1-66-ios-fatlib.sh
cmake/projects/Boost/scripts/create-boost-ios-fatlib.sh
cmake/projects/Boost/ci/matrix.json
cmake/projects/Boost/generate.shso will have to fix the following projects:
To test I'll locally try to successfully run the following command: python .github/workflows/set_matrix.py Eigen filament pthreads-win32 occt freetype-gl Boost |
3dc0793 to
536cffc
Compare
Noone looks at them and it is a hassle to implement in `set_matrix.py`
- set_matrix: add --projects override for local debugging - set_matrix: add optional --output parameter - set_matrix: default output to stdout without indent - set_matrix: create toolchain files first steps non-complete - set_matrix: handle specific android toolchains - set_matrix: handle CMAKE_C_STANDARD and set undetectable - set_matrix: toolchain creation for vs-xx-xxxx, no generator yet - set_matrix: toolchain creation for mingw and msys - set_matrix: newlines and c11 undetectable - set_matrix: comment about generator fixed and libc++ closing ) fixed - set_matrix: use android-ndk-api - set_matrix: set VCVARSALL and generator - set_matrix: add project_name to error message - set_matrix: update path to vcvarsall.bat for vs-17-2022 - set_matrix: info about deprecation of vs-16-2019 - set_matrix: specified scripts are mandatory, no default build scripts - set_matrix: no script for hunter_tests, but VERBOSE=True - set_matrix: write out project name for CI job name - set_matrix: set default python version for all jobs
- build.py: copy from jenkins.py and black format - build.py: call cmake in script - build.py: sys.exit not just exit - build.py: fix cmake calling - build.py: as posixpath for args - build.py: use full path for TOOLCHAIN - build.py: toolchain filepath as posix path - build.py: all paths as posix - build.py: alsways build with --config Release - build.py: hunter_sha1 is str, not path - build.py: only build Release config to save time - build.py: CMP0069 is on per default since 3.9 - build.py: no chdir, remove all_release flag - build.py: use cdir instead of recreate - build.py: try without archive, maybe fixes Windows Store - build.py: disable zip/archive creation, messes with Windows 10 Store - build.py: fix release type, delete build dir, no archive - build.py: handle optional SCRIPT environment variable - build.py: allow hunter_tests as toolchain, special handling - build.py: on main and master and GITHUB_USER_PASSWORD upload cache - build.py: allow toolchain name and path and find file if needed
b278c94 to
28c5ac7
Compare
|
removed the |
- ci: matrix: remove ndk from toolchain, depends on github runner image - ci: matrix: use android-ndk-api-xxx - ci: matrix: macos: move to macos-15 - ci: matrix: windows: move to vs-17-2022 on windows-2022 image - ci: matrix: use Windows SDK 22000, as it is one of the available - ci: matrix: remove python and script from default matrix, add comments
- ci: upload generated toolchains folder - ci: on windows run VCVARSALL if needed - ci: download artifacts for .github/toolchains folder - ci: windows: download artifacts downloads folder directly, forgets full path - ci: linux: download artifacts downloads folder directly, forgets full path - ci: linux: use build.py script - ci: run build.py for windows build as well - ci: use build.py for hunter_tests as well - ci: use matrix.project - ci: use TOOLCHAIN with just name, remove hunter_tests job - ci: on error show output of set_matrix.py - ci: actually pass matrix.script as SCRIPT env-variable
also update docs such that custom build scripts are called by build.py
28c5ac7 to
93a5d9c
Compare
|
something is not yet working with the |
|
made the |
* Replace boostorg.jfrog.io to archives.boost.io URLs (cpp-pm#771) Fix for cpp-pm#770 * Update minimum CMake to v3.10 (cpp-pm#769) * Update minimum CMake to v3.10 * Update docs with new CMake minimum 3.10 * Update gate submodule to v0.11.0 Updates minimum CMake to v3.10 * Update Boost to 1.84.0, 1.85.0 and 1.86.0 (cpp-pm#775) * Update 'Boost' to v1.84.0 Release notes: https://www.boost.org/users/history/version_1_84_0.html * Update libraries and tests for Boost v1.84.0 * Add example for boost::conbalt * Add Boost::cobalt to test matrix * Add example for Boost::redis * Add Boost::redis to test matrix * Update 'Boost' to v1.85.0 Release notes: https://www.boost.org/users/history/version_1_85_0.html * Update libraries and tests for Boost v1.85.0 * Add example for Boost::charconv * Add Boost::charconv to test matrix * refactor: move example for BoostProcess from Boost-process to BoostProcess Renaming because the Boost::process lib in Boost became a lib for v2 * Update 'Boost' to v1.86.0 Release notes: https://www.boost.org/users/history/version_1_86_0.html * Update libraries and tests for Boost v1.86.0 * Add example for Boost::predef * It seems something changed about Boost::predef in 1.86.0. It is listed as a lib but not build as a lib. * Add Boost::predef to test matrix * Add Boost::process to test matrix * Add example for v2 of Boost::process * Boost::process changed in the 1.86.0 release. * v1 is header only * v2 is a library --------- Co-authored-by: André Falk <andre.falk@tutanota.com> Co-authored-by: simbahebinbo <lansheng228@163.com> * hunter_get_lang_standard_flag: Add C++ 23/26 and C17/C23 check (cpp-pm#779) Fixes: cpp-pm#756 * Add support for VS2022 Build Tools (cpp-pm#783) * CI: update toolchains to use latest GitHub runner images and rewrite set_matrix.py without polly (cpp-pm#780) Update all used GitHub runner images (Linux, Windows and Mac) and update the `set_matrix.py` script to work without the `polly` toolchains. - Update from the deprecated ubuntu-20.04 images to the currently latest ubuntu-24.04 image. In the process update the `gcc` version from 7, 8, 9 to 12, 13, 14. - fixes: cpp-pm#778 - Update from the deprecated `windows-2019` image with `vs-16-2019` toolchain to `windows-2022` with `vs-17-2022` toolchain - fixes: cpp-pm#781 - Update from deprecated `macos-12` image to `macos-15` - fixes: cpp-pm#782 - rewrite `set_matrix.py` to generate the `CMAKE_TOOLCHAIN_FILE` from the `toolchain` matrix name - update CI and remove the dependency on the [polly toolchain project](https://github.com/cpp-pm/polly) - remove `analyze` and `sanitize` jobs, no-one looks at them and they are a hassle to implement, PR welcome, but for now removed - remove the `build.sh` and `build.cmd` default build scripts and replace them with a `build.py` python script - remove `build_hunter_tests.sh` build script and use `build.py` there as well - remove unused `jenkins.py` file - update custom `matrix.json` and `build.sh` for the following projects: - `filament` - `pthreads-win32` - `occt` - `freetype-gl` - `Boost` * update msgpack to v4.1.3 (cpp-pm#644) * update msgpack to v4.1.3 * update docu and example after review * Update docs/packages/pkg/msgpack.rst --------- Co-authored-by: NeroBurner <pyro4hell@gmail.com> * Add note about updating copyright notes to the manual (cpp-pm#601) one should update the copyright notices when a new package is created. Point this out to the user in the guide to create a new package. Co-authored-by: NeroBurner <pyro4hell@gmail.com> * libsodium: update version to 1.0.18-p0 (cpp-pm#672) Update the default version of libsodium to 1.0.18-p0 Fixes: cpp-pm#634 --------- Co-authored-by: NeroBurner <pyro4hell@gmail.com> * Update civetweb to v1.13-p0 (cpp-pm#304) * GIT_SUBMODULE fix: File not found .git/HEAD error (cpp-pm#168) * Fix File not found .git/HEAD error * Verify if git path output is absolute * docs: update URL to tar file for boost-components documentation (cpp-pm#561) * fix link-all.py with high CPU count and little files (cpp-pm#785) Code changes by https://github.com/aggiehorns When we have more CPUs than files to link the `files_per_job` variable got `0`. And then the begin and end indexes got wrong and multiple jobs tried to link the same file. Fixes: cpp-pm#664 * set_matrix: simplify regex, fix typos (cpp-pm#786) Fix some typos in output, variable-names and comments. The usage of `[\d]` in regex is explicit, but also uneccessarily complex. Simplify a bit. * set_matrix: format * CI: run check-urls: rework script to run on changed files and let it run in CI (cpp-pm#787) Rework the `check-urls.py` script to make it work with the changed files file from GitHub Actions. Add a new job to run `check-urls.py` and check projects with changed `hunter.cmake` files and their URLs. Fixes: cpp-pm#598 * Boost: remove USE_CONFIG_FROM_BOOST hunter option, remove old (cpp-pm#784) Remove the option `USE_BOOST_CONFIG`. It is now essentially always enabled. Remove all Boost versions older than 1.72, as the old versions don't supply a `BoostConfig.cmake` file yet. Remove the code and templates to create a `BoostConfig.cmake` with Hunter. Update the examples and remove the `-useBoostConfig` example folders. Fixes: cpp-pm#723 * OpenBLAS: silence warnings (cpp-pm#792) The OpenBLAS build has uncountable many warnings. Disable them as we don't need them. Only errors are really interesting. * docs: fix warning about matrix.json not valid for code highlighting (cpp-pm#793) Fixing the following warning: ```sh Warning, treated as error: hunter/docs/creating-new/test/ci.rst:84:Could not lex literal_block as "JSON". Highlighting skipped. ``` The referenced codeblock is the following: ```rst .. literalinclude:: ../../../.github/workflows/ci/matrix.json :language: JSON ``` The `matrix.json` file contains `//` comments, which are not valid json tripping the highlighting. Removing the `:language: JSON` part fixes this warning Fixes: cpp-pm#790 * docs: msgpack: fix spelling warnings/errors (cpp-pm#795) The documentation for `msgpack` has two spelling errors: ``` writing output... [ 63%] packages/pkg/msgpack packages/pkg/msgpack.rst:21:msgpackc:["mudpack"] packages/pkg/msgpack.rst:21:cxx:["xxx", "xx", "cox", "c xx", "OSX"] ... Spelling checker messages written to /home/runner/work/hunter/hunter/docs/_spelling/output.txt Warning, treated as error: Found 2 misspelled words ``` Update the documentation properly escaping the target names and in the process making the spellchecker ignoring the "codeblocks". Fixes: cpp-pm#794 * docs: quick-start: fix link to HUNTER_ROOT env variable (cpp-pm#800) `HUNTER_ROOT` link goes to old/not available wiki. Update the link to use an internal reference the the `HUNTER_ROOT` environment variable section in the documentation. Fixes: cpp-pm#682 * nlohmann json update to 3.12.0 (cpp-pm#803) * update nlohmann_json 3.12.0 * docs: add build_docs.sh script, installing debian packages through apt Replace the `jenskins.sh` script using pip with `make_docs.sh` script expected to run on Ubuntu. For apt install set non-interactive env variable. Update CI to run on `ubuntu-24.04` instead of `ubuntu-latest`. Remove the files `jenkins.sh` and `requirements.txt` * docs: enable myst-parser for markdown support, ignore old-wiki md files * docs: conf.py: explicitly set language "en" * docs: update version to v0.26 to match release tags * docs: conf.py: remove deprecated get_html_theme_path call (cpp-pm#810) For Sphinx `v8.1.3` compatibility remove the `get_html_theme_path` call. Fixes: cpp-pm#807 * docs: port foo.rst to foo.md (cpp-pm#808) I like to write documentation in MarkDown better than in `*.rst`. First step is to have new package documentation in MarkDown. So port the reference `foo.rst` to `foo.md`. * docs: fix deprecation warning replacing spelling with spelling:word-list (cpp-pm#811) When building the docs on Ubuntu 24.04 many deprecation warnings are shown ``` direct use of the spelling directive is deprecated, replace ".. spelling::" with ".. spelling:word-list::" ``` Fixes: cpp-pm#806 * docs: foo.md: remove double space from enumeration * libdeflate: add new package with v1.24-p0 (cpp-pm#812) Need to use a patched version of `libdeflate` as the created CMake config contains absolute paths breaking the caching. Once ebiggers/libdeflate#434 is merged and released we can use pure upstream. Fixes: cpp-pm#804 * libdeflate: update copyright * TIFF: update to v4.7.0-p0 (cpp-pm#813) Unpinn OpenCV 4.10 because TIFF 4.7.0 fixes build issues on Linux. Fixes: cpp-pm#765 Remove old `FindTIFF.cmake` module. Not needed anymore (and probably wasn't needed for a long time). Update to use https://github.com/cpp-pm/libtiff/releases/tag/v4.7.0-p0 hunterized dependencies: - `zlib`: default ON, only dependency available in previous 4.0.2 hunterized version) - `libdeflate`: default: ON, available in Hunter starting with `v0.26.3` cpp-pm#804 - `jpeg`: default OFF, newly hunterized - `lzma`: default OFF, newly hunterized - `zstd`: default OFF, newly hunterized Not hunterized dependencies: - `jbig=OFF` - `lerc=OFF` - `webp=OFF` Update the TIFF example to use use `stdio.h` `remove()` instead of `unlink()`. Fixes: cpp-pm#789 * TIFF: fix mention about zstd being supported * Add new package OpenJPH 0.23.0 (cpp-pm#814) Add upstream version of package `OpenJPH`. The `TIFF` dependency is optional and I had some issues when adding `hunter_add_package(TIFF)` with missing includes. So adding with `TIFF` disabled. Fixes: cpp-pm#805 * Imath: update to 3.2.1 (cpp-pm#817) Update `Imath` package for OpenEXR 3.4.0 compatibility as described in the `OpenEXR` install documentation (3.4.0+ should use Imath 3.2). Fixes: cpp-pm#816 * OpenSSL: strip libunwind flags for Clang-based Android NDK (cpp-pm#815) Clang-based Android NDKs bundle unwind support and do not require explicit linking. Passing malformed flags like `-l-l:libunwind.a` caused build failures with `ld.lld` on NDK r23+. This change removes those flags from configure options to ensure successful builds. * OpenEXR: add version 3.4.0-p0 (cpp-pm#818) Update `OpenEXR` with Hunterized `v3.4.0-p0` version: https://github.com/cpp-pm/openexr/releases/tag/v3.4.0-p0 Dependencies: - add `Imath` as dependency (unchanged) - add `libdeflate` as depencency, introduced in `v3.2.0` - removed `zlib` dependency since `v3.2.0`, replaced by `libdeflate` - add `OpenJPH` as dependency, since `v3.4.0` Fixes: cpp-pm#788 * CI: install python requests module to fix upload cache script (cpp-pm#820) On the `master` branch the pipelines fail at the `upload-cache-to-github.py` script as it is missing the `requests` python module. Install it before the build to have a working cache upload again. Fixes: cpp-pm#819 * PNG: update to v1.6.36-p4 with CMake 3.11 requirement (cpp-pm#823) Update to release https://github.com/cpp-pm/libpng/releases/tag/v1.6.36-p4 * Jpeg: update to v9f-p1 with CMake 3.10 (cpp-pm#822) Integrate cmakeified version: https://github.com/cpp-pm/jpeg/releases/tag/v9f-p1 Fixes: cpp-pm#821 * CI: install gitpython python module for cache upload script (cpp-pm#829) Fixes: cpp-pm#827 * giflib: update to v5.2.2-p0 (cpp-pm#828) use https://github.com/cpp-pm/giflib/releases/tag/v5.2.2-p0 with updated minimum CMake 3.10 version. Fixes: cpp-pm#824 * minizip: update to 1.0.1-p4. (cpp-pm#833) Update to release https://github.com/cpp-pm/minizip/releases/tag/v1.0.1-p4 Fixes: cpp-pm/minizip#1 * ZLIB: update to v1.3.1-p0. (cpp-pm#831) Update to release https://github.com/cpp-pm/zlib/releases/tag/v1.3.1-p0 Fixes: cpp-pm#735 * OpenCV+Extra+ippicv: add version v4.12.0-p0 (cpp-pm#832) Update OpenCV, OpenCV-Extra and ippicv to `v4.12.0-p0`. Those packages belong together and should be updated at once. - OpenCV: update to 4.12.0-p0 - OpenCV-Extra: update to 4.12.0 release - ippicv: update to 20250130 for OpenCV 4.12.0 - keep APPLE version to ippicv 20240201 from OpenCV 4.10.0 - add `WITH_VTK=OFF` - Unpin OpenCV for all as Linux build is now fixed with fixed TIFF/OpenEXR Fixes: cpp-pm#801 * opencv: update to v4.12.0-p1 with CUDA 12.9 fix (cpp-pm#835) Update to `4.12.0-p1` with CUDA 12.9 fix cherry-picked: opencv/opencv@e2d87de Link to Release: https://github.com/cpp-pm/opencv/releases/tag/v4.12.0-p1 Fixes: cpp-pm#834 * Add Boost version 1.84.0, 1.85.0, and 1.86.0 * Support for Boost 1.86 with added component configuration * Update download url for Boost Update download url from jfrog to boost-archive. --------- Co-authored-by: Aliaksandr Anishchuk <oav.other@gmail.com> Co-authored-by: Harry Mallon <hjmallon@gmail.com> Co-authored-by: Dominik Nussbaumer <4436784+tnixeu@users.noreply.github.com> Co-authored-by: André Falk <andre.falk@tutanota.com> Co-authored-by: simbahebinbo <lansheng228@163.com> Co-authored-by: NeroBurner <pyro4hell@gmail.com> Co-authored-by: Ronny Nowak <dargun@gmx.net> Co-authored-by: craffael <raffael@casagrande.ch> Co-authored-by: Graham Reeds <graham.reeds@gmail.com> Co-authored-by: Rahul Sheth <rahul@snap.com> Co-authored-by: Joerg-Christian Boehme <joerg@chaosdorf.de> Co-authored-by: Timothy Pak <httpak@connect.ust.hk> Co-authored-by: Andrei Svatko <andrei.svatko@regula.by> Co-authored-by: Olli Wang <olliwang@ollix.com> Co-authored-by: Ola Engwall <ola.engwall@hypergene.se>
Update all used GitHub runner images (Linux, Windows and Mac) and update the
set_matrix.pyscript to work without thepollytoolchains.gccversion from 7, 8, 9 to 12, 13, 14.windows-2019image withvs-16-2019toolchain towindows-2022withvs-17-2022toolchainmacos-12image tomacos-15set_matrix.pyto generate theCMAKE_TOOLCHAIN_FILEfrom thetoolchainmatrix nameanalyzeandsanitizejobs, no-one looks at them and they are a hassle to implement, PR welcome, but for now removedbuild.shandbuild.cmddefault build scripts and replace them with abuild.pypython scriptbuild_hunter_tests.shbuild script and usebuild.pythere as welljenkins.pyfilematrix.jsonandbuild.shfor the following projects:filamentpthreads-win32occtfreetype-glBoost