Conversation
|
It's hard to tell where you're going with the first commit. Will The second commit looks good to me. |
|
Oh I just split the first commit out because I thought there might be some comments on the style. We haven't used this style of shell in this project yet (I think), putting everything in functions and calling main from the bottom of the script. I learned this style from the old |
|
Ah! If it's purely stylistic then concept ACK. |
|
Cool I'll flesh this PR out then. Thanks man. |
574bbb5 to
f80b57e
Compare
|
The lint fail is due to the duplicate deps test, and locally I cannot reproduce? |
|
And the ASAN stuff is ugly. |
|
|
Add a `main` function to script in preparation for adding more functions. Move the current script into a function and call it from main. Refactor only, no logic changes.
Use `bash` in all the CI `test.sh` scripts, for no particular reason other than to be uniform with other scripts in the repository.
Instead of running the linter in each individual CI script we can just run it once from the top level script and exit. This has the benefit of being quicker with no loss of linting coverage. Note this does remove sanity builds from the linting job, this does not reduce overall coverage of the whole CI workflow.
As we did for the linting stuff, move the docs stuff out of each individual CI script and into the top level script. Reducing code duplication with no reduction in test coverage.
These MSRV pinning blocks are stale, remove them.
The CI script is overly complex to reason about because of how it interacts with the github actions. In an effort to make it more obvious exactly what is going on and what is failing when CI is red split the linting and docs stuff out into separate jobs. In the CI script once these are run exit. This has the added advantage of making these jobs fast to run and fast to fail, hopefully making the dev feedback loop quicker.
We do not run the formatter in CI anymore, remove the stale logic from all test scripts.
3a43af8 to
369f6bf
Compare
We can export this from the top level script, thereby reducing code duplication. If devs want this when running individual test scripts they can set it in the script invocation.
As we did for linting and docs; move the bench stuff to the top level script and run it in a separate job.
These variables are no longer used, remove them.
The `AS_DEPENDENCY` test does not need to be run for the recent and minimal lock files, just run it once in a separate job.
This test does not use the top level lock file, no need to run it multiple times, just run it once from the top level script and do so from a separate CI job.
The ASAN tests do not use `cargo --locked` so can be run from the top level CI script. This patch _increases_ coverage because the job that enables DO_ASAN was only being run in `hashes/` - now we run it in `fuzz/` as well. Note please I'm not very knowledgeable about ASAN stuff or what exactly is going on in these commands but I remove `--lib` (and added `+nightly` to be explicit) in order to get the it working in `fuzz/`.
The WASM test does not use `cargo --locked` so we can run it from the top level CI script.
The individual CI scripts are expected to be run after copying the recent/minimal lock files into place. As such all `cargo` commands should be run with `--locked` otherwise the lockfile will be updated. To save us making mistakes in the future introduce a variable `CARGO` and always use it instead of `cargo`. This patch changes the current behaviour because in some places the sanity checks are being run without using `--locked` - this is a bug.
The current CI github action uses the name 'Continuous integration' which is really long and noisy - use CI with no loss of clarity.
TIL, one can use curly braces and only redirect a single time - thanks `shellcheck`. Refactor only, no logic changes.
`shellcheck` complains about unused `FEATURES` variable, leave it there commented out to make explicit the non-uniformity of this script compared to the others. I'd say there is some work left to do to clean up the feature matrix stuff.
Add a separate github action to run the duplicate dependencies job.
|
A bit of help here please.
|
|
OMG, I've just started doing pretty much this exact thing. I'll defer to you. I chose a bit different approach:
The advantages are: even more parallel builds - each crate and task gets its own. I've had plans to add caching as well, even more de-duplication of the script - it's practically completely de-duplicated since all logic is unified. Only exceptional cases are handled in separate script files. |
|
Sweet, can you push up what ever you have done to a branch somewhere so I can grab it and put it into this PR? |
|
See #2343 (there's clearly something broken since Prepare job didn't run, who would have guessed? :D) |
|
Closing in favour of #2353 |
01a66a7 CI: Check for required commands (Tobin C. Harding) 5c15ed5 CI: Epic overhaul (Martin Habovstiak) 242aa67 Use env bash instead of /bin/bash (Tobin C. Harding) 422d301 Use bash to run shell scripts (Tobin C. Harding) Pull request description: The combination of some work by myself [0] and Kix [1]. Draft so I can use github's infrastructure to test it all out. Includes some patches at the front to fix real issues that the new test infrastructure found - WIN. [0] #2328 [1] #2343 Coincidentally this closes 1124 Resolve: #1124 ACKs for top commit: apoelstra: ACK 01a66a7 Tree-SHA512: 026a0948a181102246702eadc3ff245c319c456b03ada9ca269141d006146f30fd8eb50377062735a06c3e369f7edac2e334587120338a3747810d999177d930
This is now an epic re-work of the CI scripts. The aims are:
I've made the changes really small and said why in each commit log - I did this, more than usual, so as to help catch either bugs in the change set or bugs that currently exist in the CI scripts but we didn't notice them.