Skip to content

CI: Fail the rebuild command if buildcache push failed#40045

Merged
haampie merged 11 commits intospack:developfrom
kwryankrattiger:ci_failed_push_failed_job
Mar 28, 2024
Merged

CI: Fail the rebuild command if buildcache push failed#40045
haampie merged 11 commits intospack:developfrom
kwryankrattiger:ci_failed_push_failed_job

Conversation

@kwryankrattiger
Copy link
Copy Markdown
Contributor

Make jobs fail when binaries fail to push.

@spackbot-app spackbot-app bot added commands core PR affects Spack core functionality labels Sep 15, 2023
@kwryankrattiger kwryankrattiger force-pushed the ci_failed_push_failed_job branch 2 times, most recently from 5786437 to 335c5d1 Compare September 26, 2023 22:11
@spackbot-app spackbot-app bot added the tests General test capability(ies) label Sep 26, 2023
@kwryankrattiger
Copy link
Copy Markdown
Contributor Author

@spackbot fix style

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Oct 2, 2023

Let me see if I can fix that for you!

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Oct 2, 2023

I was able to run spack style --fix for you!

spack style --fix
==> Running style checks on spack
  selected: isort, black, flake8, mypy
==> Modified files
  lib/spack/spack/binary_distribution.py
  lib/spack/spack/cmd/buildcache.py
  lib/spack/spack/cmd/ci.py
  lib/spack/spack/test/bindist.py
==> Running isort checks
  isort checks were clean
==> Running black checks
All done! ✨ 🍰 ✨
4 files left unchanged.
  black checks were clean
==> Running flake8 checks
lib/spack/spack/test/bindist.py:473: [E722] do not use bare 'except'
lib/spack/spack/test/bindist.py:482: [E722] do not use bare 'except'
lib/spack/spack/test/bindist.py:501: [E722] do not use bare 'except'
lib/spack/spack/test/bindist.py:511: [E722] do not use bare 'except'
  flake8 found errors
==> Running mypy checks
Success: no issues found in 586 source files
  mypy checks were clean
Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I wasn't able to make any further changes, but please see the message above for remaining issues you can fix locally!

@kwryankrattiger kwryankrattiger force-pushed the ci_failed_push_failed_job branch 2 times, most recently from 5b01243 to 48dd932 Compare October 3, 2023 22:48
@kwryankrattiger
Copy link
Copy Markdown
Contributor Author

Push failure results in pipeline failure berkeley-db
Reindex fails when unable to list keys reindex

Successful push and usage of packages diffutils

@haampie
Copy link
Copy Markdown
Member

haampie commented Oct 5, 2023

There are a lot of basic issues with this PR. Can you improve it?

@kwryankrattiger
Copy link
Copy Markdown
Contributor Author

@haampie are you okay with these changes? I would like to get this in sooner than later.

@kwryankrattiger kwryankrattiger force-pushed the ci_failed_push_failed_job branch from 978f3c4 to 4e097d7 Compare October 18, 2023 16:06
@kwryankrattiger kwryankrattiger force-pushed the ci_failed_push_failed_job branch 3 times, most recently from da00fb8 to fda4eba Compare January 18, 2024 23:07
@kwryankrattiger
Copy link
Copy Markdown
Contributor Author

@haampie I took another pass on this with fresh eyes. I think because all of these APIs are internal the required flag to maintain the old, non-signaling failure behavior was not really worth it.

I added new exceptions, GenerateIndexError and PushToBuildCacheError, and simply raise the error and the message to the next level to be handled. This seems to work well and l think it is a much cleaner solution.

@kwryankrattiger kwryankrattiger force-pushed the ci_failed_push_failed_job branch from 8761e87 to 1a2b090 Compare February 8, 2024 03:52
@kwryankrattiger
Copy link
Copy Markdown
Contributor Author

The two code paths mentioned by @haampie should be tested now. Letting the coverage run to verify and will clean up the commit history after.

@kwryankrattiger kwryankrattiger requested a review from haampie March 21, 2024 15:50
@haampie
Copy link
Copy Markdown
Member

haampie commented Mar 22, 2024

lgtm, but please remove the unused bits

@kwryankrattiger kwryankrattiger force-pushed the ci_failed_push_failed_job branch from 660b331 to 1fd0c6b Compare March 22, 2024 14:17
@kwryankrattiger kwryankrattiger requested a review from haampie March 22, 2024 14:19
@kwryankrattiger kwryankrattiger force-pushed the ci_failed_push_failed_job branch 2 times, most recently from 9fb6d3e to 54b6a1e Compare March 25, 2024 20:31
@kwryankrattiger kwryankrattiger force-pushed the ci_failed_push_failed_job branch from 54b6a1e to 2cf3c6b Compare March 25, 2024 21:05
@kwryankrattiger
Copy link
Copy Markdown
Contributor Author

removed debug, rebased to fix merge conflict

@haampie haampie merged commit ae2d0ff into spack:develop Mar 28, 2024
G-Ragghianti pushed a commit to G-Ragghianti/spack that referenced this pull request Apr 2, 2024
tldahlgren pushed a commit to tldahlgren/spack that referenced this pull request Apr 23, 2024
teaguesterling pushed a commit to teaguesterling/spack that referenced this pull request Jun 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

binary-packages commands core PR affects Spack core functionality fetching tests General test capability(ies) utilities

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants