Skip to content

Support content addressable buildcache#1058

Merged
scottwittenburg merged 19 commits intospack:mainfrom
scottwittenburg:support-buildcache-migration
Feb 27, 2025
Merged

Support content addressable buildcache#1058
scottwittenburg merged 19 commits intospack:mainfrom
scottwittenburg:support-buildcache-migration

Conversation

@scottwittenburg
Copy link
Copy Markdown
Collaborator

Until spack has buildcache views, pipelines must be run pointing at independent mirrors, and protected-publish is responsible for making stack-specific binaries available at the mirror root.

Add support for "publishing" both old style as well as content addressable (v2 vs v3) buildcaches from stacks to the root. By default the script assumes v2 layout, but it can be changed by adding --version <version> on the command line. The only accepted options are 2 and 3.

Add an endpoint in the container that can migrate a binary mirror from the previous version to the content-addressed layout. Migration consists of, for each spec in the mirror:

  • validate the current signature on the metadata
  • add timestamp, size, and compression type attributes, and update layout version attribute from 2 to 3
  • re-sign the updated spec with the original signing key
  • upload the metadata and copy the compressed tarball to content-addressed locations

Copy link
Copy Markdown
Collaborator

@zackgalbreath zackgalbreath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a couple questions and a couple typos, otherwise LGTM!

@scottwittenburg scottwittenburg merged commit 5426eff into spack:main Feb 27, 2025
@scottwittenburg scottwittenburg deleted the support-buildcache-migration branch February 27, 2025 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants