Add 'bootstrap tools' (bootstrap-cache-key) via PANTS_BOOTSTRAP_TOOLS=1#128
Merged
benjyw merged 2 commits intopantsbuild:gh-pagesfrom Oct 5, 2022
Merged
Conversation
This functionality that can introspect the bootstrapping process, similar to `PEX_TOOLS=1 ./some.pex ...` being able to introspect a PEX file. The motivating tool, added here, is `PANTS_BOOTSTRAP_TOOLS=1 ./pants bootstrap-cache-key` that prints a somewhat opaque string designed to be used as a key for CI caching systems. This is designed to help with pantsbuild/actions#5.
benjyw
approved these changes
Oct 5, 2022
Contributor
benjyw
left a comment
There was a problem hiding this comment.
This is super-cool! I tried it out and it works great.
In the future I can imagine wanting to also emit a more parseable version, but for now this is 👨🍳
Contributor
Author
|
Thanks for merging. Based on your comment about adding new subcommands in future, I was prompted to think about versioning for scripts, and thus opened #129. |
Contributor
Author
|
Ah, and I discovered a small bug: #130 |
benjyw
pushed a commit
that referenced
this pull request
Oct 7, 2022
) This fixes a (small) bug in #128: the virtualenv may have a symlink to the 'actual' path of the python executable, and the python_path value used may've itself been a symlink or a shim. This uses sys.executable as an estimate of this path, which will hopefully be good enough for the circumstances where bootstrap-cache-key is useful.
benjyw
pushed a commit
that referenced
this pull request
Oct 7, 2022
This expands on #128 to add another bootstrap tool subcommand: `bootstrap-version`. This is designed to just be a simple number that's bumped whenever there's a feature scripts might need to query for. The version number is also automatically checked against the (numeric) value of the `PANTS_BOOTSTRAP_TOOLS` key. For instance, suppose the bootstrap tools already existed (at version 1), and then `bootstrap-cache-key` command was added, the version number would be bumped to 2, and then a consumer (such as pantsbuild/actions#6) could run something like: ```shell PANTS_BOOTSTRAP_CACHE_KEY=$(PANTS_BOOTSTRAP_TOOLS=2 ./pants bootstrap-cache-key) ```
benjyw
pushed a commit
to pantsbuild/actions
that referenced
this pull request
Oct 10, 2022
The setup cache includes symlinks to things outside the cache, such as the (system) Python executable. Thus, the cache should only be reused on machines where those parts of the system exactly match up. This switches from trying to compute an appropriate cache key externally to letting the pants bootstrap script do it itself, via the new `PANTS_BOOTSTRAP_TOOLS=1 ./pants bootstrap-cache-key` (pantsbuild/setup#128). This is a breaking change, in that it requires updating the checked in pants scripts to support the new tool. Fixes #5
cognifloyd
added a commit
to StackStorm/st2
that referenced
this pull request
Oct 22, 2022
Github updated the default minor version of python 3.9.x The cache key did not include the minor version which broke things for us. Luckily, this was already fixed in pantsbuild/actions#6. The fix requires an update to the `./pants` entrypoint/bootstrap script which was updated in pantsbuild/setup#128. updated `./pants` with: curl -L -O https://static.pantsbuild.org/setup/pants && chmod +x ./pants
cognifloyd
added a commit
to StackStorm/st2
that referenced
this pull request
Oct 24, 2022
update GHA pants-init action to fix build Github updated the default minor version of python 3.9.x The cache key did not include the minor version which broke things for us. Luckily, this was already fixed in pantsbuild/actions#6. The fix requires an update to the `./pants` entrypoint/bootstrap script which was updated in pantsbuild/setup#128. updated `./pants` with: curl -L -O https://static.pantsbuild.org/setup/pants && chmod +x ./pants
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds functionality that can introspect the bootstrapping process, similar to
PEX_TOOLS=1 ./some.pex ...being able to introspect a PEX file.The motivating tool, added here, is
PANTS_BOOTSTRAP_TOOLS=1 ./pants bootstrap-cache-keythat prints a somewhat opaque string designed to be used as a key for CI caching systems. This is designed to help with pantsbuild/actions#5, and is prompted by the discussion in pantsbuild/actions#6 (comment).For example, on my system:
(This PR is a quick sketch, please lemme know if it doesn't make sense!)