Update release process documentation and scripts#5920
Merged
chrisd8088 merged 9 commits intogit-lfs:mainfrom Dec 2, 2024
Merged
Update release process documentation and scripts#5920chrisd8088 merged 9 commits intogit-lfs:mainfrom
chrisd8088 merged 9 commits intogit-lfs:mainfrom
Conversation
Before updating the documentation of our release process, we first adjust the indentation of several paragraphs to match the preceding ones.
Before updating the documentation of our release process, we first correct a minor misspelling.
In commit d61b3ea of PR git-lfs#5607 we added support in our Makefile and scripts to build and upload a version of our Linux release assets for the LoongArch64 platform, and at the same time we added an entry for this platform to the end of the list of Linux assets in our release process documentation. Before updating the documentation of our release process, we first reorder this list to keep it in alphanumeric order, so as to match how assets are listed in our public release notes.
In PR git-lfs#5438 we added support in our Makefile and scripts to build and upload a version of our Linux release assets for the RISC-V 64 platform. As the documentation of our release process includes a list of the Linux assets we build, we now add an entry for the RISC-V 64 platform so as to match the assets listed in our public release notes.
Based on the recent experience of releasing the 3.6.0 version of Git LFS, we update several steps in our documentation to more accurately reflect our current release process. First, we add a reminder that we should include in our release notes any changes from the previous Git LFS release in terms of platform requirements or the set of operating system versions we support. As well, we more fully document the dependencies of our script/finalize script, and note that a PR to update Homebrew should now be generated automatically within a few hours of our making a new release public.
Before we release a new version of Git LFS, we generally want to upgrade the version of Go used in the Docker container images generated by our git-lfs/build-dockers repository, so we add notes regarding this step to our release process documentation. Although Markdown does not require us to renumber the items in an ordered list, we do so anyway as this makes our documentation more readable when viewed as a plain text file.
Our script/hash-files script is executed by our script/upload script to generate hashes of the asset files built by our release process, so that we can then sign the resulting "hashes" file with a GPG signing key and publish the signed file in our release announcement. This script makes use of the OpenSSL::Digest Ruby gem to perform hashing with a variety of algorithms, including BLAKE2b, BLAKE2s, SHA-512, and others. On macOS, a number of these digests require the Ruby 3.x interpreter and "openssl" gem available via Homebrew rather than the Ruby 2.x versions provided by current versions of macOS; otherwise, the script returns "first num too large" errors from OpenSSL. We therefore add a simple check to our script which exits with a non-zero code and an error message if the Ruby 3.x interpreter is not in use.
The verify_assets() function in our script/upload script performs a final verification of the contents of the GPG-signed "sha25sum.asc" and "hashes.asc" files we include in the assets attached to our public release notes. To perform these checks, it runs the shasum(1) utility for all the SHA-2 hashes, the sha3sum(1) utility for the SHA-3 hashes, and the b2sum(1) utility for the BLAKE2b hashes, passing each utility the appropriate lines from the "hashes.asc" file. These lines have the format output by our script/hash-files script, namely: <hash-name> (<file-name>) = <hash-value> While this format is accepted and parsed successfully by all of the utilities on most Linux systems, the sha3sum(1) program available on macOS via Homebrew does not permit this input format with its -c option. Instead, it requires as input exactly the same format as it outputs when hashing a set of files, i.e.: <hash-value> <file-name> This disparity is due to the fact that Homebrew packages as the "sha3sum" utility a different implementation than is packaged for Debian Linux. Specifically, the Homebrew package is built from the following source project: https://codeberg.org/maandree/sha3sum https://formulae.brew.sh/formula/sha3sum On Debian, however, the sha3sum(1p) utility is a Perl script provided with the Digest::SHA3 Perl extension: https://metacpan.org/dist/Digest-SHA3/view/sha3sum https://sources.debian.org/src/libdigest-sha3-perl/ https://manpages.debian.org/unstable/libdigest-sha3-perl/sha3sum.1p.en.html This Perl script accepts a wider range of input formats than the program packaged by Homebrew, including the one created by our script/hash-files script. It is also able to deduce the SHA-3 algorithm to use based on the hash name in each line, which the sha3sum(1) utility on macOS is unable to do. As we sometimes want to run our scripts on macOS when generating a new Git LFS release, we adjust our script/upload script so it rewrites the lines from the "hashes.asc" file into the format accepted by the Homebrew version of sha3sum(1), and also so it invokes that utility separately for each hash algorithm. As this also works with the Perl script version of sha3sum(1p), our script/upload script should now succeed on both macOS and Linux in the future.
larsxschneider
approved these changes
Dec 2, 2024
| and then use that branch as the base for the PATCH release. | ||
|
|
||
| 1. If the `release-M.N` branch does not already exist, create it from | ||
| 1. Upgrade or downgrade the Go version used in the `git-lfs/build-dockers` |
Member
Author
There was a problem hiding this comment.
The most obvious situation occurs when Go releases a new minor (or major) version which drops support for older platforms, and we've moved our development branches ahead to keep pace with that.
However, we normally want our patch releases to support the same set of platform versions as all the other Git LFS releases we've made in that series, so we generally don't want to upgrade to a new minor (or major) version of Go for a patch release, and might have to downgrade Go temporarily as a result.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR updates our "Releasing Git LFS" documentation page to reflect the current state of our release process, and also adjusts two of the scripts used in that process so they function successfully on macOS systems (or provide more useful error messages if the necessary dependencies are not available).
In particular, our
script/uploadscript expects that thesha3sum(1)utility is available and that it accepts input in the format generated by ourscript/hash-filesscript, i.e.:However, the version of the
sha1sum(1)program packaged and installed by Homebrew is different from the one packaged by Debian, which is a Perl script that accepts a wider range of input formats, including the format output by ourscript/hash-filesscript. The program provided by Homebrew'ssha3sumformula requires input in the following format, and is not able to determine the appropriate algorithm to use without it being specified by an-acommand-line option.Therefore we adjust our
script/uploadscript so it is able to function with either implementation of thesha1sum(1)utility, since the Perl script also accepts the same format as the Homebrew version.This PR will be most easily reviewed on a commit-by-commit basis, with whitespace changes ignored.