Skip to content

Implement calendar versioning#162

Merged
seberg merged 8 commits intorapidsai:mainfrom
jameslamb:versioning
Oct 15, 2024
Merged

Implement calendar versioning#162
seberg merged 8 commits intorapidsai:mainfrom
jameslamb:versioning

Conversation

@jameslamb
Copy link
Member

@jameslamb jameslamb commented Oct 10, 2024

Contributes to #101 and #115

Establishes calendar versioning for packages. The versions here match the outcome of the discussion at https://github.com/nv-legate/legate.core.internal/issues/1197

  • calendar versioning, exactly following the version of legate / cunumeric depended on
    • e.g. legate-boost=24.06.01 depends on legate=24.06.01 and cunumeric=24.06.01
  • pre-releases are of the form {YY}.{MM}.{patch}dev{N}, where {N} is the number of commits since the beginning of development on that version
    • e.g. after 5 PRs have been merged to main in development towards the 24.09.00 version, the corresponding pre-release would be legate-boost=24.09.00dev5

Notes for Reviewers

This is still a work in progress, just pushing what I have so far so you can see I'm working on it.

References

I borrowed some ideas from the discussion on rapidsai/build-planning#63 here, combined with trying to re-use as much of the RAPIDS tooling as possible. Especially this: https://github.com/rapidsai/gha-tools/blob/8bd8fca71b5fae38b1493c547d15e73da40b32e1/tools/rapids-generate-version

How I tested this

I tagged the latest commit on main prior to this branch as v24.06.01dev, as we might do (under my proposal in this PR) at the beginning of development that builds towards a 24.06.01 release.

git tag -a v24.06.01dev -m "start v24.06.01 development"
git push upstream v24.06.01dev

ref: https://github.com/rapidsai/legate-boost/releases/tag/v24.06.01dev

After I did that, saw builds produce packages with versions like 24.06.01dev{n}, where {n} is "number of commits since the one tagged 24.06.01dev".

legate-boost 24.06.01dev5 cuda12_py310_0_cpu
--------------------------------------------
file name   : legate-boost-24.06.01dev5-cuda12_py310_0_cpu.tar.bz2
name        : legate-boost
version     : 24.06.01dev5
build       : cuda12_py310_0_cpu
build number: 0
size        : 319 KB
license     : Apache 2.0
subdir      : linux-64
url         : file:///tmp/conda-bld-output/linux-64/legate-boost-24.06.01dev5-cuda12_py310_0_cpu.tar.bz2
md5         : d81831a8376a948c1d79f9b771850d18
timestamp   : 2024-10-11 19:39:48 UTC
...

legate-boost 24.06.01dev5 cuda12_py310_0_gpu
--------------------------------------------
file name   : legate-boost-24.06.01dev5-cuda12_py310_0_gpu.tar.bz2
name        : legate-boost
version     : 24.06.01dev5
build       : cuda12_py310_0_gpu
build number: 0
size        : 6.2 MB
license     : Apache 2.0
subdir      : linux-64
url         : file:///tmp/conda-bld-output/linux-64/legate-boost-24.06.01dev5-cuda12_py310_0_gpu.tar.bz2
md5         : c184a017013a6af7348e9e44cc9a4686
timestamp   : 2024-10-11 19:43:47 UTC

(build link)

@jameslamb jameslamb added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Oct 10, 2024
@jameslamb jameslamb self-assigned this Oct 10, 2024
@jameslamb jameslamb mentioned this pull request Oct 11, 2024
6 tasks
@jameslamb jameslamb changed the title WIP: Implement calendar versioning Implement calendar versioning Oct 11, 2024
@jameslamb jameslamb marked this pull request as ready for review October 11, 2024 22:46
Copy link
Contributor

@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments/question, but likely not relevant, looks good to me either way.

git push upstream v24.09.01
```

With that hotfix release complete, merge the fixes into `main`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the work-flow in RAPIDS? In NumPy we merge fixes to main first (unless they don't apply). Which is fine, but could add, something like "if the hotfix was not backported from main".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In RAPIDS, no developer ever targets a main branch directly. RAPIDS projects one long-lived branch per release (e.g. branch-24.10, branch-24.12).

Changes on those release branches are not merged to main until the entire branch is released.

A hotfix to the 24.10 release after work on the 24.12 release has already begun means new pull requests targeting branch-24.10. Those are then merged to main, a new v24.10.01 tag is cut (pointing at a commit on main), and hotfix packages are released.

It's rare (I think) that we apply hotfixes more than 1 major release back (e.g., releasing a cudf==24.10.02 when there is already a stable cudf==24.12.00). But if we did, then those tags could point to commit IDs that exist on branch-24.10 but not on main.

Here, I'm proposing using a different, every-PR-targets-main approach for the following reasons:

  • to avoid the pain of having long-lived release branches (until we find we need one, for hotfixes)
  • to test out workflows to serve as an example for maybe changing the RAPIDS flow in the future: Simplify RAPIDS branching model build-planning#63
  • for consistency with how cunumeric and legate operate

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great. I half thought I heard rumblings for rapids to switch to this branching model, but maybe I am hallucinating :).

1. Create a pull request updating the `VERSION` file on the `main` branch to the desired version, with no leading `v`

```shell
echo "24.09.00" > ./VERSION
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this does not include the .dev, but the .dev part is found via the tag/branch?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it's expected to be found via the tag.

This is what RAPIDS does, see https://github.com/rapidsai/cudf/tags

Screenshot 2024-10-14 at 10 28 19 AM

And it's how legate is doing it too: https://github.com/nv-legate/legate.core.internal/tags

Screenshot 2024-10-14 at 10 29 36 AM

Sorry, I'm struggling today to remember exactly why that's preferable to including the dev part in the file or in the code that calculates "number of commits since the last tag" to generate a version like 24.12.00dev5.


rapids-print-env

rapids-generate-version > ./VERSION
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this sets the rapids-version above? (I think it's all good. I was just pausing now because for projects that also depend on RAPIDS, we may have to align the RAPIDS versions we use with the rapids-cmake version in legate.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually not related to the version of RAPIDS (e.g. the version used to pull in rapids-cmake), though I understand how the command names make it seem that way 😅

rapids-generate-version comes from here: https://github.com/rapidsai/gha-tools/blob/8bd8fca71b5fae38b1493c547d15e73da40b32e1/tools/rapids-generate-version

It's just a small shell script that computes a version string from git tags, like this:

dunamai_format="{base}"
if ! rapids-is-release-build; then
    # Nightlies include the distance from the last tag as the alpha version.
    dunamai_format="{base}{stage}{distance}"
fi

# Now change the version.
dunamai_version=$(python -m dunamai from git --format ${dunamai_format})

echo -n "${dunamai_version}"

I'm proposing using that tool to avoid duplicating that logic here in legate-boost.

The rapids-* prefix on all the tools from that repo is primarily intended to avoid conflicts with other tools (generate-version is a pretty generic name, for example)... it doesn't by itself mean that a particular tool does something RAPIDS-specific.

@seberg seberg merged commit f2a98bf into rapidsai:main Oct 15, 2024
@seberg
Copy link
Contributor

seberg commented Oct 15, 2024

Thanks James!

@jameslamb jameslamb deleted the versioning branch October 15, 2024 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improves an existing functionality non-breaking Introduces a non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants