What/Why

What are you proposing?

The current OpenSearch bundle[1] release process looks like this:

1. We follow a release train model for minor releases. At the beginning of the year we publish two dates for each train that year: A Code Freeze date and (about 1 week later) a Release Date.
2. For each release we cut a release ticket per repo that has milestones in it. Those milestones work backwards from the release date.
3. After the code freeze date we generate a build and do performance testing, integration testing, as well as update documentation, release notes and a write a blog post.
4. On the release date we build the artifacts and release them out on all platforms.

Some issues with this process:

1. Repos are waiting until the last minute to add features (with large merges happening in the last hours). This means we either have to move the freeze date and the release date, or it means we only move the freeze date and then we don't have enough time to test.
2. Some of the things that we need, like release notes, don't get generated until the last minute, and require lots of cat herding to obtain.
3. Because features are arriving late, it's very hard for the documentation team to be able to turn around updates in a week.
4. Churn on features and docs make it difficult to finalize blog content.
5. Again, because things are coming in so late, the community doesn't have enough visibility into the features/changes that are going into releases.
6. Several of the issues we've found post freeze are not integration issues, but are instead standard quality misses. This implies teams are not testing until the code freeze date.
7. Perhaps related, we've found that one week is not enough time to find and fix bugs.

Proposed change:

To address these issue, I propose we make four significant changes:

• We continue to plan release every 6 weeks for the year, but we only announce the start of the release process, not the actual release date.
• We add "entrance criteria" that must be met to join the release and "exit criteria" that must be met for the artifacts to be released.
• If a plug-in doesn't meet the entrance criteria we will take the previous version, bump its version and use that for the release. They will not be able to add features to the release if they miss the start of the window.
• Every day we generate a release candidate. When the exit criteria are met by a release candidate, we announce the release date (~2 days later) and publish artifacts on that date. If we cannot pass the exit criteria by 2 weeks after the start of the window, we will cancel the release and hold changes until the next window.

Why these changes?
I believe these changes will put the right emphasis on quality, rather than our current system which is more date driven. I looked at putting in more gates and checks to try and make sure we were ready, but I think setting expectations on what has to happen before you join the release puts in the right check and balances. I would like to get to the point where we run a tool that looks for things like release notes, branches being cut, etc, and publishes a readiness checklist. If repo are ready, we automatically pick up the latest version.

Next steps?

Please see the related PR for more details about process, particularly the exit and entrance criteria. I thought it'd be easier to review it line by line there than in an issue. It's also possible this information should move over to the build repo and it will definitely require updates to the issue template.

This issue will stay open until April 14th, and if the process is agreed upon, would be used starting with the 2.8 release of OpenSearch.

Thank you!

[1] The OpenSearch bundle contains OpenSearch core and its plugins, as well as OpenSearch Dashboards and its plugins. It does not include Data Prepper or clients. We release as a bundle because of the tight coupling between the cores and their plugins. There's a good RFC about plans to change that. You should check it out :)