Skip to content

[WIP, DNM] upgrade,*: drop payload and progress columns for system.jobs#98989

Closed
adityamaru wants to merge 2 commits intocockroachdb:masterfrom
adityamaru:drop-column-info
Closed

[WIP, DNM] upgrade,*: drop payload and progress columns for system.jobs#98989
adityamaru wants to merge 2 commits intocockroachdb:masterfrom
adityamaru:drop-column-info

Conversation

@adityamaru
Copy link
Copy Markdown
Contributor

WIP

Release note: None

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

…naryVersionOverride

Previously, tests that set BinaryVersionOverride would bootstrap
their test clusters at this overridden value. As part of this bootstrap
we would initialize the cluster with data as it appears on `binaryVersion`
i.e. system tables would be created using the schemas defined on the current `binaryVersion`.
Most tests that set BinaryVersionOverride would then manually call into `upgrade.Upgrade`
to move the cluster version forward. Since we have already bootstrapped the initial
data at `binaryVersion` these upgrades would not *really* run. We would just be testing
their idempotency unless the test did some gymnastics to "undo" the bootstrap and
then test the actual migration. So effectively, the `BinaryVersionOverride` made
the server think it was running that version but none of the associated upgrade
logic to reach that version was exercised.

In 23.1 we baked in the initial data that is bootstrapped on `binaryMinSupportedVersion`.
This development allows us to bootstrap the test cluster to `binaryMinSupportedVersion`
and *actually* step through the upgrades until `BinaryVersionOverride` before running the
test.

The new behaviour of setting this override is described below:

This value, when set, influences test cluster/server creation in two
different ways:

	 Case 1:
	 ------
	 If the test has not overridden the
	 `cluster.Settings.Version.BinaryMinSupportedVersion`, then the cluster will
	 be bootstrapped at `binaryMinSupportedVersion`  (if this server is the one
	 bootstrapping the cluster). After all the servers in the test cluster have
	 been started, `SET CLUSTER SETTING version = BinaryVersionOverride` will be
	 run to step through the upgrades until the specified override.

	 Case 2:
	 ------
	 If the test has overridden the
	 `cluster.Settings.Version.BinaryMinSupportedVersion` then it is not safe
	 for us to bootstrap at `binaryMinSupportedVersion` as it might be less than
	 the overridden minimum supported version. Furthermore, we do not have the
	 initial cluster data (system tables etc.) to bootstrap at the overridden
	 minimum supported version. In this case we bootstrap at
	 `BinaryVersionOverride` and populate the cluster with initial data
	 corresponding to the `binaryVersion`. In other words no upgrades are
	 *really* run and the server only thinks that it is running at
	 `BinaryVersionOverride`. Tests that fall in this category should be audited
	 for correctness.
	The version that we bootstrap at is also used when advertising this
	server's binary version when sending out join requests.

Informs: cockroachdb#97762

Release note: None
@adityamaru adityamaru closed this Apr 3, 2023
@adityamaru adityamaru deleted the drop-column-info branch April 3, 2023 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants