Skip to content

partitionccl: various improvements to TestRepartitioning#41240

Closed
ajwerner wants to merge 2 commits intocockroachdb:masterfrom
ajwerner:ajwerner/partitioning-test-with-upgrades-and-leaseholder-preferences
Closed

partitionccl: various improvements to TestRepartitioning#41240
ajwerner wants to merge 2 commits intocockroachdb:masterfrom
ajwerner:ajwerner/partitioning-test-with-upgrades-and-leaseholder-preferences

Conversation

@ajwerner
Copy link
Copy Markdown
Contributor

@ajwerner ajwerner commented Oct 1, 2019

This PR has two commits:

  1. Expand TestRepartitioning to additionally use leaseholders and operate on more nodes
    • This change which dramatically increases the run time of the test overall then adds logic to choose a random subset of the tests to run to shorten the test overall.
  2. Add logic which starts the cluster at 19.1 (preemptive snapshots) and then upgrades the cluster during its execution

Release Justification: just testing.

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@ajwerner ajwerner force-pushed the ajwerner/partitioning-test-with-upgrades-and-leaseholder-preferences branch from b6655d0 to 29f6091 Compare October 1, 2019 19:54
Andrew Werner added 2 commits October 1, 2019 20:07
Prior to this commit the partitioning tests worked by creating a 3 node cluster
and then expressed constraints over the three nodes. It then validates that
the cluster conforms to the constraints by querying data and examining the
trace to determine which node held the data.

This is problematic for one because it is succeptible to cockroachdb#40333. In rare
cases we'll down-replicate to the wrong single node (e.g. if the right one
is not live) and we won't ever fix it.

It also doesn't exercise leaseholder preferences.

This PR adds functionality to configure clusters with larger numbers of nodes
where each expectation in the config can now refer to a leaseholder_preference
rather than a constraint and we'll allocate the additional nodes to 3
datacenters.

This larger test creates dramatically more data movement and has been useful
when testing cockroachdb#40892.

The PR also adds a flag to control how many of these subtests to run.

Release justification: Only touches testing and is useful for testing a
release blocker.

Release note: None
This PR extends the partition test to sometimes start as a 19.1 cluster and
upgrade to 19.2. This test will age poorly valuable during the transition.
This form of testing will only remain active on the 19.1 branch and should be
removed from 20.1 when that happens. It's sort of too little, too late but it's
now typed and I hope will remain valuable for the remainder of this release.

Release justification: testing only.

Release note: None
@ajwerner ajwerner force-pushed the ajwerner/partitioning-test-with-upgrades-and-leaseholder-preferences branch from 29f6091 to 6d77677 Compare October 1, 2019 20:10
@ajwerner
Copy link
Copy Markdown
Contributor Author

Closing in favor of #42758 which just includes the first commit. The second commit is no longer relevant. Perhaps we could backport the first commit to 19.2 and then add the second commit but it's not obviously worth it presently.

@ajwerner ajwerner closed this Nov 25, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants