This change is 

Did you manage to work around the flakiness described in #19269?

Reviewed 4 of 4 files at r1.
Review status: all files reviewed at latest revision, 4 unresolved discussions, some commit checks pending.

pkg/acceptance/version_upgrade_test.go, line 158 at r1 (raw file):

		binaryVersionUpgrade("v1.1.5"),
		binaryVersionUpgrade(sourceVersion),
		clusterVersionUpgrade("1.1-6"),

Any particular reason for choosing this waypoint?

pkg/acceptance/version_upgrade_test.go, line 165 at r1 (raw file):

		cfg.Nodes = cfg.Nodes[:3]
	}
	startingBinVerion := steps[0].(binaryVersionUpgrade)

"startingBinVersion"

pkg/acceptance/version_upgrade_test.go, line 182 at r1 (raw file):

			// If the subtest already failed, don't run any future steps.
			break
		}

This seems like an abuse of subtests, given that go test -run TestVersionUpgrade/binary=v1.1.5 is going to do something nonsensical. Perhaps just t.Logf(step.name())?

pkg/acceptance/localcluster/cluster.go, line 710 at r1 (raw file):

		urlBytes, err := ioutil.ReadFile(n.listeningURLFile())
		if err != nil {
			log.Info(ctx, err)

💯

Comments from Reviewable

Reviewed 4 of 4 files at r1.
Review status: all files reviewed at latest revision, 7 unresolved discussions, some commit checks failed.

pkg/acceptance/version_upgrade_test.go, line 145 at r1 (raw file):

	// TODO(nvanbenschoten): add upgrade qualification step.
	time.Sleep(1 * time.Second)

So this sleep alone was enough to make it reliably trigger the checksum problem?

pkg/acceptance/version_upgrade_test.go, line 159 at r1 (raw file):

		binaryVersionUpgrade(sourceVersion),
		clusterVersionUpgrade("1.1-6"),
		clusterVersionUpgrade(clusterversion.BinaryServerVersion.String()),

Each of these takes at least a second (since there's a sleep above), so it's going to get annoying to always go through the full history. I don't think we need all the 1.1 variants; one version per branch should be enough.

pkg/acceptance/version_upgrade_test.go, line 165 at r1 (raw file):

		cfg.Nodes = cfg.Nodes[:3]
	}
	startingBinVerion := steps[0].(binaryVersionUpgrade)

Can't we start with 1.0 and then upgrade to 1.1 (as long as we don't touch the cluster version before the upgrade)?

Comments from Reviewable

Review status: 2 of 4 files reviewed at latest revision, 7 unresolved discussions, all commit checks successful.

pkg/acceptance/version_upgrade_test.go, line 145 at r1 (raw file):

Yes, I never saw it pass with this sleep.

pkg/acceptance/version_upgrade_test.go, line 158 at r1 (raw file):

No, I just figured it would be nice to test somewhere in the middle and these clusterVersionUpgrade steps are quick.

pkg/acceptance/version_upgrade_test.go, line 159 at r1 (raw file):

Done.

pkg/acceptance/version_upgrade_test.go, line 165 at r1 (raw file):

Done.

pkg/acceptance/version_upgrade_test.go, line 165 at r1 (raw file):

Done.

pkg/acceptance/version_upgrade_test.go, line 182 at r1 (raw file):

Good point. Done.

Comments from Reviewable