Currently, the vast majority of mixed version tests use the master workload binary to invoke workloads for the entirety of the test. While this generally has worked so far due to workload seeing few changes, we occasionally run into backwards compatibility issues that break mixed version tests. For example, see the failure here where the workload data load attempts to set a cluster setting not present in all versions.

Historically, the workload package does not attempt to maintain backwards compatibility. As a result, the fix in the above failure should be to use the sync the workload binary version with the cluster version. e.g. if the cluster is running on v24.3, we should invoke the v24.3 workload binary.

Note the admission-control/elastic-workload/mixed-version which does this manually via:

The mixed version framework currently manages uploading each version of cockroach for us as we progress through the plan.

These binaries are uploaded to a deterministic path and not overwritten:

However, this only uploads versioned binaries to cockroach nodes. We should extend this functionality to workload nodes as well, which will require the mixed version framework to keep track of the workload node.

Once we do that, we should expose these binaries to users via the mixed version helper API. The mixed-version helper API already exposes h.Context().FromVersion(), as well as h.ClusterVersion() to us.

At the time of writing this, I can't think of a scenario where we wouldn't want to use the cluster version. For simplicity perhaps a h.CockroachPath that always returns the cluster version may suffice and reduce any confusion from test writers on which version to use.

Jira issue: CRDB-51025