Currently, roachtests don't have an established mechanism for specifying a set of cloud providers which are compatible with a given test. Theoretically, a roachtest should be cloud-agnostic since it doesn't directly interact with cloud APIs, a task that's delegated to roachprod. In practice, several roachtests may in fact be incompatible with a set of cloud providers. E.g.,

• schemachange/mixed-versions-compat uses gsutil to copy corpus data from a GCS bucket [1]
• restore variant uses AWS-specific zones [2]
• variants of clearrange and yscb/A use ZFS, available only in GCE [3], [4], [5]

Note, while gsutil in the first example may seem like a superficial incompatibility, in reality large-scale backup/restore tests may induce large egress if data is pulled from a cloud provider, different from where the test is scheduled to execute. Hence, we need to ensure, either data is sufficiently replicated (i.e., local to the test's cloud provider), or the test is specified to be incompatible with the cloud providers which lack the required test data (fixtures).
In practice, there may be additional, albeit rare reasons for incompatibility; e.g., quota, price, specific machine type, etc.

Consequently, there must be an established mechanism, both for specifying when a test is incompatible with a cloud provider, and for skipping the test from executing against all incompatible cloud providers. Currently, ClusterSpec.Cloud denotes the cloud provider that's been provided via roachtest run --cloud, not a compatible cloud provider, specified by the test author. That is, the framework makes an implicit (and wrong) assumption that every test should be executable against ClusterSpec.Cloud. As a further confounding factor, CI uses tags to select roachtests per given cloud provider [6].

As for skipping incompatible tests, test authors came up with ad hoc workarounds, e.g., [7], [8], thereby complicating both the setup logic, as well as, future refactoring.

[1]

[2]
[3]
[4]
[5]
[6]
[7]
[8]

Jira issue: CRDB-28322