roachtest currently always creates an ephemeral cluster (either local or remote) for running a test. Especially for remote tests, this dramatically slows test iteration as the roachprod {create,destroy} calls take minutes. We should add a flag to specify that tests are to be run on an existing cluster. roachtest would verify the cluster contains a sufficient number of machines, but the test would otherwise be free to wipe the machines and be responsible for pushing binaries to run. Similar to the --local flag, when using an explicit cluster parallelism should be set to 1.
Cc @danhhz, @benesch, @tschottdorf