In #30591 I'm adding support for running qa tests with the distribution configured using -Dtests.distribution. Most of the tests work except smoke-test-rank-eval-with-mustache fails because it tries to form a cluster with an ESIntegTestCase. I'm not 100% sure what the test is testing so I didn't want to change it. But I think it should be changed to work with the zip distribution.