Currently, we have slow/fast tests in the test suite, which are intended to allow people to add long running tests to the test suite without gumming up wait times for PR CI turnaround.
I propose to add another new category to the test suite: boring noarch tests. Noarch tests don't exercise any interesting aspects about the underlying CI configuration; in other words, if the noarch tests pass on one configuration, they probably pass everywhere, no need to burn CPU hours in other configurations. A single test configuration will be anointed the "noarch" test runner and we will only run noarch tests there.
Alternative: mark noarch tests as slow tests. I don't like this alternative because it implies that the noarch tests are slow, which they are not; the tests I intend to add in #53682 individually run very fast; there are just a lot of them so it takes time to get through all of them.
cc @mruberry @VitalyFedyunin @walterddr @malfet @seemethere
Currently, we have slow/fast tests in the test suite, which are intended to allow people to add long running tests to the test suite without gumming up wait times for PR CI turnaround.
I propose to add another new category to the test suite:
boringnoarch tests. Noarch tests don't exercise any interesting aspects about the underlying CI configuration; in other words, if the noarch tests pass on one configuration, they probably pass everywhere, no need to burn CPU hours in other configurations. A single test configuration will be anointed the "noarch" test runner and we will only run noarch tests there.Alternative: mark noarch tests as slow tests. I don't like this alternative because it implies that the noarch tests are slow, which they are not; the tests I intend to add in #53682 individually run very fast; there are just a lot of them so it takes time to get through all of them.
cc @mruberry @VitalyFedyunin @walterddr @malfet @seemethere