Initially we only had two types of test runners: pipeline and system. Both operated at the data stream level (as opposed to the package level). Then we introduced a third type of test runner: asset. This one operated at the package level. To make this work with the existing test runner implementation, we had to introduce a TestRunner interface with a method named CanRunPerDataStream, which feels a bit smelly.
More recently, we've introduced a feature of letting tests be skippable via a configuration block in a test definition. While implementing this feature, @mtojek observed how it could be quite easy in the current test runner implementation to forget to add this feature for future test runner types. More discussion here: https://github.com/elastic/elastic-package/pull/221/files#r565955299.
It's starting to become apparent that the original test runner implementation might be ripe for a refactor. This issue is to track this change.
Initially we only had two types of test runners:
pipelineandsystem. Both operated at the data stream level (as opposed to the package level). Then we introduced a third type of test runner:asset. This one operated at the package level. To make this work with the existing test runner implementation, we had to introduce aTestRunnerinterface with a method namedCanRunPerDataStream, which feels a bit smelly.More recently, we've introduced a feature of letting tests be skippable via a configuration block in a test definition. While implementing this feature, @mtojek observed how it could be quite easy in the current test runner implementation to forget to add this feature for future test runner types. More discussion here: https://github.com/elastic/elastic-package/pull/221/files#r565955299.
It's starting to become apparent that the original test runner implementation might be ripe for a refactor. This issue is to track this change.