Description

Compatible REST tests should support copying files from named modules (or some other more expressive way to find tests) from elder branches. Currently the compatible REST tests support copying files from the elder branch if those files are

• found in rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test and specified via restResources{includeCore
• found in x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test and specficied via restResources{includeXpack
• found in the mirrored directory structure. For example, if the current project is in modules/ingest-common, then we will copy test from modules/ingest-common in the elder branch

This strategy works for majority of the REST compatible testing concerns. However, there are two use cases missing.

• Moving/renaming the module between major branches. For example, if modules/ingest-common is renamed in main, but not 7.x. Then we no longer have the mirrored directory structure and will not find the compatible tests.
• Tests sourced from restXpackTestConfig and restTestConfig. It is possible for module B to pull the tests from module A via the test configuration. For example, :x-pack:plugin:watcher:qa:with-security sources it's tests from :x-pack:plugin:watcher:qa:rest via the restXpackTestConfig. This can not work for compatible REST tests since we source the tests from the elder branch and the elder branch is treated as files on a filesystem, not a Gradle project.

The only known current case is the :x-pack:plugin:watcher:qa:with-security and for that specific case, there is not strong need to run REST compatible tests for the full set of Watcher REST tests. However, I assume at some point in the future, especially if we plan to break up the monolithic testing projects we may need a richer way to tell how/where to source the compatible REST tests.