Run search CCS YAML tests in a separate suite#139107
Merged
davidkyle merged 5 commits intoelastic:mainfrom Dec 8, 2025
Merged
Conversation
Collaborator
|
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
rjernst
approved these changes
Dec 5, 2025
Member
rjernst
left a comment
There was a problem hiding this comment.
This seems ok as a temporary mechanism to reduce the test time. Can you please add a comment stressing that point? I don't want someone else coming along and copying this to another rest test project, at that point we need to implement some suite splitting in the rest test runner itself.
benwtrent
approved these changes
Dec 5, 2025
Member
benwtrent
left a comment
There was a problem hiding this comment.
I agree, we should be clear that this is temporary and not a common patter to be copied, on all the classes
This was referenced Dec 9, 2025
carlosdelest
added a commit
to carlosdelest/elasticsearch
that referenced
this pull request
Dec 9, 2025
carlosdelest
added a commit
that referenced
this pull request
Dec 9, 2025
This was referenced Dec 9, 2025
davidkyle
added a commit
that referenced
this pull request
Dec 12, 2025
Follow on from #139107, those tests are handled by a different IT
john-wagster
added a commit
to john-wagster/elasticsearch
that referenced
this pull request
Dec 12, 2025
elasticsearchmachine
pushed a commit
that referenced
this pull request
Dec 12, 2025
parkertimmins
pushed a commit
to parkertimmins/elasticsearch
that referenced
this pull request
Dec 17, 2025
Unmuting additional test CCS test since it's a timeout related to fixes here: elastic#139107 Fixes: elastic#139381
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Running the entire yaml test set in a single test suite can often fail due to suite timeouts. Those timeouts lead to spurious failures that have to be investigated. This change reduces the number of tests run in a single suite by moving the
searchandmsearchtests into a new test suite and introduces a new class controlling which suite should execute a particular test.The changes here are easily extensible; just subclass
CcsCommonYamlTestSuiteITorRcsCcsCommonYamlTestSuiteITthen add a entry toTestSuiteApiCheckand the mapped APIs will only execute in the new suite.Important: This is a temporary solution not a precedent
This change is to reduce failures due to timeouts in single large test suite. It is not necessary a pattern that should be copied elsewhere.