[9.0] tests: make integration split quantity configurable (backport #17219)#17371
Merged
[9.0] tests: make integration split quantity configurable (backport #17219)#17371
Conversation
* tests: make integration split quantity configurable Refactors shared splitter bash function to take a list of files on stdin and split into a configurable number of partitions, emitting only those from the currently-selected partition to stdout. Also refactors the only caller in the integration_tests launcher script to accept an optional partition_count parameter (defaulting to `2` for backward- compatibility), to provide the list of specs to the function's stdin, and to output relevant information about the quantity of partition splits and which was selected. * ci: run integration tests in 3 parts (cherry picked from commit 3e0f488)
|
💚 Build Succeeded
cc @yaauie |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.





Release notes
[rn:skip]
What does this PR do?
Adds new optional positional parameter to
ci/integration_tests.sh split, so that we can tell it how many partitions to split the tests into. When this parameter is missing, it is assumed to be 2 for backward-compatibility.Why is it important/What is the impact to the user?
No user impact.
This allows us to configure a larger quantity of integration spec splits so that we can run more jobs in parallel.
Checklist
[ ] I have made corresponding changes to the documentation[ ] I have made corresponding change to the default configuration files (and/or docker env variables)[ ] I have added tests that prove my fix is effective or that my feature worksHow to test this PR locally
Observe the splits in CI:
1.Split 0
Running integration specs split[0] of 2: specs/01_logstash_bin_smoke_spec.rb specs/cli/http_proxy_install_spec.rb specs/cli/keystore_spec.rb specs/cli/prepare_offline_pack_spec.rb specs/command_line_spec.rb specs/direct_heap_allocator_flag_spec.rb specs/env_variables_condition_spec.rb specs/es_output_how_spec.rb specs/install_java_plugin_spec.rb specs/kafka_input_spec.rb specs/mixed_codec_spec.rb specs/multiple_pipeline_spec.rb specs/plugin_name_log_spec.rb specs/reserved_tags_field_spec.rb specs/settings_spec.rb specs/webserver_spec.rb2.Split 1
Running integration specs split[1] of 2: specs/beats_input_spec.rb specs/cli/install_spec.rb specs/cli/list_spec.rb specs/cli/remove_spec.rb specs/deprecation_log_spec.rb specs/dlq_spec.rb specs/env_variables_config_spec.rb specs/fatal_error_spec.rb specs/java_api_spec.rb specs/logstash_to_logstash_spec.rb specs/monitoring_api_spec.rb specs/pipeline_log_spec.rb specs/reload_config_spec.rb specs/secret_store_spec.rb specs/slowlog_spec.rbOR: