Skip to content

Set mget task claim strategy as the default#197070

Merged
mikecote merged 22 commits intoelastic:mainfrom
mikecote:task-manager/mget-default-claim-strategy-2
Oct 25, 2024
Merged

Set mget task claim strategy as the default#197070
mikecote merged 22 commits intoelastic:mainfrom
mikecote:task-manager/mget-default-claim-strategy-2

Conversation

@mikecote
Copy link
Copy Markdown
Contributor

@mikecote mikecote commented Oct 21, 2024

Resolves #194625

In this PR, I'm setting mget as the default task claiming strategy along the following changes:

Flaky test runners:

@mikecote mikecote added Feature:Task Manager Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// labels Oct 21, 2024
@mikecote mikecote self-assigned this Oct 21, 2024
@mikecote mikecote added release_note:skip Skip the PR/issue when compiling release notes v9.0.0 backport:prev-minor v8.17.0 labels Oct 21, 2024
@kibanamachine
Copy link
Copy Markdown
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7197

[✅] x-pack/test/alerting_api_integration/observability/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/basic/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 10/10 tests passed.
[❌] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts: 4/10 tests passed.
[❌] x-pack/test/alerting_api_integration/security_and_spaces/group4/config.ts: 2/10 tests passed.
[❌] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts: 0/10 tests passed.
[❌] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group3/config.ts: 8/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/actions/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/action_task_params/config.ts: 10/10 tests passed.
[❌] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group4/config.ts: 9/10 tests passed.
[❌] x-pack/test/task_manager_claimer_update_by_query/config.ts: 0/10 tests passed.
[✅] x-pack/test/plugin_api_integration/config.ts: 10/10 tests passed.

see run history

…om:mikecote/kibana into task-manager/mget-default-claim-strategy-2
@kibanamachine
Copy link
Copy Markdown
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7199

[✅] x-pack/test/alerting_api_integration/observability/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/actions/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/basic/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/action_task_params/config.ts: 10/10 tests passed.
[❌] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group4/config.ts: 0/10 tests passed.
[❌] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts: 0/10 tests passed.
[❌] x-pack/test/task_manager_claimer_update_by_query/config.ts: 4/10 tests passed.
[❌] x-pack/test/plugin_api_integration/config.ts: 9/10 tests passed.

see run history

@kibanamachine
Copy link
Copy Markdown
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7206

[✅] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/observability/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/action_task_params/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/basic/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/actions/config.ts: 10/10 tests passed.
[❌] x-pack/test/task_manager_claimer_update_by_query/config.ts: 1/10 tests passed.
[❌] x-pack/test/plugin_api_integration/config.ts: 0/10 tests passed.

see run history

@kibanamachine
Copy link
Copy Markdown
Contributor

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7209

[✅] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/observability/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/basic/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/action_task_params/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/actions/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/task_manager_claimer_update_by_query/config.ts: 10/10 tests passed.
[✅] x-pack/test/plugin_api_integration/config.ts: 10/10 tests passed.

see run history

@kibanamachine
Copy link
Copy Markdown
Contributor

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7224

[✅] x-pack/test/alerting_api_integration/observability/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/basic/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/action_task_params/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/actions/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/task_manager_claimer_update_by_query/config.ts: 10/10 tests passed.
[✅] x-pack/test/plugin_api_integration/config.ts: 10/10 tests passed.

see run history

@mikecote mikecote marked this pull request as ready for review October 23, 2024 23:33
@mikecote mikecote requested review from a team as code owners October 23, 2024 23:33
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@kc13greiner kc13greiner self-requested a review October 24, 2024 12:19
Copy link
Copy Markdown
Contributor

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.buildkite/ftr_platform_stateful_configs.yml

Copy link
Copy Markdown
Contributor

@kc13greiner kc13greiner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Copy Markdown
Contributor

@ymao1 ymao1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
taskManager 65 64 -1
Unknown metric groups

API count

id before after diff
taskManager 109 108 -1

History

cc @mikecote

@mikecote mikecote merged commit c31f11e into elastic:main Oct 25, 2024
@kibanamachine
Copy link
Copy Markdown
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11518666754

kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Oct 25, 2024
Resolves elastic#194625

In this PR, I'm setting `mget` as the default task claiming strategy
along the following changes:
- Given we no longer need the 8.16 specific PRs
(elastic#196317 and
elastic#196757), I've also reverted them.
- Given we now use `met` as the default, I've renamed
`task_manager_claimer_mget` to `task_manager_claimer_update_by_query`
and made tests in that folder test using the `update_by_query` claim
strategy.
- Stabilize flaky tests caused by mget + polling for tasks more
frequently

Flaky test runners:
-
[[59b71bc](elastic@59b71bc)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197
-
[[aea910e](elastic@aea910e)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199
-
[[4723ced](elastic@4723ced)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206
-
[[d28c8c5](elastic@d28c8c5)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209
-
[[dd7773a](elastic@dd7773a)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit c31f11e)
@kibanamachine
Copy link
Copy Markdown
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 25, 2024
# Backport

This will backport the following commits from `main` to `8.x`:
- [Set mget task claim strategy as the default
(#197070)](#197070)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Mike
Côté","email":"mikecote@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-25T12:57:46Z","message":"Set
mget task claim strategy as the default (#197070)\n\nResolves
https://github.com/elastic/kibana/issues/194625\r\n\r\nIn this PR, I'm
setting `mget` as the default task claiming strategy\r\nalong the
following changes:\r\n- Given we no longer need the 8.16 specific
PRs\r\n(#196317
and\r\nhttps://github.com//pull/196757), I've also
reverted them.\r\n- Given we now use `met` as the default, I've
renamed\r\n`task_manager_claimer_mget` to
`task_manager_claimer_update_by_query`\r\nand made tests in that folder
test using the `update_by_query` claim\r\nstrategy.\r\n- Stabilize flaky
tests caused by mget + polling for tasks more\r\nfrequently\r\n\r\nFlaky
test
runners:\r\n-\r\n[[59b71bc](https://github.com/elastic/kibana/pull/197070/commits/59b71bcdbe4d617a6d91131976540b334c9220ff)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197\r\n-\r\n[[aea910e](https://github.com/elastic/kibana/pull/197070/commits/aea910e36dc71116dee708a7168971df30a18a3d)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199\r\n-\r\n[[4723ced](https://github.com/elastic/kibana/pull/197070/commits/4723ced751f0e5114a9bc7a2928dcf0cb326472e)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206\r\n-\r\n[[d28c8c5](https://github.com/elastic/kibana/pull/197070/commits/d28c8c56f67802107c17a627357251b9eff797ba)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209\r\n-\r\n[[dd7773a](https://github.com/elastic/kibana/pull/197070/commits/dd7773aebad5664e725c9849c0ed9418f9dc68ed)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c31f11e7d8c5c586258399c5e702e2247e05d0e4","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Feature:Task
Manager","Team:ResponseOps","v9.0.0","backport:prev-minor","v8.17.0"],"title":"Set
mget task claim strategy as the
default","number":197070,"url":"https://github.com/elastic/kibana/pull/197070","mergeCommit":{"message":"Set
mget task claim strategy as the default (#197070)\n\nResolves
https://github.com/elastic/kibana/issues/194625\r\n\r\nIn this PR, I'm
setting `mget` as the default task claiming strategy\r\nalong the
following changes:\r\n- Given we no longer need the 8.16 specific
PRs\r\n(#196317
and\r\nhttps://github.com//pull/196757), I've also
reverted them.\r\n- Given we now use `met` as the default, I've
renamed\r\n`task_manager_claimer_mget` to
`task_manager_claimer_update_by_query`\r\nand made tests in that folder
test using the `update_by_query` claim\r\nstrategy.\r\n- Stabilize flaky
tests caused by mget + polling for tasks more\r\nfrequently\r\n\r\nFlaky
test
runners:\r\n-\r\n[[59b71bc](https://github.com/elastic/kibana/pull/197070/commits/59b71bcdbe4d617a6d91131976540b334c9220ff)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197\r\n-\r\n[[aea910e](https://github.com/elastic/kibana/pull/197070/commits/aea910e36dc71116dee708a7168971df30a18a3d)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199\r\n-\r\n[[4723ced](https://github.com/elastic/kibana/pull/197070/commits/4723ced751f0e5114a9bc7a2928dcf0cb326472e)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206\r\n-\r\n[[d28c8c5](https://github.com/elastic/kibana/pull/197070/commits/d28c8c56f67802107c17a627357251b9eff797ba)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209\r\n-\r\n[[dd7773a](https://github.com/elastic/kibana/pull/197070/commits/dd7773aebad5664e725c9849c0ed9418f9dc68ed)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c31f11e7d8c5c586258399c5e702e2247e05d0e4"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/197070","number":197070,"mergeCommit":{"message":"Set
mget task claim strategy as the default (#197070)\n\nResolves
https://github.com/elastic/kibana/issues/194625\r\n\r\nIn this PR, I'm
setting `mget` as the default task claiming strategy\r\nalong the
following changes:\r\n- Given we no longer need the 8.16 specific
PRs\r\n(#196317
and\r\nhttps://github.com//pull/196757), I've also
reverted them.\r\n- Given we now use `met` as the default, I've
renamed\r\n`task_manager_claimer_mget` to
`task_manager_claimer_update_by_query`\r\nand made tests in that folder
test using the `update_by_query` claim\r\nstrategy.\r\n- Stabilize flaky
tests caused by mget + polling for tasks more\r\nfrequently\r\n\r\nFlaky
test
runners:\r\n-\r\n[[59b71bc](https://github.com/elastic/kibana/pull/197070/commits/59b71bcdbe4d617a6d91131976540b334c9220ff)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197\r\n-\r\n[[aea910e](https://github.com/elastic/kibana/pull/197070/commits/aea910e36dc71116dee708a7168971df30a18a3d)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199\r\n-\r\n[[4723ced](https://github.com/elastic/kibana/pull/197070/commits/4723ced751f0e5114a9bc7a2928dcf0cb326472e)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206\r\n-\r\n[[d28c8c5](https://github.com/elastic/kibana/pull/197070/commits/d28c8c56f67802107c17a627357251b9eff797ba)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209\r\n-\r\n[[dd7773a](https://github.com/elastic/kibana/pull/197070/commits/dd7773aebad5664e725c9849c0ed9418f9dc68ed)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c31f11e7d8c5c586258399c5e702e2247e05d0e4"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Mike Côté <mikecote@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature:Task Manager release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// v8.17.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make mget task claim strategy the default

7 participants