Skip to content

Onboard elastic owned ECH clusters to use mget task claiming#196757

Merged
mikecote merged 2 commits intoelastic:mainfrom
mikecote:task-manager/onboard-mget-internal-clusters-ech
Oct 18, 2024
Merged

Onboard elastic owned ECH clusters to use mget task claiming#196757
mikecote merged 2 commits intoelastic:mainfrom
mikecote:task-manager/onboard-mget-internal-clusters-ech

Conversation

@mikecote
Copy link
Copy Markdown
Contributor

@mikecote mikecote commented Oct 17, 2024

Similar to #196317

In this PR, I'm flipping the mget feature flag to on for all elastic owned ECH clusters. Elastic owned clusters are determined by looking at plugins.cloud?.isElasticStaffOwned.

To verify

Observe the PR deployment which doesn't start with a or b yet is using the mget claim strategy by logging Using claim strategy mget on startup.

@mikecote mikecote added release_note:skip Skip the PR/issue when compiling release notes Feature:Task Manager Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// v9.0.0 backport:prev-minor ci:cloud-deploy Create or update a Cloud deployment v8.16.0 v8.17.0 labels Oct 17, 2024
@mikecote mikecote self-assigned this Oct 17, 2024
Copy link
Copy Markdown
Contributor

@ymao1 ymao1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Verified the cloud deployment, which starts with 0, is set to the mget claim strategy.

@mikecote mikecote marked this pull request as ready for review October 17, 2024 21:05
@mikecote mikecote requested a review from a team as a code owner October 17, 2024 21:05
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Oct 17, 2024

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

cc @mikecote

@mikecote mikecote merged commit 97f2a90 into elastic:main Oct 18, 2024
@kibanamachine
Copy link
Copy Markdown
Contributor

Starting backport for target branches: 8.16, 8.x

https://github.com/elastic/kibana/actions/runs/11402898172

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 18, 2024
…ic#196757)

Similar to elastic#196317

In this PR, I'm flipping the mget feature flag to on for all elastic
owned ECH clusters. Elastic owned clusters are determined by looking at
`plugins.cloud?.isElasticStaffOwned`.

## To verify
Observe the PR deployment which doesn't start with `a` or `b` yet is
using the mget claim strategy by logging `Using claim strategy mget` on
startup.

(cherry picked from commit 97f2a90)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 18, 2024
…ic#196757)

Similar to elastic#196317

In this PR, I'm flipping the mget feature flag to on for all elastic
owned ECH clusters. Elastic owned clusters are determined by looking at
`plugins.cloud?.isElasticStaffOwned`.

## To verify
Observe the PR deployment which doesn't start with `a` or `b` yet is
using the mget claim strategy by logging `Using claim strategy mget` on
startup.

(cherry picked from commit 97f2a90)
@kibanamachine
Copy link
Copy Markdown
Contributor

💚 All backports created successfully

Status Branch Result
8.16
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 18, 2024
…k claiming (#196757) (#196855)

# Backport

This will backport the following commits from `main` to `8.16`:
- [Onboard elastic owned ECH clusters to use `mget` task
claiming (#196757)](#196757)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Mike
Côté","email":"mikecote@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-18T11:31:32Z","message":"Onboard
elastic owned ECH clusters to use `mget` task claiming
(#196757)\n\nSimilar to
https://github.com/elastic/kibana/pull/196317\r\n\r\nIn this PR, I'm
flipping the mget feature flag to on for all elastic\r\nowned ECH
clusters. Elastic owned clusters are determined by looking
at\r\n`plugins.cloud?.isElasticStaffOwned`.\r\n\r\n## To
verify\r\nObserve the PR deployment which doesn't start with `a` or `b`
yet is\r\nusing the mget claim strategy by logging `Using claim strategy
mget`
on\r\nstartup.","sha":"97f2a9098fb91708250459910820b1b99d40f1c4","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Feature:Task
Manager","Team:ResponseOps","v9.0.0","backport:prev-minor","ci:cloud-deploy","v8.16.0","v8.17.0"],"title":"Onboard
elastic owned ECH clusters to use `mget` task
claiming","number":196757,"url":"https://github.com/elastic/kibana/pull/196757","mergeCommit":{"message":"Onboard
elastic owned ECH clusters to use `mget` task claiming
(#196757)\n\nSimilar to
https://github.com/elastic/kibana/pull/196317\r\n\r\nIn this PR, I'm
flipping the mget feature flag to on for all elastic\r\nowned ECH
clusters. Elastic owned clusters are determined by looking
at\r\n`plugins.cloud?.isElasticStaffOwned`.\r\n\r\n## To
verify\r\nObserve the PR deployment which doesn't start with `a` or `b`
yet is\r\nusing the mget claim strategy by logging `Using claim strategy
mget`
on\r\nstartup.","sha":"97f2a9098fb91708250459910820b1b99d40f1c4"}},"sourceBranch":"main","suggestedTargetBranches":["8.16","8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/196757","number":196757,"mergeCommit":{"message":"Onboard
elastic owned ECH clusters to use `mget` task claiming
(#196757)\n\nSimilar to
https://github.com/elastic/kibana/pull/196317\r\n\r\nIn this PR, I'm
flipping the mget feature flag to on for all elastic\r\nowned ECH
clusters. Elastic owned clusters are determined by looking
at\r\n`plugins.cloud?.isElasticStaffOwned`.\r\n\r\n## To
verify\r\nObserve the PR deployment which doesn't start with `a` or `b`
yet is\r\nusing the mget claim strategy by logging `Using claim strategy
mget`
on\r\nstartup.","sha":"97f2a9098fb91708250459910820b1b99d40f1c4"}},{"branch":"8.16","label":"v8.16.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Mike Côté <mikecote@users.noreply.github.com>
kibanamachine added a commit that referenced this pull request Oct 18, 2024
… claiming (#196757) (#196856)

# Backport

This will backport the following commits from `main` to `8.x`:
- [Onboard elastic owned ECH clusters to use &#x60;mget&#x60; task
claiming (#196757)](#196757)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Mike
Côté","email":"mikecote@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-18T11:31:32Z","message":"Onboard
elastic owned ECH clusters to use `mget` task claiming
(#196757)\n\nSimilar to
https://github.com/elastic/kibana/pull/196317\r\n\r\nIn this PR, I'm
flipping the mget feature flag to on for all elastic\r\nowned ECH
clusters. Elastic owned clusters are determined by looking
at\r\n`plugins.cloud?.isElasticStaffOwned`.\r\n\r\n## To
verify\r\nObserve the PR deployment which doesn't start with `a` or `b`
yet is\r\nusing the mget claim strategy by logging `Using claim strategy
mget`
on\r\nstartup.","sha":"97f2a9098fb91708250459910820b1b99d40f1c4","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Feature:Task
Manager","Team:ResponseOps","v9.0.0","backport:prev-minor","ci:cloud-deploy","v8.16.0","v8.17.0"],"title":"Onboard
elastic owned ECH clusters to use `mget` task
claiming","number":196757,"url":"https://github.com/elastic/kibana/pull/196757","mergeCommit":{"message":"Onboard
elastic owned ECH clusters to use `mget` task claiming
(#196757)\n\nSimilar to
https://github.com/elastic/kibana/pull/196317\r\n\r\nIn this PR, I'm
flipping the mget feature flag to on for all elastic\r\nowned ECH
clusters. Elastic owned clusters are determined by looking
at\r\n`plugins.cloud?.isElasticStaffOwned`.\r\n\r\n## To
verify\r\nObserve the PR deployment which doesn't start with `a` or `b`
yet is\r\nusing the mget claim strategy by logging `Using claim strategy
mget`
on\r\nstartup.","sha":"97f2a9098fb91708250459910820b1b99d40f1c4"}},"sourceBranch":"main","suggestedTargetBranches":["8.16","8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/196757","number":196757,"mergeCommit":{"message":"Onboard
elastic owned ECH clusters to use `mget` task claiming
(#196757)\n\nSimilar to
https://github.com/elastic/kibana/pull/196317\r\n\r\nIn this PR, I'm
flipping the mget feature flag to on for all elastic\r\nowned ECH
clusters. Elastic owned clusters are determined by looking
at\r\n`plugins.cloud?.isElasticStaffOwned`.\r\n\r\n## To
verify\r\nObserve the PR deployment which doesn't start with `a` or `b`
yet is\r\nusing the mget claim strategy by logging `Using claim strategy
mget`
on\r\nstartup.","sha":"97f2a9098fb91708250459910820b1b99d40f1c4"}},{"branch":"8.16","label":"v8.16.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Mike Côté <mikecote@users.noreply.github.com>
mikecote added a commit to mikecote/kibana that referenced this pull request Oct 21, 2024
mikecote added a commit that referenced this pull request Oct 25, 2024
Resolves #194625

In this PR, I'm setting `mget` as the default task claiming strategy
along the following changes:
- Given we no longer need the 8.16 specific PRs
(#196317 and
#196757), I've also reverted them.
- Given we now use `met` as the default, I've renamed
`task_manager_claimer_mget` to `task_manager_claimer_update_by_query`
and made tests in that folder test using the `update_by_query` claim
strategy.
- Stabilize flaky tests caused by mget + polling for tasks more
frequently

Flaky test runners:
-
[[59b71bc](59b71bc)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197
-
[[aea910e](aea910e)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199
-
[[4723ced](4723ced)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206
-
[[d28c8c5](d28c8c5)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209
-
[[dd7773a](dd7773a)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Oct 25, 2024
Resolves elastic#194625

In this PR, I'm setting `mget` as the default task claiming strategy
along the following changes:
- Given we no longer need the 8.16 specific PRs
(elastic#196317 and
elastic#196757), I've also reverted them.
- Given we now use `met` as the default, I've renamed
`task_manager_claimer_mget` to `task_manager_claimer_update_by_query`
and made tests in that folder test using the `update_by_query` claim
strategy.
- Stabilize flaky tests caused by mget + polling for tasks more
frequently

Flaky test runners:
-
[[59b71bc](elastic@59b71bc)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197
-
[[aea910e](elastic@aea910e)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199
-
[[4723ced](elastic@4723ced)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206
-
[[d28c8c5](elastic@d28c8c5)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209
-
[[dd7773a](elastic@dd7773a)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit c31f11e)
kibanamachine added a commit that referenced this pull request Oct 25, 2024
# Backport

This will backport the following commits from `main` to `8.x`:
- [Set mget task claim strategy as the default
(#197070)](#197070)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Mike
Côté","email":"mikecote@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-25T12:57:46Z","message":"Set
mget task claim strategy as the default (#197070)\n\nResolves
https://github.com/elastic/kibana/issues/194625\r\n\r\nIn this PR, I'm
setting `mget` as the default task claiming strategy\r\nalong the
following changes:\r\n- Given we no longer need the 8.16 specific
PRs\r\n(#196317
and\r\nhttps://github.com//pull/196757), I've also
reverted them.\r\n- Given we now use `met` as the default, I've
renamed\r\n`task_manager_claimer_mget` to
`task_manager_claimer_update_by_query`\r\nand made tests in that folder
test using the `update_by_query` claim\r\nstrategy.\r\n- Stabilize flaky
tests caused by mget + polling for tasks more\r\nfrequently\r\n\r\nFlaky
test
runners:\r\n-\r\n[[59b71bc](https://github.com/elastic/kibana/pull/197070/commits/59b71bcdbe4d617a6d91131976540b334c9220ff)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197\r\n-\r\n[[aea910e](https://github.com/elastic/kibana/pull/197070/commits/aea910e36dc71116dee708a7168971df30a18a3d)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199\r\n-\r\n[[4723ced](https://github.com/elastic/kibana/pull/197070/commits/4723ced751f0e5114a9bc7a2928dcf0cb326472e)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206\r\n-\r\n[[d28c8c5](https://github.com/elastic/kibana/pull/197070/commits/d28c8c56f67802107c17a627357251b9eff797ba)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209\r\n-\r\n[[dd7773a](https://github.com/elastic/kibana/pull/197070/commits/dd7773aebad5664e725c9849c0ed9418f9dc68ed)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c31f11e7d8c5c586258399c5e702e2247e05d0e4","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Feature:Task
Manager","Team:ResponseOps","v9.0.0","backport:prev-minor","v8.17.0"],"title":"Set
mget task claim strategy as the
default","number":197070,"url":"https://github.com/elastic/kibana/pull/197070","mergeCommit":{"message":"Set
mget task claim strategy as the default (#197070)\n\nResolves
https://github.com/elastic/kibana/issues/194625\r\n\r\nIn this PR, I'm
setting `mget` as the default task claiming strategy\r\nalong the
following changes:\r\n- Given we no longer need the 8.16 specific
PRs\r\n(#196317
and\r\nhttps://github.com//pull/196757), I've also
reverted them.\r\n- Given we now use `met` as the default, I've
renamed\r\n`task_manager_claimer_mget` to
`task_manager_claimer_update_by_query`\r\nand made tests in that folder
test using the `update_by_query` claim\r\nstrategy.\r\n- Stabilize flaky
tests caused by mget + polling for tasks more\r\nfrequently\r\n\r\nFlaky
test
runners:\r\n-\r\n[[59b71bc](https://github.com/elastic/kibana/pull/197070/commits/59b71bcdbe4d617a6d91131976540b334c9220ff)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197\r\n-\r\n[[aea910e](https://github.com/elastic/kibana/pull/197070/commits/aea910e36dc71116dee708a7168971df30a18a3d)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199\r\n-\r\n[[4723ced](https://github.com/elastic/kibana/pull/197070/commits/4723ced751f0e5114a9bc7a2928dcf0cb326472e)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206\r\n-\r\n[[d28c8c5](https://github.com/elastic/kibana/pull/197070/commits/d28c8c56f67802107c17a627357251b9eff797ba)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209\r\n-\r\n[[dd7773a](https://github.com/elastic/kibana/pull/197070/commits/dd7773aebad5664e725c9849c0ed9418f9dc68ed)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c31f11e7d8c5c586258399c5e702e2247e05d0e4"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/197070","number":197070,"mergeCommit":{"message":"Set
mget task claim strategy as the default (#197070)\n\nResolves
https://github.com/elastic/kibana/issues/194625\r\n\r\nIn this PR, I'm
setting `mget` as the default task claiming strategy\r\nalong the
following changes:\r\n- Given we no longer need the 8.16 specific
PRs\r\n(#196317
and\r\nhttps://github.com//pull/196757), I've also
reverted them.\r\n- Given we now use `met` as the default, I've
renamed\r\n`task_manager_claimer_mget` to
`task_manager_claimer_update_by_query`\r\nand made tests in that folder
test using the `update_by_query` claim\r\nstrategy.\r\n- Stabilize flaky
tests caused by mget + polling for tasks more\r\nfrequently\r\n\r\nFlaky
test
runners:\r\n-\r\n[[59b71bc](https://github.com/elastic/kibana/pull/197070/commits/59b71bcdbe4d617a6d91131976540b334c9220ff)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197\r\n-\r\n[[aea910e](https://github.com/elastic/kibana/pull/197070/commits/aea910e36dc71116dee708a7168971df30a18a3d)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199\r\n-\r\n[[4723ced](https://github.com/elastic/kibana/pull/197070/commits/4723ced751f0e5114a9bc7a2928dcf0cb326472e)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206\r\n-\r\n[[d28c8c5](https://github.com/elastic/kibana/pull/197070/commits/d28c8c56f67802107c17a627357251b9eff797ba)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209\r\n-\r\n[[dd7773a](https://github.com/elastic/kibana/pull/197070/commits/dd7773aebad5664e725c9849c0ed9418f9dc68ed)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c31f11e7d8c5c586258399c5e702e2247e05d0e4"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Mike Côté <mikecote@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:cloud-deploy Create or update a Cloud deployment Feature:Task Manager release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// v8.16.0 v8.17.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants