[Watcher] Prevent expensive queries on ES by using PointInTimeFinder to get indexPatterns by sabarasaba · Pull Request #119717 · elastic/kibana

sabarasaba · 2021-11-25T10:56:20Z

Addresses #93770 in which Watcher could potentially create expensive queries to ES.

Summary

Size parameter was changed from 10000 to 1000
A new server side route that uses createPointInTimeFinder for paging. The client side code only uses the indexPattern names, so we can just return that from the server and avoid sending unnecessary information.

…formant way

sabarasaba · 2021-11-29T13:13:43Z

@elasticmachine merge upstream

…formance

elasticmachine · 2021-11-30T10:00:50Z

Pinging @elastic/kibana-stack-management (Team:Stack Management)

sabarasaba · 2021-11-30T13:00:40Z

@elasticmachine merge upstream

…formance

x-pack/plugins/watcher/__jest__/client_integration/watch_edit.test.ts

alexwizp

Code LGTM! Thank you

sebelga

LGTM! Thanks for fixing this @sabarasaba. Tested locally and works as expected 👍 Left a tiny comment about at type.

x-pack/plugins/watcher/server/routes/api/indices/register_get_index_patterns_route.ts

sabarasaba · 2021-12-03T09:15:52Z

@elasticmachine merge upstream

…formance

kibana-ci · 2021-12-03T10:39:32Z

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`watcher`	272.6KB	272.6KB	-68.0B

History

💚 Build #10286 succeeded 85512ed
💚 Build #9960 succeeded de58a9c
💚 Build #9902 succeeded c15530a
💔 Build #9594 failed 14e2720
💔 Build #9573 failed 99ec995
💔 Build #9556 failed 80d7c01

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @sabarasaba

…to get indexPatterns (elastic#119717) * Use PointInTimeFinder to paginate through indexPatterns in a more performant way * commit using @elastic.co * Start working on tests * Add tests * Add missing types * Fix tests * Update tests responses * Remove unnecessary mocks * Fix type Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Contributes to: elastic#194817 Why I assigned them: Assigned advanced_settings due to elastic#175255 Assigned dashboard_controls due to elastic#190797 Assigned dashboard_links due to elastic#174772 Assigned dashboard_panel_options due to elastic#178596 Assigned grok_debugger due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/grokdebugger/kibana.jsonc#L4 Assigned helpers due to elastic#164341 (call site) Assigned home due to elastic#103192 Assigned index_lifecycle_management due to elastic#116207 Assigned ingest_node_pipelines due to elastic#113783 Assigned kibana_overview due to https://github.com/elastic/kibana/blob/f00ac7a8a21463e6bb4a2784c3a3884f36c62900/x-pack/plugins/grokdebugger/kibana.jsonc#L4 Assigned management due to elastic#165705 Assigned painless_lab due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/painless_lab/kibana.jsonc#L4 Assigned search_profiler due to elastic#195343 Assigned uptime due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/uptime/kibana.jsonc#L4 Assigned lens due to elastic#175893 Assigned ml_anomaly_detection due to elastic#162126 Assigned canvas due to elastic#164376 Assigned cc replication due to elastic#149069 Assigned enterpise search due to elastic#79359 Assigned graph to due elastic#190797 Assigned license_management due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/license_management/kibana.jsonc#L4 Assigned maps due to elastic#155161 Assigned observability due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/observability/kibana.jsonc#L4 Assigned remote clusters due to elastic#96989 Assigned reporting due to elastic#121435 Assigned rollup_jobs due to https://github.com/elastic/kibana/blob/d57bc9a5d7d64f86b550eff7997605a3090aee9a/x-pack/plugins/rollup/kibana.jsonc#L4 Assigned watcher due to elastic#119717

## Summary Assign files within `x-pack/test/accessibility/apps/group[1|2|3]` ### Why I assigned them: Assigned advanced_settings due to #175255 Assigned dashboard_controls due to #190797 Assigned dashboard_links due to #174772 Assigned dashboard_panel_options due to #178596 Assigned grok_debugger due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/grokdebugger/kibana.jsonc#L4 Assigned helpers due to #164341 (call site) Assigned home due to #103192 Assigned index_lifecycle_management due to #116207 Assigned ingest_node_pipelines due to #113783 Assigned kibana_overview due to https://github.com/elastic/kibana/blob/f00ac7a8a21463e6bb4a2784c3a3884f36c62900/x-pack/plugins/grokdebugger/kibana.jsonc#L4 Assigned management due to #165705 Assigned painless_lab due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/painless_lab/kibana.jsonc#L4 Assigned search_profiler due to #195343 Assigned uptime due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/uptime/kibana.jsonc#L4 Assigned lens due to #175893 Assigned ml_anomaly_detection due to #162126 Assigned canvas due to #164376 Assigned cc replication due to #149069 Assigned enterpise search due to #79359 Assigned graph to due #190797 Assigned license_management due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/license_management/kibana.jsonc#L4 Assigned maps due to #155161 Assigned observability due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/observability/kibana.jsonc#L4 Assigned remote clusters due to #96989 Assigned reporting due to #121435 Assigned rollup_jobs due to https://github.com/elastic/kibana/blob/d57bc9a5d7d64f86b550eff7997605a3090aee9a/x-pack/plugins/rollup/kibana.jsonc#L4 Assigned watcher due to #119717 Contributes to: #194817 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Rodney Norris <rodney@tattdcodemonkey.com>

Use PointInTimeFinder to paginate through indexPatterns in a more per…

8acd6f8

…formant way

sabarasaba self-assigned this Nov 25, 2021

sabarasaba and others added 4 commits November 25, 2021 11:57

commit using @elastic.co

db61939

Start working on tests

f380922

Add tests

80d7c01

Add missing types

99ec995

sabarasaba added backport:skip This PR does not require backporting chore Feature:Watcher Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// v8.1.0 release_note:skip Skip the PR/issue when compiling release notes labels Nov 29, 2021

kibanamachine and others added 3 commits November 29, 2021 08:13

Merge branch 'main' into watcher/use_pointInTimeFinder_to_improve_per…

14e2720

…formance

Fix tests

c33b64f

Update tests responses

c15530a

sabarasaba requested a review from alexwizp November 30, 2021 10:00

sabarasaba marked this pull request as ready for review November 30, 2021 10:00

sabarasaba requested a review from a team as a code owner November 30, 2021 10:00

sabarasaba added the ci:deploy-cloud label Nov 30, 2021

Merge branch 'main' into watcher/use_pointInTimeFinder_to_improve_per…

de58a9c

…formance

alexwizp reviewed Dec 1, 2021

View reviewed changes

x-pack/plugins/watcher/__jest__/client_integration/watch_edit.test.ts Outdated Show resolved Hide resolved

alexwizp approved these changes Dec 1, 2021

View reviewed changes

Remove unnecessary mocks

85512ed

sebelga self-requested a review December 2, 2021 09:14

sebelga approved these changes Dec 2, 2021

View reviewed changes

x-pack/plugins/watcher/server/routes/api/indices/register_get_index_patterns_route.ts Outdated Show resolved Hide resolved

Fix type

f152e4b

Merge branch 'main' into watcher/use_pointInTimeFinder_to_improve_per…

d7dfd4b

…formance

sabarasaba added auto-backport Deprecated - use backport:version if exact versions are needed and removed backport:skip This PR does not require backporting labels Dec 3, 2021

sabarasaba merged commit fd741b4 into elastic:main Dec 3, 2021

kibanamachine added the backport:skip This PR does not require backporting label Dec 3, 2021

tylersmalley added ci:cloud-deploy Create or update a Cloud deployment and removed ci:deploy-cloud labels Aug 17, 2022

wayneseymour mentioned this pull request Oct 11, 2024

[FTR][Ownership] Assign ally/grps 1,2 & 3 #195964

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Watcher] Prevent expensive queries on ES by using PointInTimeFinder to get indexPatterns#119717

[Watcher] Prevent expensive queries on ES by using PointInTimeFinder to get indexPatterns#119717
sabarasaba merged 12 commits intoelastic:mainfrom
sabarasaba:watcher/use_pointInTimeFinder_to_improve_performance

sabarasaba commented Nov 25, 2021 •

edited

Loading

Uh oh!

sabarasaba commented Nov 29, 2021

Uh oh!

elasticmachine commented Nov 30, 2021

Uh oh!

sabarasaba commented Nov 30, 2021

Uh oh!

Uh oh!

alexwizp left a comment

Uh oh!

sebelga left a comment

Uh oh!

Uh oh!

sabarasaba commented Dec 3, 2021

Uh oh!

kibana-ci commented Dec 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

sabarasaba commented Nov 25, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

sabarasaba commented Nov 29, 2021

Uh oh!

elasticmachine commented Nov 30, 2021

Uh oh!

sabarasaba commented Nov 30, 2021

Uh oh!

Uh oh!

alexwizp left a comment

Choose a reason for hiding this comment

Uh oh!

sebelga left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sabarasaba commented Dec 3, 2021

Uh oh!

kibana-ci commented Dec 3, 2021

💚 Build Succeeded

Metrics [docs]

Async chunks

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

sabarasaba commented Nov 25, 2021 •

edited

Loading