[index management] Faster index list loading#246276
Conversation
Signed-off-by: Matthew Kime <matt@mattki.me>
x-pack/platform/plugins/shared/index_management/public/application/services/api.ts
Outdated
Show resolved
Hide resolved
x-pack/platform/plugins/shared/index_management/public/application/services/api.ts
Outdated
Show resolved
Hide resolved
…ibana into index_mgmt_faster_list_loading
x-pack/platform/plugins/private/index_lifecycle_management/common/types/index.ts
Outdated
Show resolved
Hide resolved
x-pack/platform/plugins/shared/index_management/public/index_stats_enricher.ts
Outdated
Show resolved
Hide resolved
x-pack/platform/plugins/shared/index_management/public/application/services/api.ts
Show resolved
Hide resolved
...ation/sections/home/index_list/details_page/details_page_overview/size_doc_count_details.tsx
Show resolved
Hide resolved
|
/ci |
…ibana into index_mgmt_faster_list_loading
|
/ci |
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Page load bundle
Unknown metric groupsAPI count
History
cc @mattkime |
|
|
||
| import React from 'react'; | ||
| import type { Index } from '@kbn/index-management-shared-types'; | ||
| import { formatBytes } from '@kbn/index-management-plugin/public'; |
There was a problem hiding this comment.
Is importing this function the only reason we're adding a dependency to this plugin? If so, it might be better just to duplicate the function in the plugin. I'll defer to @TattdCodeMonkey on this one though.
There was a problem hiding this comment.
Yes, I was aiming to keep the formatting consistent. Obviously I could duplicate the functionality if its preferred.
There was a problem hiding this comment.
this import looks to only add 144b to the plugin bundle, given that I'm fine with the import to have the consistency.
TattdCodeMonkey
left a comment
There was a problem hiding this comment.
search_indices changes LGTM
## Summary The index list can be very slow to load on large clusters with a lot of indices. In some cases it can even fail to load. Previously up to five requests were made and combined to produce the index list. If even one of those requests would fail then the whole list would fail. Now the requests are made in parallel and failed requests will result in partial data. The indexDataEnricher concept has been brought to the client side which allows the display of data as its loaded, rather than waiting for everything to load. Endpoints were added to support these additional client side requests. Previously byte values were being formatted server side but that no longer makes sense when keeping kibana server processing minimal. A significant percentage of code changes in this PR are dedicated to this. Some of the formatting has been fixed as previously lowercase abbreviations were used when they should be uppercase. In particular, the `B` should be capitalized to indicate bytes and not bits. ### Release notes Index Management index list loads much faster on large clusters with a lot of indices. --------- Signed-off-by: Matthew Kime <matt@mattki.me> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…iew_cps * commit '32efd9b2fb078ade51073fd2d0068bc74c029d6b': (49 commits) [Security Solution] Rules exceptions subfeatures (elastic#245722) [BK] Upgrade axios (elastic#251150) Fix AI Connector form fields resetting to default value when cleared by user (elastic#251095) deduplicate otel dependencies (elastic#250841) Adds initial agents.md file (elastic#250833) [index management] Faster index list loading (elastic#246276) skip failing test suite (elastic#251086) skip failing test suite (elastic#251048) [Security Solutions] Trial Companion - adjust UX design (elastic#250910) [Traces][Discover] Prevent flyout remount when switching document types in Trace Waterfall (elastic#250406) [DOCS][Cases][9.4 & Serverless]: Doc new `Maximum amount of cases to open` setting for case action (elastic#250993) [Discover][Traces] Explore trace.id from logs in Discover (elastic#249632) Remove ! from SOs docs link (elastic#251097) [ML] Maps: Add telemetry events for file uploads (elastic#247543) [Fleet] Fix dupplicate ids when copying an integration policy or an agent policy (elastic#250971) [Dashboards as Code] Add snake case object keys util (elastic#250962) [Core] Remove URL Overflow & Deprecate `storeInSessionStorage` setting (elastic#242972) [One Workflow] fix: Fix Variable Retrieval in Workflow Execution Engine (elastic#250852) Rework Elastic Managed LLMs page (elastic#251069) [Lens powered by ES|QL] Update Switch to Query mode modal warning message (elastic#251051) ...
Summary
The index list can be very slow to load on large clusters with a lot of indices. In some cases it can even fail to load. Previously up to five requests were made and combined to produce the index list. If even one of those requests would fail then the whole list would fail.
Now the requests are made in parallel and failed requests will result in partial data. The indexDataEnricher concept has been brought to the client side which allows the display of data as its loaded, rather than waiting for everything to load. Endpoints were added to support these additional client side requests.
Previously byte values were being formatted server side but that no longer makes sense when keeping kibana server processing minimal. A significant percentage of code changes in this PR are dedicated to this. Some of the formatting has been fixed as previously lowercase abbreviations were used when they should be uppercase. In particular, the
Bshould be capitalized to indicate bytes and not bits.Release notes
Index Management index list loads much faster on large clusters with a lot of indices.