Skip to content

[index management] Faster index list loading#246276

Merged
mattkime merged 64 commits intomainfrom
index_mgmt_faster_list_loading
Jan 30, 2026
Merged

[index management] Faster index list loading#246276
mattkime merged 64 commits intomainfrom
index_mgmt_faster_list_loading

Conversation

@mattkime
Copy link
Copy Markdown
Contributor

@mattkime mattkime commented Dec 13, 2025

Summary

The index list can be very slow to load on large clusters with a lot of indices. In some cases it can even fail to load. Previously up to five requests were made and combined to produce the index list. If even one of those requests would fail then the whole list would fail.

Now the requests are made in parallel and failed requests will result in partial data. The indexDataEnricher concept has been brought to the client side which allows the display of data as its loaded, rather than waiting for everything to load. Endpoints were added to support these additional client side requests.

Previously byte values were being formatted server side but that no longer makes sense when keeping kibana server processing minimal. A significant percentage of code changes in this PR are dedicated to this. Some of the formatting has been fixed as previously lowercase abbreviations were used when they should be uppercase. In particular, the B should be capitalized to indicate bytes and not bits.

Release notes

Index Management index list loads much faster on large clusters with a lot of indices.

@mattkime mattkime changed the title partial progress [index management] Faster index list loading Jan 22, 2026
@mattkime mattkime requested a review from a team as a code owner January 29, 2026 15:25
@mattkime
Copy link
Copy Markdown
Contributor Author

/ci

@mattkime
Copy link
Copy Markdown
Contributor Author

/ci

@elasticmachine
Copy link
Copy Markdown
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #109 / serverless observability UI - ML and Discover discover/observabilitySolution/context_awareness extension getRowIndicatorProvider should render log.level row indicators on Surrounding documents page

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
crossClusterReplication 111 112 +1
indexLifecycleManagement 189 190 +1
indexManagement 701 705 +4
rollup 119 120 +1
total +7

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/index-management-shared-types 182 193 +11
indexManagement 256 208 -48
total -37

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
indexManagement 737.6KB 739.1KB +1.5KB
searchIndices 215.9KB 215.9KB -35.0B
total +1.4KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
crossClusterReplication 11.2KB 11.6KB +354.0B
indexLifecycleManagement 27.9KB 28.2KB +328.0B
indexManagement 32.7KB 34.5KB +1.8KB
rollup 11.4KB 11.8KB +364.0B
searchIndices 9.2KB 9.4KB +144.0B
total +3.0KB
Unknown metric groups

API count

id before after diff
@kbn/index-management-shared-types 187 199 +12
indexManagement 262 214 -48
total -36

History

cc @mattkime

Copy link
Copy Markdown
Contributor

@kapral18 kapral18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixes


import React from 'react';
import type { Index } from '@kbn/index-management-shared-types';
import { formatBytes } from '@kbn/index-management-plugin/public';
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is importing this function the only reason we're adding a dependency to this plugin? If so, it might be better just to duplicate the function in the plugin. I'll defer to @TattdCodeMonkey on this one though.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was aiming to keep the formatting consistent. Obviously I could duplicate the functionality if its preferred.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this import looks to only add 144b to the plugin bundle, given that I'm fine with the import to have the consistency.

Copy link
Copy Markdown
Contributor

@TattdCodeMonkey TattdCodeMonkey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

search_indices changes LGTM

@mattkime mattkime merged commit a07b7e9 into main Jan 30, 2026
16 checks passed
@mattkime mattkime deleted the index_mgmt_faster_list_loading branch January 30, 2026 19:49
hannahbrooks pushed a commit to hannahbrooks/kibana that referenced this pull request Jan 30, 2026
## Summary

The index list can be very slow to load on large clusters with a lot of
indices. In some cases it can even fail to load. Previously up to five
requests were made and combined to produce the index list. If even one
of those requests would fail then the whole list would fail.

Now the requests are made in parallel and failed requests will result in
partial data. The indexDataEnricher concept has been brought to the
client side which allows the display of data as its loaded, rather than
waiting for everything to load. Endpoints were added to support these
additional client side requests.

Previously byte values were being formatted server side but that no
longer makes sense when keeping kibana server processing minimal. A
significant percentage of code changes in this PR are dedicated to this.
Some of the formatting has been fixed as previously lowercase
abbreviations were used when they should be uppercase. In particular,
the `B` should be capitalized to indicate bytes and not bits.


### Release notes

Index Management index list loads much faster on large clusters with a
lot of indices.

---------

Signed-off-by: Matthew Kime <matt@mattki.me>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
mbondyra added a commit to mbondyra/kibana that referenced this pull request Feb 2, 2026
…iew_cps

* commit '32efd9b2fb078ade51073fd2d0068bc74c029d6b': (49 commits)
  [Security Solution] Rules exceptions subfeatures (elastic#245722)
  [BK] Upgrade axios (elastic#251150)
  Fix AI Connector form fields resetting to default value when cleared by user (elastic#251095)
  deduplicate otel dependencies (elastic#250841)
  Adds initial agents.md file (elastic#250833)
  [index management] Faster index list loading (elastic#246276)
  skip failing test suite (elastic#251086)
  skip failing test suite (elastic#251048)
  [Security Solutions] Trial Companion - adjust UX design (elastic#250910)
  [Traces][Discover] Prevent flyout remount when switching document types in Trace Waterfall (elastic#250406)
  [DOCS][Cases][9.4 & Serverless]: Doc new `Maximum amount of cases to open` setting for case action (elastic#250993)
  [Discover][Traces] Explore trace.id from logs in Discover (elastic#249632)
  Remove ! from SOs docs link (elastic#251097)
  [ML] Maps: Add telemetry events for file uploads (elastic#247543)
  [Fleet] Fix dupplicate ids when copying an integration policy or an agent policy (elastic#250971)
  [Dashboards as Code] Add snake case object keys util (elastic#250962)
  [Core] Remove URL Overflow & Deprecate `storeInSessionStorage` setting (elastic#242972)
  [One Workflow] fix: Fix Variable Retrieval in Workflow Execution Engine (elastic#250852)
  Rework Elastic Managed LLMs page (elastic#251069)
  [Lens powered by ES|QL] Update Switch to Query mode modal warning message (elastic#251051)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting Feature:Index Management Index and index templates UI release_note:feature Makes this part of the condensed release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants