Skip to content

[BUG] Improving SourceLookup performance for multi segment use cases #17743

@jainankitk

Description

@jainankitk

Describe the bug

As part of #15072, we initialize a new SourceLookup and pass it into each new LeafSearchLookup instance. This is done as the SearchLookup is not thread safe. We noticed performance regression for one of the customer workload with script queries.

Below is the flame graph for same:

Image

Related component

Search

To Reproduce

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior

SearchLookup should be either:

  • Threadsafe, so that all QueryContext can share the instance
  • Shared across slices to not build for every segment

Ideally, I will prefer the first if possible. Since intuitively, it is okay for thread to block on shared resource (assuming only writes are blocking), instead of duplicating the work across separate SourceLookup object

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Labels

SearchSearch query, autocomplete ...etcbugSomething isn't workingv3.0.0Issues and PRs related to version 3.0.0

Type

No type

Projects

Status

✅ Done

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions