Skip to content

Move the state of search requests to the coordinator node #46523

@jimczi

Description

@jimczi

This is a meta issue to track the tasks to move the state of a search in the coordinator node.
Today the initial phase of any search creates a SearchContext on each node that contains a shard selected for the request. This SearchContext is then used as a state on each shard for the subsequent phases. This issue proposes to move from a SearchContext on each shard to a ReaderContext that would keep track of the index reader that should be used for the entire lifecycle of a search request and to move all the state of the search to the coordinating node.
To achieve this we need to re-create the search context for each phase based on the results of the previous phase. The state of the previous phase can be passed through the result of the phase and added to the request of the next one in order to be able to rebuild the search state.
Here is a list (hopefully exhaustive) of the tasks that need to be done to achieve this:

There are plenty of follow ups that we could do once we move the state of the request to the coordinator node. For instance we could create a single reader context per directory reader based on sequence numbers that we could check when a replica fails in order to move the search to a different node if another replica is at the same checkpoint (always the case for read-only indices/frozen indices).

Closes #26472

Metadata

Metadata

Assignees

Labels

:Search/SearchSearch-related issues that do not fall into other categoriesMetaTeam:SearchMeta label for search team

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions