Make AllocationService#adaptAutoExpandReplicas Faster#83092
Merged
original-brownbear merged 7 commits intoelastic:masterfrom Jan 26, 2022
original-brownbear:faster-auto-expand-replicas
Merged
Make AllocationService#adaptAutoExpandReplicas Faster#83092original-brownbear merged 7 commits intoelastic:masterfrom original-brownbear:faster-auto-expand-replicas
original-brownbear merged 7 commits intoelastic:masterfrom
original-brownbear:faster-auto-expand-replicas
Conversation
This method is getting fairly expensive for large cluster states. In most cases it is not necessary to actually compute the `RoutingAllocation` so I made that lazy to save potentially needlessly building routing nodes. Also, parsing the auto-expand-replicas setting gets quite expensive when looping over thousands of shards in this method so I moved the auto-expand setting value into the index metadata. These changes make the method disappear from profiling in most cases and help make reroute yet a bit faster.
Collaborator
|
Pinging @elastic/es-distributed (Team:Distributed) |
97 tasks
Collaborator
|
Hi @original-brownbear, I've created a changelog YAML for you. |
| private final AllocationDeciders deciders; | ||
|
|
||
| @Nullable | ||
| private final RoutingNodes routingNodes; |
Contributor
Author
There was a problem hiding this comment.
We don't need to generate this when constructing the object. In some cases the routing nodes are never needed for the concrete cluster state passed in here because none of the allocators care about it. So we can selectively make it lazy like this and pass in the mutable variant as an override.
Contributor
Author
|
Thanks David! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This method is getting fairly expensive for large cluster states.
In most cases it is not necessary to actually compute the
RoutingAllocationso I made that lazy to save potentially needlessly building routing nodes.
Also, parsing the auto-expand-replicas setting gets quite expensive when looping
over thousands of shards in this method so I moved the auto-expand setting value
into the index metadata.
These changes make the method disappear from profiling in most cases and help
make reroute yet a bit faster.
relates #77466