Describe the bug
Description
List.removeAll is a relatively expensive operation. LocalShardsBalancer filtering out remote unassigned shards from local unassigned shards while making relocation decisions. In order to achieve this, it first filters local unassigned shards from given list of total unassigned shards and then removes (via removeAll call) this new list of local shards with orignal shards to get list of remote unassigned shards. This issue got introduced due to this PR. This List.removeAll takes about 40% of the time to allocate unassigned shards on node as shown in below pic.
Related component
Indexing:Replication
To Reproduce
Create 500k shards on a setup with 1000 data nodes and 3 master nodes.
Expected behavior
Removes this unnecessary call to List.removeAll function optimising performance of LocalShardsBalancer.
Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
- OS: [e.g. iOS]
- Version [e.g. 22]
Additional context
Add any other context about the problem here.
Describe the bug
Description
List.removeAllis a relatively expensive operation.LocalShardsBalancerfiltering out remote unassigned shards from local unassigned shards while making relocation decisions. In order to achieve this, it first filters local unassigned shards from given list of total unassigned shards and then removes (via removeAll call) this new list of local shards with orignal shards to get list of remote unassigned shards. This issue got introduced due to this PR. This List.removeAll takes about 40% of the time to allocate unassigned shards on node as shown in below pic.Related component
Indexing:Replication
To Reproduce
Create 500k shards on a setup with 1000 data nodes and 3 master nodes.
Expected behavior
Removes this unnecessary call to List.removeAll function optimising performance of LocalShardsBalancer.
Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
Additional context
Add any other context about the problem here.