Skip to content

[BUG] Removing unnecessary List.removeAll from LocalShardsBalancer to filter remote search shard from relocation decision #15106

@RS146BIJAY

Description

@RS146BIJAY

Describe the bug

Description

List.removeAll is a relatively expensive operation. LocalShardsBalancer filtering out remote unassigned shards from local unassigned shards while making relocation decisions. In order to achieve this, it first filters local unassigned shards from given list of total unassigned shards and then removes (via removeAll call) this new list of local shards with orignal shards to get list of remote unassigned shards. This issue got introduced due to this PR. This List.removeAll takes about 40% of the time to allocate unassigned shards on node as shown in below pic.

Screenshot 2024-07-01 at 3 37 25 PM

Related component

Indexing:Replication

To Reproduce

Create 500k shards on a setup with 1000 data nodes and 3 master nodes.

Expected behavior

Removes this unnecessary call to List.removeAll function optimising performance of LocalShardsBalancer.

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Indexing:ReplicationIssues and PRs related to core replication framework eg segrepbugSomething isn't workinguntriaged

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions