Skip to content

[8.0] Make TaskBatcher Less Lock-Heavy (#82227)#82624

Merged
elasticsearchmachine merged 1 commit intoelastic:8.0from
original-brownbear:backport/8.0/pr-82227
Jan 14, 2022
Merged

[8.0] Make TaskBatcher Less Lock-Heavy (#82227)#82624
elasticsearchmachine merged 1 commit intoelastic:8.0from
original-brownbear:backport/8.0/pr-82227

Conversation

@original-brownbear
Copy link
Copy Markdown
Contributor

Backports the following commits to 8.0:

In many shards benchmarks we see a lot of contention when submitting tasks.
This is obvious when working with lots of large task batches and doing long iterations.

We don't need to lock in `runIfNotProcessed` past removign the task set for a key
and can be a little more efficient when it comes to creating the new tasks set in
`submitTasks` as well.
Also, we don't need to use a fully locking map as we often have operations for different
batching keys interleaved so moving to CHM as well.

This change is particularly relevant for stability because we often submit tasks from
network threads directly where grinding through a e.g. bunch of shard state updates and
having to lock on the map over and over while e.g. a huge batch of index create or so
was iterated over in `runIfNotProcessed` caused very visible latency.
@original-brownbear original-brownbear added auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport labels Jan 14, 2022
@elasticsearchmachine elasticsearchmachine merged commit 826c451 into elastic:8.0 Jan 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport v8.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants