Skip to content

[8.3] Avoid capturing SnapshotsInProgress$Entry in queue (#88707)#88724

Merged
elasticsearchmachine merged 1 commit intoelastic:8.3from
DaveCTurner:backport/8.3/pr-88707
Jul 22, 2022
Merged

[8.3] Avoid capturing SnapshotsInProgress$Entry in queue (#88707)#88724
elasticsearchmachine merged 1 commit intoelastic:8.3from
DaveCTurner:backport/8.3/pr-88707

Conversation

@DaveCTurner
Copy link
Copy Markdown
Member

Backports the following commits to 8.3:

Today each time there's shards to snapshot we enqueue a lambda which
captures the current `SnapshotsInProgress$Entry`. This is a pretty
heavyweight object, possibly several MB in size, most of which is not
necessary to capture, and with concurrent snapshots across thousands of
shards we may enqueue many hundreds of slightly different such objects.
With this commit we compute a more efficient representation of the work
to be done by each task in the queue instead.

Relates elastic#77466
@DaveCTurner DaveCTurner added :Distributed/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >bug auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport Team:Distributed Meta label for distributed team. labels Jul 22, 2022
@elasticsearchmachine elasticsearchmachine merged commit 8c118e1 into elastic:8.3 Jul 22, 2022
@DaveCTurner DaveCTurner deleted the backport/8.3/pr-88707 branch July 22, 2022 13:40
@mark-vieira mark-vieira added v8.3.3 and removed v8.3.4 labels Jul 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport >bug :Distributed/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed Meta label for distributed team. v8.3.3

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants