vinyl: skip invisible read sources#10851
Merged
locker merged 1 commit intotarantool:masterfrom Nov 27, 2024
Merged
Conversation
lenkis
approved these changes
Nov 25, 2024
changelogs/unreleased/gh-10846-fix-vinyl-replica-join-degradation-under-write-load.md
Outdated
Show resolved
Hide resolved
A Vinyl read iterator scans all read sources (memory and disk levels) even if it's executed in a read view from which most of the sources are invisible. As a result, a long running scanning request may spend most of the time skipping invisible statements. The situation is exacerbated if the instance is experiencing a heavy write load because it would pile up old statement versions in memory and force the iterator to skip over them after each disk read. Since the replica join procedure in Vinyl uses a read view iterator under the hood, the issue is responsible for a severe performance degradation of the master instance and the overall join procedure slowdown when a new replica is joined to an instance running under a heavy write load. Let's fix this issue by making a read iterator skip read sources that aren't visible from its read view. Closes tarantool#10846 NO_DOC=bug fix
487192c to
9c4bc38
Compare
Member
Author
|
Cherry-picked to 2.11 and 3.2. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A Vinyl read iterator scans all read sources (memory and disk levels) even if it's executed in a read view from which most of the sources are invisible. As a result, a long running scanning request may spend most of the time skipping invisible statements. The situation is exacerbated if the instance is experiencing a heavy write load because it would pile up old statement versions in memory and force the iterator to skip over them after each disk read.
Since the replica join procedure in Vinyl uses a read view iterator under the hood, the issue is responsible for a severe performance degradation of the master instance and the overall join procedure slowdown when a new replica is joined to an instance running under a heavy write load.
Let's fix this issue by making a read iterator skip read sources that aren't visible from its read view.
Closes #10846