Fix stream records transfer data race, losing pending updates#8103
Merged
Fix stream records transfer data race, losing pending updates#8103
Conversation
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
ffuugoo
approved these changes
Feb 11, 2026
timvisee
commented
Feb 11, 2026
| /// | ||
| /// Returns oneshot channel receiver that will be notified once the plunger operation is | ||
| /// processed. | ||
| pub async fn plunge_async(&self) -> CollectionResult<oneshot::Receiver<()>> { |
Member
Author
There was a problem hiding this comment.
I tried to polish this a bit more - abstracting away the receiver by just returning a future that needs to be resolved.
But it turns out to be quite the pain with life times further down the line. So I skipped this effort for now.
agourlay
approved these changes
Feb 11, 2026
9 tasks
timvisee
added a commit
that referenced
this pull request
Feb 13, 2026
* Add integration test to assert all queued updates are also transferred * Plunge update queue in stream records transfers * Migrate existing plunger usages to new plunge helper * Skip test if not compiled with staging flag * Only send delay operation when staging feature is enabled * Reformat * Fix review remarks
This was referenced Mar 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A steam record transfer did not include pending items in the update queue (channel). Such pending operations were effectively lost. This problem seems to have existed for a very long time.
A specific scenario is required to make the problem appear. You must have pending operations in the queue. And such update must remain in the queue until after the transfer has sent its respective points batch. It takes time to bootstrap a transfer and Qdrant <1.17.0 limited the queue to 200 operations, making it not very likely to happen.
Since allowing 1 million queued items in Qdrant 1.17 (#8046) the problem suddenly does become much more visible.
This PR fixes the problem and adds a test to assert correct behavior.
Tasks
All Submissions:
devbranch. Did you create your branch fromdev?Changes to Core Features: