dedupe: Skip extents with insufficient duplicates#365
Merged
JackSlateur merged 1 commit intomarkfasheh:masterfrom Feb 7, 2025
Merged
dedupe: Skip extents with insufficient duplicates#365JackSlateur merged 1 commit intomarkfasheh:masterfrom
JackSlateur merged 1 commit intomarkfasheh:masterfrom
Conversation
Contributor
Author
|
A second crash for reference. |
Contributor
Author
|
Oh... that might be #341 |
Add safety check in push_extents() to prevent queuing work items with less than 2 duplicates. This prevents assertion failures in dedupe_extent_list() when the duplicate count drops below 2 between the initial duplicate detection and worker thread processing. Fixes: markfasheh#341
Collaborator
|
Thank you for your contribution |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
De-duping my Unity Avatar's library on bcachefs always resulted in aborts of this type:
I had to add a safety check in push_extents() to prevent those crashes from randomly happening.
This prevents assertion failures in dedupe_extent_list() when the duplicate count drops below 2 between the initial duplicate detection and worker thread processing. Even when reducing I/O thread count I did encounter this on my setup. Maybe because I have many CPU cores? (Ryzen 9 7950X3D / 32 threads).
I'm not sure I understand correctly why the problem happened in the first place so I'll let that to you to review.
Note: Many of the lines above say the file has changed, but those are archives, nothing should change in those folders.