stored: fix incoherent meta data when concurrently writing to the same volume#1495
Merged
BareosBot merged 19 commits intobareos:masterfrom Aug 1, 2023
Merged
Conversation
190ef05 to
956ee25
Compare
alaaeddineelamri
suggested changes
Jul 7, 2023
Contributor
alaaeddineelamri
left a comment
There was a problem hiding this comment.
you can make the PR title a bit more explicit to what is being fixed.
maybe fix parallel jobs incoherent meta data when concurrently writing to the same volume for example?
alaaeddineelamri
approved these changes
Jul 24, 2023
4709a9d to
7f92559
Compare
14 tasks
This does not belong here. This causes big problems. The device might have changed volumes etc. and the values are not related to anything anymore. The dcr gets updated after every write so this is unneccessary as well.
Sometimes we still have some meta data left over after completing a job. This ensures that it gets send to the director.
Since our volumes only can contain 70KiB of data, we can never write even a single 1MiB block to it.
544c88e to
a9a0351
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for contributing to the Bareos Project!
Currently letting multiple jobs write to a device in parallel will cause the bareos database to go out of sync with reality leading to data loss at worst. This PR improves the communication between the sd and the director so that this does not happen.
We do this by
This should stop the director from recycling actually used volumes and stop the storage daemon from not sending all jobmedia records.
We also slim down the jobmedia table somewhat by always deleting all null jobmedia records whenever a job is finished.
Please check
If you have any questions or problems, please give a comment in the PR.
Helpful documentation and best practices
Checklist for the reviewer of the PR (will be processed by the Bareos team)
Make sure you check/merge the PR using
devtools/pr-toolto have some simple automated checks run and a proper changelog record added.General
Source code quality
Tests