Fix problem with reoccuring files in always incremental#1395
Merged
arogge merged 6 commits intobareos:masterfrom Mar 21, 2023
Merged
Fix problem with reoccuring files in always incremental#1395arogge merged 6 commits intobareos:masterfrom
arogge merged 6 commits intobareos:masterfrom
Conversation
sebsura
requested changes
Mar 10, 2023
systemtests/tests/virtualfull-deletedfiles/testrunner-05-consolidate
Outdated
Show resolved
Hide resolved
sebsura
reviewed
Mar 10, 2023
sebsura
approved these changes
Mar 15, 2023
Introduce a way to express hard dependencies between systemtests. Using DEPENDS on a test only ensures ordering, but does not imply the need to run the other test or that the other test must have finished successfully. Using the new function systemtest_requires() you can now express that a test A will require successful execution of test B by simply calling systemtest_requires(A B). As this is tailored to be used inside a single directory, the required prefixes are added automatically. The function also honors existing fixtures. Under the hood this is implemented by adding a setup fixture to every systemtest that is then required by the depending test.
This test will reproduces a problem where consolidating multiple incrementals into one, will lose the information about deleted files.
Replicate the deleted file information using a simple SQL query. While this works the deleted files will not be added to the job data stored on the volume.
d7013a7 to
0c6349b
Compare
When reloading the configuration failed, the working_directory would not be reset correctly which made it point to the freed string of the new configuration. Now SetWorkingDirectory() is called after reloading in case of success and failure. This also allows to run all of the reload systemtests under a sanitizer which failed previously because of the use-after-free.
252bfa6 to
91f35fa
Compare
an allocated ua context was not released.
sebsura
approved these changes
Mar 15, 2023
Contributor
sebsura
left a comment
There was a problem hiding this comment.
Looks good to me. We can merge if jenkins gives its OK!
This was referenced Mar 21, 2023
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When utilizing always incremental backup scheme while using
Always Incremental Max Full Age, Bareos will run backups with level VirtualFull to merge existing Incrementals into a new Incremental.As VirtualFull was intended to create new Full backups, the resulting Job will not contain removed files. As a result files that were marked as deleted in the Incremental jobs that were merged will reoccur when asking for a restore.
During the next Incremental job, these files will then be marked as deleted again.
In this pull request we add a test to reproduce the issue and a simple workaround that will replicate the deleted file information in the catalog.
To actually fix the underlying problem, we would have to change how deleted files are represented in the catalog, so we can add their file-indexes to the BSR that is used when creating a virtual full.
As doing this in a backwards-compatible manner is not trivial, it has been postponed for now.
Please check
If you have any questions or problems, please give a comment in the PR.
Helpful documentation and best practices
Checklist for the reviewer of the PR (will be processed by the Bareos team)
Make sure you check/merge the PR using
devtools/pr-toolto have some simple automated checks run and a proper changelog record added.General
Source code quality
Tests