Libcloud Accurate File Backup#1903
Conversation
sduehr
left a comment
There was a problem hiding this comment.
Thanks a lot for your contribution.
We hope you noticed that this plugin never left the experimental state and as it has serious issues with Python version > 3.7, for details see
https://bugs.bareos.org/view.php?id=1422
This means that is doesn't work on all the current Linux distributions.
So we must probably decide to remove it for the upcoming Bareos 24.
However, I tested your change and it worked fine. So we're still going to merge it.
I had to add a commit to set the correct copyright year in the script headers to comply with our code checking.
Please add your Name the the AUTHORS file, or if you don't want that, please notify us here.
|
Understood thanks for the view, I wasn't aware of the issue with python 3.9 and greater, that is a bit of a challenge pity there isn't another way around it. The backup of a cloud bucket is still very valuable in my opinion. Authors has been updated Brian |
sduehr
left a comment
There was a problem hiding this comment.
Thanks a lot, looks good.
We will probably have to redesign the whole architecture of the Python plugins, as there are also other issues with newer Python versions.
For backing up cloud storage, we are aiming a more flexible approach, and probably using OpenDAL as it's made for data access, so it should be also possible to backup other cloud storage than S3. However, this needs funding or contributions, so it will take time.
027dfbb to
cf0bbb7
Compare
b1687d1 to
66a3a37
Compare
Libcloud Accurate File Backup
Thank you for contributing to the Bareos Project!
Currently with libcloud with accurate backup mode on all files are added to the backup queue including unchanged files. This causes the unchanged files to be downloaded or a stream resource to be created prior to the start_backup_file method being called. It appears in an accurate only situation the resources are available but never used as no IO is ever called when there's no changes. This change creates a 4th file type called "ACCURATE" which is effectively a dummy record with no backing stream or temp file. It simply passes through the accurate information and is removed from the queue. This fixes issues where in an accurate incremental backup the temp space is filled with accurate only files which are never cleaned up due to the file never being opened for IO.
Open to other suggestions or solutions
Please check
If you have any questions or problems, please give a comment in the PR.
Helpful documentation and best practices
Checklist for the reviewer of the PR (will be processed by the Bareos team)
Make sure you check/merge the PR using
devtools/pr-toolto have some simple automated checks run and a proper changelog record added.General
Source code quality