Skip to content

VMware Plugin: Fix CBT query handling#2152

Merged
BareosBot merged 6 commits intobareos:masterfrom
sduehr:dev/sduehr/master/fix-vmware-cbt-query
Feb 20, 2025
Merged

VMware Plugin: Fix CBT query handling#2152
BareosBot merged 6 commits intobareos:masterfrom
sduehr:dev/sduehr/master/fix-vmware-cbt-query

Conversation

@sduehr
Copy link
Member

@sduehr sduehr commented Feb 5, 2025

The CBT query API call QueryChangedDiskAreas must be used repeatedly to get the complete change information. Additionally, for full backups the changeId * should not be used, the plugin now sends the whole capacity disk range so that bareos_vadp_dumper will effectively only use the allocated blocks information in this case. Also successful API connects and reconnects are now reported as informational job message for more clarity.

Thank you for contributing to the Bareos Project!

Please check

  • Short description and the purpose of this PR is present above this paragraph
  • Your name is present in the AUTHORS file (optional)

If you have any questions or problems, please give a comment in the PR.

Helpful documentation and best practices

Checklist for the reviewer of the PR (will be processed by the Bareos team)

Make sure you check/merge the PR using devtools/pr-tool to have some simple automated checks run and a proper changelog record added.

General
  • Is the PR title usable as CHANGELOG entry?
  • Purpose of the PR is understood
  • Commit descriptions are understandable and well formatted
  • Required backport PRs have been created
  • Correct milestone is set
Source code quality
  • Source code changes are understandable
  • Variable and function names are meaningful
  • Code comments are correct (logically and spelling)
  • Required documentation changes are present and part of the PR

Copy link
Contributor

@sebsura sebsura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-sdks-tools/7-0/virtual-disk-development-kit-programming-guide/backing-up-virtual-disks-in-vsphere/low-level-backup-procedures/changed-block-tracking-on-virtual-disks/gathering-changed-block-information.html

This also contains the following section:

The changeId
(changed block ID) contains a sequence number in the form <UUID>/<nnn>
. If <UUID>
changes, it indicates that tracking information has become invalid, necessitating a full backup. Otherwise incremental backups can continue in the usual pattern.
  • Do you know if pyvmomi already takes care of this somehow? (I.e. throwing an error)
  • Should we take this into account somewhere ? Maybe throwing an error if the changeids are not compatible ?

If we decide not to handle this for now, we should probably at least write a comment for it in the code so that we may remember it in the future.

sduehr added a commit to sduehr/bareos that referenced this pull request Feb 13, 2025
Also skip saving the CBT list in restoreobjects, as that is not
necessary and could cause problems if the size of the CBT list is
huge.
@sduehr sduehr requested a review from sebsura February 13, 2025 20:34
@sduehr
Copy link
Member Author

sduehr commented Feb 17, 2025

https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-sdks-tools/7-0/virtual-disk-development-kit-programming-guide/backing-up-virtual-disks-in-vsphere/low-level-backup-procedures/changed-block-tracking-on-virtual-disks/gathering-changed-block-information.html

This also contains the following section:

The changeId
(changed block ID) contains a sequence number in the form <UUID>/<nnn>
. If <UUID>
changes, it indicates that tracking information has become invalid, necessitating a full backup. Otherwise incremental backups can continue in the usual pattern.
  • Do you know if pyvmomi already takes care of this somehow? (I.e. throwing an error)
  • Should we take this into account somewhere ? Maybe throwing an error if the changeids are not compatible ?

If we decide not to handle this for now, we should probably at least write a comment for it in the code so that we may remember it in the future.

I checked pyvmomi, I couldn't find any code which check this. pyvmomi is mostly automatically generated code without special handling, it just returns what it gets from the API, including error handling. There are situations where QueryChangedDiskAreas throws a vim.fault.FileFault exception, which we already handle in the plugin.

@sduehr
Copy link
Member Author

sduehr commented Feb 19, 2025

https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-sdks-tools/7-0/virtual-disk-development-kit-programming-guide/backing-up-virtual-disks-in-vsphere/low-level-backup-procedures/changed-block-tracking-on-virtual-disks/gathering-changed-block-information.html
This also contains the following section:

The changeId
(changed block ID) contains a sequence number in the form <UUID>/<nnn>
. If <UUID>
changes, it indicates that tracking information has become invalid, necessitating a full backup. Otherwise incremental backups can continue in the usual pattern.
  • Do you know if pyvmomi already takes care of this somehow? (I.e. throwing an error)
  • Should we take this into account somewhere ? Maybe throwing an error if the changeids are not compatible ?

If we decide not to handle this for now, we should probably at least write a comment for it in the code so that we may remember it in the future.

I checked pyvmomi, I couldn't find any code which check this. pyvmomi is mostly automatically generated code without special handling, it just returns what it gets from the API, including error handling. There are situations where QueryChangedDiskAreas throws a vim.fault.FileFault exception, which we already handle in the plugin.

The plugin now compares the UUID parts of the changeIds from restoreobject and current snapshot. If a mismatch is detected, the job will fail. The error message indicates that a new full level job run is required.

@sduehr sduehr requested a review from sebsura February 19, 2025 00:01
Copy link
Contributor

@sebsura sebsura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the good work

sebsura pushed a commit to sebsura/bareos that referenced this pull request Feb 19, 2025
Also skip saving the CBT list in restoreobjects, as that is not
necessary and could cause problems if the size of the CBT list is
huge.

(cherry picked from commit faeba39)
sebsura pushed a commit to sebsura/bareos that referenced this pull request Feb 19, 2025
Also skip saving the CBT list in restoreobjects, as that is not
necessary and could cause problems if the size of the CBT list is
huge.

(cherry picked from commit faeba39)
sduehr and others added 6 commits February 20, 2025 01:08
The CBT query API call QueryChangedDiskAreas must be used repeatedly
to get the complete change information. Additionally, for full backups
the changeId * should not be used, the plugin now sends the whole
capacity disk range so that bareos_vadp_dumper will effectively only use
the allocated blocks information in this case. Also successful API
connects and reconnects are now reported as informational job message
for more clarity.
Using the grpc plugin, messages which go to stderr cause warnings,
so the urllib3 SSL warning will be suppressed when verifyssl=no is set.
Otherwise backup/restor jobs would unnessarily terminate with warning
or error although it worked as expected.
Also skip saving the CBT list in restoreobjects, as that is not
necessary and could cause problems if the size of the CBT list is
huge.
The UUID part of the changeId from the restoreobject is now compared
with the UUID part of the changeId from the current snapshot. A mismatch
indicates that a new full level job run is required, so in that case the
job will fail. The optional script reschedule_job_as_full.sh can be used
to trigger new full jobs in that case automatically, as that is
currently not possible from the plugin itself.
@BareosBot BareosBot force-pushed the dev/sduehr/master/fix-vmware-cbt-query branch from ba585b0 to 936c87d Compare February 20, 2025 01:08
@BareosBot BareosBot merged commit 5d92b12 into bareos:master Feb 20, 2025
BareosBot pushed a commit to sebsura/bareos that referenced this pull request Feb 20, 2025
Also skip saving the CBT list in restoreobjects, as that is not
necessary and could cause problems if the size of the CBT list is
huge.

(cherry picked from commit faeba39)
BareosBot pushed a commit to sebsura/bareos that referenced this pull request Feb 20, 2025
Also skip saving the CBT list in restoreobjects, as that is not
necessary and could cause problems if the size of the CBT list is
huge.

(cherry picked from commit faeba39)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows® VMs cannot be restored to VMware VMware: Add success message after reconnect VMware Plugin Backups corrupted on Backups >2TB or >2hr

3 participants