VMware Plugin: Backup and Restore of VMs using multiple datastores by sduehr · Pull Request #1473 · bareos/bareos

sduehr · 2023-05-17T22:16:24Z

Thank you for contributing to the Bareos Project!

Checklist for the reviewer of the PR (will be processed by the Bareos team)

Make sure you check/merge the PR using devtools/pr-tool to have some simple automated checks run and a proper changelog record added.

General

Is the PR title usable as CHANGELOG entry?
Purpose of the PR is understood
Commit descriptions are understandable and well formatted
Check backport line
Required backport PRs have been created

Source code quality

Source code changes are understandable
Variable and function names are meaningful
Code comments are correct (logically and spelling)
Required documentation changes are present and part of the PR

pstorz

Good work! Please see comments and change suggestions.

pstorz · 2023-05-30T15:02:54Z

core/src/plugins/filed/python/vmware/BareosFdPluginVMware.py

+            else:
+                # if vcthumbprint is not given in options, retrieve it
+                if not self.vadp.retrieve_vcthumbprint():
+                    return bareosfd.bRC_Error


Is the automatically retrieved thumbprint of interest? Would it make sense to also put it into the joblog?

pstorz · 2023-05-30T15:04:26Z

core/src/plugins/filed/python/vmware/BareosFdPluginVMware.py

        self.vm = create_vm_task.info.result

+        # If transformer.disk_device_change_delayed is not empty, there are disks in other
+        # datastores that must be added one-by-one here after VM was created.


Suggested change

# datastores that must be added one-by-one here after VM was created.

# datastores which must be added one-by-one here after VM was created.

pstorz · 2023-05-30T15:08:15Z

core/src/plugins/filed/python/vmware/BareosFdPluginVMware.py

+                bareosfd.JobMessage(
+                    bareosfd.M_FATAL,
+                    "Disk %s not found in previous backup, migrating disks is not yet supported for "
+                    "incremental or differential. A new full level backup of this job is required.\n"


Suggested change

"incremental or differential. A new full level backup of this job is required.\n"

"incremental or differential backups. A new full level backup of this job is required.\n"

pstorz · 2023-05-30T15:10:39Z

core/src/plugins/filed/python/vmware/BareosFdPluginVMware.py

+        # which is fixed now. So now try to transform the VM metadata as it
+        # would be done when recreating the VM for restore. Note that a job
+        # message with level M_ERROR will cause the backup job to terminate
+        # as Backup OK -- with warnings


Suggested change

# as Backup OK -- with warnings

# as "Backup OK -- with warnings"

pstorz · 2023-05-30T15:23:20Z

core/src/plugins/filed/python/vmware/BareosFdPluginVMware.py

+            return backing_ds_match.group(1)
+        else:
+            raise RuntimeError(
+                "Error getting datastore name from backing path: %s" % (backing_path)


Suggested change

"Error getting datastore name from backing path: %s" % (backing_path)

"Error getting datastore name from backing path %s" % (backing_path)

pstorz · 2023-05-30T15:23:46Z

core/src/plugins/filed/python/vmware/BareosFdPluginVMware.py

                )
-                add_device = self._transform_virtual_disk(device)
+                # As _transform_virtual_disk() will only change the backing datastore
+                # for disks which were in same datastore than VM, The backing datastore


Suggested change

# for disks which were in same datastore than VM, The backing datastore

# for disks which were in same datastore as the VM, The backing datastore

pstorz · 2023-05-30T15:25:06Z

core/src/plugins/filed/python/vmware/BareosFdPluginVMware.py

+                orig_disk_backing_path
+            )
+
+            # When datastore is not changed, restore disk path will be the same as backed up


Suggested change

# When datastore is not changed, restore disk path will be the same as backed up

# When datastore is not changed, restore disk path will be the same as backed up disk path

pstorz · 2023-05-30T15:31:07Z

docs/manuals/source/TasksAndConcepts/Plugins/FileDaemonPlugins/VMwarePlugin.rst.inc

                ":vcthumbprint=AABBCCDDEEFF11223344556677889900AABBCCDD"
       ...

+If the **vcthumbprint** option is used and the thumbprint on the server changes, for example by renewing or replacing the SSL certificate but not adapting the vcthumbprint parameter in the Bareos configuration, backup jobs will fail and the API will only return an "unknown" error. Since :sinceVersion:`22.1.0: VMware Plugin:` the plugin will compare the configured with the server thumbprint and emit an appropriate error message and advice to update vcthumbprint parameter.


Suggested change

If the **vcthumbprint** option is used and the thumbprint on the server changes, for example by renewing or replacing the SSL certificate but not adapting the vcthumbprint parameter in the Bareos configuration, backup jobs will fail and the API will only return an "unknown" error. Since :sinceVersion:`22.1.0: VMware Plugin:` the plugin will compare the configured with the server thumbprint and emit an appropriate error message and advice to update vcthumbprint parameter.

If the **vcthumbprint** option is used and the thumbprint on the server changes, for example by renewing or replacing the SSL certificate but not adapting the vcthumbprint parameter in the Bareos configuration, backup jobs will fail and the API will only return an "unknown" error. Since :sinceVersion:`22.1.0: VMware Plugin:` the plugin will compare the configured thumbprint with the server thumbprint and emit an appropriate error message and advice to update vcthumbprint parameter.

pstorz · 2023-05-30T15:32:16Z

docs/manuals/source/TasksAndConcepts/Plugins/FileDaemonPlugins/VMwarePlugin.rst.inc


 restore_datastore (optional)
-   By default, if a VM to be restored does not exist, it will be recreated in the same datastore where it was stored at backup time. Use this option to restore on the given datastore. Since :sinceVersion:`22.0.0: VMware Plugin`
+   By default, if a VM to be restored does not exist, it will be recreated in the same datastore where it was stored at backup time. Use this option to restore on the given datastore. Since :sinceVersion:`22.0.0: VMware Plugin`. As :sinceVersion:`22.1.0: VMware Plugin` it is possible to backup and restore VMs with disks on multiple datastores, when using this option, it will only change the datastore of the disks which were stored in the same datastore than the VM, the other disks will be recreated on the same datastore they were backed up from.


Suggested change

By default, if a VM to be restored does not exist, it will be recreated in the same datastore where it was stored at backup time. Use this option to restore on the given datastore. Since :sinceVersion:`22.0.0: VMware Plugin`. As :sinceVersion:`22.1.0: VMware Plugin` it is possible to backup and restore VMs with disks on multiple datastores, when using this option, it will only change the datastore of the disks which were stored in the same datastore than the VM, the other disks will be recreated on the same datastore they were backed up from.

By default, if a VM to be restored does not exist, it will be recreated in the same datastore where it was stored at backup time. Use this option to restore on the given datastore. Since :sinceVersion:`22.0.0: VMware Plugin`. As :sinceVersion:`22.1.0: VMware Plugin` it is possible to backup and restore VMs with disks on multiple datastores, when using this option, it will only change the datastore of the disks which were stored in the same datastore as the VM, the other disks will be recreated on the same datastore they were backed up from.

pstorz · 2023-05-30T15:35:04Z

docs/manuals/source/TasksAndConcepts/Plugins/FileDaemonPlugins/VMwarePlugin.rst.inc

   By **default**, after restore a VM will be set to its **previous powerstate** which means the powerstate at backup time. When specifying ``restore_powerstate=off`` the VM will stay powered off after restore. Also can be forced to on with ``restore_powerstate=on``. Note that this will only work if DRS is configured to **fully automated**, otherwise the API request to power on a VM will be ignored. Since :sinceVersion:`22.0.0: VMware Plugin`

+snapshot_retries (optional)
+   Number of retries when taking a snapshot fails (default: 3). The most common cause of snapshot failure is error while quiescing the virtual machine. Mostly retrying helps. If not also check if a pre-freeze script is used on the VM, a non-zero exit code will cause a quiescing error. The pre-freeze and post-thaw scripts are executed by VMwareTools. Since :sinceVersion:`22.1.0: VMware Plugin`


Suggested change

Number of retries when taking a snapshot fails (default: 3). The most common cause of snapshot failure is error while quiescing the virtual machine. Mostly retrying helps. If not also check if a pre-freeze script is used on the VM, a non-zero exit code will cause a quiescing error. The pre-freeze and post-thaw scripts are executed by VMwareTools. Since :sinceVersion:`22.1.0: VMware Plugin`

Number of retries when taking a snapshot fails (default: 3). The most common cause of snapshot failure is "error while quiescing the virtual machine". In this case usually retrying helps. If not, check if a pre-freeze script is used on the VM, as a non-zero exit code will cause a quiescing error. The pre-freeze and post-thaw scripts are executed by VMwareTools. Since :sinceVersion:`22.1.0: VMware Plugin`

pstorz

Good Work!

(cherry picked from commit c89a58c)

The plugin can now backup and restore VMs which use multiple disks on different datastores. Other enhancements: Configurable snapshot retries to mitigate quiescing errors, detection of server thumbprint changes and appropriate warning when a fallback to full CBT occurs.

When a snapshot existed at backup time, properly adapt the disk path on restore.

(cherry picked from commit c89a58c)

pstorz self-assigned this May 30, 2023

pstorz requested changes May 30, 2023

View reviewed changes

sduehr added a commit that referenced this pull request May 31, 2023

VMware Plugin: Changes requested in PR #1473

c89a58c

sduehr requested a review from pstorz May 31, 2023 16:13

pstorz approved these changes Jun 1, 2023

View reviewed changes

pstorz added the requires backport to 22 label Jun 1, 2023

pstorz pushed a commit that referenced this pull request Jun 1, 2023

VMware Plugin: Changes requested in PR #1473

3f9249b

(cherry picked from commit c89a58c)

pstorz mentioned this pull request Jun 1, 2023

VMware Plugin: Backup and Restore of VMs using multiple datastores #1478

Merged

8 tasks

sduehr and others added 5 commits June 7, 2023 12:16

docs: updated VMware plugin documentation

69e53f7

VMware Plugin: fix restore when snapshot existed

03e755c

When a snapshot existed at backup time, properly adapt the disk path on restore.

VMware Plugin: Changes requested in PR #1473

cc6e213

Update CHANGELOG.md

e806212

pstorz force-pushed the dev/sduehr/master/vmware-restore-multidatastore branch from faee1a2 to e806212 Compare June 7, 2023 10:16

pstorz merged commit f1e3801 into master Jun 7, 2023

pstorz deleted the dev/sduehr/master/vmware-restore-multidatastore branch June 7, 2023 10:16

pstorz pushed a commit that referenced this pull request Jun 7, 2023

VMware Plugin: Changes requested in PR #1473

20ab10e

(cherry picked from commit c89a58c)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VMware Plugin: Backup and Restore of VMs using multiple datastores#1473

VMware Plugin: Backup and Restore of VMs using multiple datastores#1473
pstorz merged 5 commits intomasterfrom
dev/sduehr/master/vmware-restore-multidatastore

sduehr commented May 17, 2023 •

edited by pstorz

Loading

Uh oh!

pstorz left a comment

Uh oh!

pstorz May 30, 2023

Uh oh!

pstorz May 30, 2023

Uh oh!

pstorz May 30, 2023

Uh oh!

pstorz May 30, 2023

Uh oh!

pstorz May 30, 2023

Uh oh!

pstorz May 30, 2023

Uh oh!

pstorz May 30, 2023

Uh oh!

pstorz May 30, 2023

Uh oh!

pstorz May 30, 2023

Uh oh!

pstorz May 30, 2023

Uh oh!

pstorz left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	# datastores that must be added one-by-one here after VM was created.
	# datastores which must be added one-by-one here after VM was created.

	"incremental or differential. A new full level backup of this job is required.\n"
	"incremental or differential backups. A new full level backup of this job is required.\n"

	# as Backup OK -- with warnings
	# as "Backup OK -- with warnings"

	"Error getting datastore name from backing path: %s" % (backing_path)
	"Error getting datastore name from backing path %s" % (backing_path)

	# for disks which were in same datastore than VM, The backing datastore
	# for disks which were in same datastore as the VM, The backing datastore

	# When datastore is not changed, restore disk path will be the same as backed up
	# When datastore is not changed, restore disk path will be the same as backed up disk path

Conversation

sduehr commented May 17, 2023 • edited by pstorz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Thank you for contributing to the Bareos Project!

Checklist for the reviewer of the PR (will be processed by the Bareos team)

General

Source code quality

Uh oh!

pstorz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pstorz left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sduehr commented May 17, 2023 •

edited by pstorz

Loading