Skip to content

Add options to fine tune VSS snapshots#3067

Merged
MichaelEischer merged 16 commits intorestic:masterfrom
DRON-666:vss-options
Apr 29, 2024
Merged

Add options to fine tune VSS snapshots#3067
MichaelEischer merged 16 commits intorestic:masterfrom
DRON-666:vss-options

Conversation

@DRON-666
Copy link
Copy Markdown
Contributor

@DRON-666 DRON-666 commented Nov 6, 2020

What does this PR change? What problem does it solve?

Original VSS PR use some not-so-universal defaults.

First of all, current Restic architecture don't allow listing all mount points required for backup and we have to enumerate mount points for a given volume and create VSS snapshots for all of them. This can lead to problems if some of this unnecessary mount points links to locked, filled or otherwise unsupported volumes. @fgma has made every effort to avoid this problems, but it's not feasible in all cases. On the other hand, Restic don't follow mount points: if you backup C:\, mount point C:\mnt will be saved as link /c/mnt -> d:\. So snapshotting mount points may only make sense for paths like C:\mnt\somedir.

Extended option -o vss.excludeallmountpoints solves this problem by enterely disabling mount points snapshotting.

Another problem can occurred with backups from multiple volumes (e.g. I backup files with some extensions from all drives), but VSS required only on certain volumes. Why not take a snapshot of them all: because snapshotting of large volumes can take several minutes.

Extended option -o vss.excludevolumes allows you to fine-tune VSS snapshots by excluding specific volumes. Mount points to this volumes are also excluded.

Thus, volumes can be specified in three ways: as drive letter D:\, as mount point C:\mnt, or as volume GUID path \\?\Volume{b151601a-34a4-11e0-b644-806e6f6e6963}\. The trailing slash can always be omitted F:. Before comparison, all specified volumes are converted (with reporting in case of errors) to the form of a volume GUID path, so if C:\mnt links to D:\ then -o vss.excludevolumes=d: and -o vss.excludevolumes=c:\mnt will give the same results.

Also, default VSS provider may lead to some problems. With the option -o vss.provider, vss provider can be specified in three ways: GUID, name and MS as alias for {b5946137-7b9f-4925-af80-51abd60b20d5}.

And the last potential problem is timeout: in my case, it takes more than a minute to take the longest snapshot. It's too close to the default timeout of 120 seconds.

The extended option -o vss.timeout option allows you to change this timeout. My implementation internally uses deadline concept (idiomatic for Go) instead of timeouts.

Was the change discussed in an issue or in the forum before?

I mention some of this issues in the VSS Support PR
Also see timeout issue, provider issue #1 and provider issue #2.

Checklist

  • I have read the Contribution Guidelines
  • I have enabled maintainer edits for this PR
  • I have added tests for all changes in this PR
  • I have added documentation for the changes (in the manual)
  • There's a new file in changelog/unreleased/ that describes the changes for our users (template here)
  • I have run gofmt on the code in all commits
  • All commit messages are formatted in the same style as the other commits in the repo
  • I'm done, this Pull Request is ready for review

@DRON-666
Copy link
Copy Markdown
Contributor Author

Rebased. Fixed some error checking/ignoring.

@jdiercks
Copy link
Copy Markdown

@DRON-666 : Is it too late to ask you to consider one more option to add in this PR?

-o vss.provider="{GUID}"

Backstory: https://forum.restic.net/t/windows-shadow-copy-snapshot-vss-unexpected-provider-error/3674

What do you think? I've done a proof-of-concept hack/patch with a hard-coded GUID for the standard MS system VSS provider, and it solved my problem. I could try rolling my own PR (it would be my first; I have no real experience with Go), but it would seem more expedient to piggy-back on yours, if you're amenable to it.

@DRON-666
Copy link
Copy Markdown
Contributor Author

@jdiercks As I expected, the go-ole package is not compatible with all targets.
I don't know how to fix it, but you can compile and use this PR without problems.

@jdiercks
Copy link
Copy Markdown

Tested in my sandbox environment.
Smooth as silk so far.
Hope to roll it into production tomorrow.
Thanks very much!

C:\>restic -r test-repo backup --use-fs-snapshot -o vss.provider=b5946137-7b9f-4925-af80-51abd60b20d5 C:\ N:\ X:\
enter password for repository:
repository 273d5070 opened successfully, password is correct
creating VSS snapshot for [n:\]
use non default vss provider: {B5946137-7B9F-4925-AF80-51ABD60B20D5}
successfully created snapshot for [n:\]
creating VSS snapshot for [x:\]
use non default vss provider: {B5946137-7B9F-4925-AF80-51ABD60B20D5}
successfully created snapshot for [x:\]
creating VSS snapshot for [c:\]
use non default vss provider: {B5946137-7B9F-4925-AF80-51ABD60B20D5}
successfully created snapshot for [c:\]

@fgma
Copy link
Copy Markdown
Contributor

fgma commented Mar 22, 2021

@jdiercks I'd suggest to use the name of the provider instead of its GUID. GUIDs are a somewhat hard to understand for end users.

@DRON-666
Copy link
Copy Markdown
Contributor Author

@fgma Microsoft iSCSI Target VSS Hardware Provider looks no better and, if I remember correctly, can be localized.

@jdiercks
Copy link
Copy Markdown

@fgma I have to agree with @DRON-666 on that one. GUID is more straightforward and reliable for this scenario. If a user is far enough along to be in a situation where explicitly specifying the provider even matters, then they can probably figure out they need to copy and paste a GUID from VSSADMIN LIST PROVIDERS.

@DRON-666
Copy link
Copy Markdown
Contributor Author

@fgma @jdiercks Maybe automatic fallback to Microsoft Software Shadow Copy provider 1.0 can resolve all basic provider issues?

@jdiercks
Copy link
Copy Markdown

@DRON-666 that seems to be what VSSADMIN itself does. I would still leave the option in place to override if needed, but fallback would probably save a few people some confusion and grief.

@DRON-666
Copy link
Copy Markdown
Contributor Author

that seems to be what VSSADMIN itself does.

@jdiercks I want to check this with API monitor, but I need to find system with more than one provider.
Can you try latest commit without specifying vss.provider option?

@bjoe2k4
Copy link
Copy Markdown

bjoe2k4 commented Mar 22, 2021

I'm currently using restic in a powershell script that creates a VSS snapshot in the beginning, then calls restic 3 times to backup the same data to 3 different repos (redundancy) and at the end deletes the snapshot. Is there any way this can be simplified with restics own VSS snapshots, but without creating 3 (different) snapshots on every restic call?

@jdiercks
Copy link
Copy Markdown

Can you try latest commit without specifying vss.provider option?

@DRON-666 Looks like without vss.provider it's still using IID_NULL in all the VSS calls.

@DRON-666
Copy link
Copy Markdown
Contributor Author

@jdiercks @fgma I just checked: vssadmin always uses VSS_SWPRV_ProviderId not IID_NULL and I would suggest to do the same.
@bjoe2k4 No, you can't do this.

@jdiercks
Copy link
Copy Markdown

@DRON-666 I think I'm fine with restic defaulting to VSS_SWPRV_ProviderId as long as there is an option to override and specify a different provider if needed.

There is a slight risk that changing restic's default behavior would break some other user's backup, if they have multiple providers and they were relying on the hierarchical behavior produced by using IID_NULL, but I suspect that's a corner case at worst, and as long as it's documented they can just start using the new option to get back the results they want.

@jdiercks
Copy link
Copy Markdown

@bjoe2k4 can you share your powershell script? Curious to see how you implemented that. Could be useful to me in a different context.

@bjoe2k4
Copy link
Copy Markdown

bjoe2k4 commented Mar 22, 2021

@DRON-666 ok thanks, that would have been too good to be true.
@jdiercks unfortunately it is not really polished, so i'm uncomfortable to share. However, for creating snapshots with powershell, have a look at old versions of https://github.com/kmwoley/restic-windows-backup/releases

@DRON-666
Copy link
Copy Markdown
Contributor Author

@fgma @jdiercks Now you can use vss.provider={guid}, vss.provider="name" or vss.provider=MS (alias for vss.provider={b5946137-7b9f-4925-af80-51abd60b20d5})

@jdiercks
Copy link
Copy Markdown

@DRON-666 Wow, that's a lot more code! But definitely very flexible. Am I reading right that you decided to keep the original default behavior, with restic passing IID_NULL as provider if none specified via options? I'm not seeing anything that looks like it's doing a fallback to ms sys provider.

@DRON-666
Copy link
Copy Markdown
Contributor Author

@jdiercks somehow missed your question, sorry for that.
Yes, I removed the fallback, because in some situations all code can run without errors, but resulting shadow copy just doesn't work, and at this point it's too late for fallback.

@jdiercks
Copy link
Copy Markdown

jdiercks commented Apr 30, 2021

@DRON-666 Thanks for clarifying. Good call.

I've been running nightly backups with this PR for several weeks now, and it seems pretty solid.

@KuestenKeks
Copy link
Copy Markdown

I think the VSS implementation in restic doesn't support setting the VSS backup type so far?

Differentiation between "full" backups and "copy" backups can be important when for example backing up an Exchange Server or an MS SQL Server.

Support for this would be really nice :)

See: https://docs.microsoft.com/en-us/windows/win32/vss/vss-backup-state

@DRON-666
Copy link
Copy Markdown
Contributor Author

DRON-666 commented Jul 9, 2021

@kscheel VSS backup types irrelevant to restic backup/restore and you link confirms this:

It is possible for requesters to implement these backups using information and methods outside of VSS. Only when a requester implements a backup using the VSS API should it be said to have one of the listed backup types.

restic doesn't use VSS for backup, only for reading locked files. The restore operation doesn't use VSS at all.

@KuestenKeks
Copy link
Copy Markdown

@DRON-666 oh sorry, I guess I misunderstood a few things, nevermind then 🙈 Thanks for the explanation!

Copy link
Copy Markdown
Member

@MichaelEischer MichaelEischer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry for not taking a look at this PR for several years 😞 ; it turned out to be easier to review than I've expected it to be. Thanks for keeping the PR up to date all this time!

Most comments are just minor nitpicks. The only more important one is that the option names should use kebab case, similar to other already existing options, that is the name should be vss.exclude-volumes instead of vss.excludevolumes and similar for the other option names.

Copy link
Copy Markdown
Member

@MichaelEischer MichaelEischer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for addressing the feedback so quick!

@MichaelEischer MichaelEischer added this pull request to the merge queue Apr 29, 2024
Merged via the queue into restic:master with commit ccac7c7 Apr 29, 2024
@DRON-666 DRON-666 deleted the vss-options branch April 29, 2024 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

8 participants