Skip to content

Improve handling of temporary files on windows#3610

Merged
fd0 merged 3 commits intorestic:masterfrom
MichaelEischer:windows-temp-files
Apr 10, 2022
Merged

Improve handling of temporary files on windows#3610
fd0 merged 3 commits intorestic:masterfrom
MichaelEischer:windows-temp-files

Conversation

@MichaelEischer
Copy link
Copy Markdown
Member

@MichaelEischer MichaelEischer commented Dec 29, 2021

What does this PR change? What problem does it solve?

restic currently uses ioutil.TempFile(...) to create its temporary files on Windows. As files cannot be deleted on Windows while these are still opened, restic sometimes leaves temporary files behind. This PR sets the FILE_FLAG_DELETE_ON_CLOSE flag to let windows delete the file automatically once all references were deleted. In addition, the flag FILE_ATTRIBUTE_TEMPORARY is added to avoid unnecessary writes of the temporary files to disk.

As os.Open and therefore ioutil.TempFile do not support these flags, this PR introduces a custom TempFile implementation, that directly calls syscall.CreateFile.

This PR is based on the code from @ducalex in #3465.

The temporary file is also created using shareMode 0, which prevents other processes from accessing the file in parallel. Thus restic is the only process which can open the file. As it will be deleted on close, this ensures that a tempfile can be removed, which fixes #1551.

Was the change previously discussed in an issue or on the forum?

Closes #3465.
Closes #1551.
Supersedes #3306.

Checklist

  • I have read the contribution guidelines.
  • I have enabled maintainer edits.
  • I have added tests for all code changes.
  • [ ] I have added documentation for relevant changes (in the manual).
  • There's a new file in changelog/unreleased/ that describes the changes for our users (see template).
  • I have run gofmt on the code in all commits.
  • All commit messages are formatted in the same style as the other commits in the repo.
  • I'm done! This pull request is ready for review.

@MichaelEischer MichaelEischer force-pushed the windows-temp-files branch 2 times, most recently from 89d5cf9 to 2a73af6 Compare December 29, 2021 22:33
@MichaelEischer MichaelEischer marked this pull request as draft December 29, 2021 22:45
@MichaelEischer MichaelEischer marked this pull request as ready for review February 10, 2022 20:29
@greatroar
Copy link
Copy Markdown
Contributor

FILE_ATTRIBUTE_TEMPORARY and FILE_FLAG_DELETE_ON_CLOSE are both available in golang.org/x/sys/windows. That also offers everything now imported from syscall.

Copy link
Copy Markdown
Member

@fd0 fd0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@fd0 fd0 merged commit 89ee1cf into restic:master Apr 10, 2022
@MichaelEischer MichaelEischer deleted the windows-temp-files branch April 10, 2022 19:17
@MichaelEischer MichaelEischer mentioned this pull request Aug 8, 2022
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve temporary files handling on Windows Warn and continue when tempfiles cannot be removed

4 participants