Output of restic version

restic 0.12.0-dev (compiled manually) compiled with go1.15.2 on windows/amd64

What should restic do differently? Which functionality do you think we should add?

I would like to reduce the amount of disk writes when restic uses temporary files and prevent those files from being occasionally left behind.

What are you trying to do? What problem would this solve?

Ideally I would have liked to replace temporary files by in-memory buffers, which would have been a more elegant solution, but from this thread it is unclear if it would be accepted. The solution on Linux might be to use a ramdisk, but on Windows it's not as easy.

My solution is to add a few flags (also mentioned in the aforementioned thread) when creating a temporary file:

Specifying the FILE_ATTRIBUTE_TEMPORARY attribute causes file systems to avoid writing data back to mass storage if sufficient cache memory is available, because an application deletes a temporary file after a handle is closed. In that case, the system can entirely avoid writing the data.

I have already implemented a working proof of concept. I am opening this issue as per the contribution guidelines to see if there is interest before polishing and creating the pull request. The change would consist of re-implementing TempFile() in a lower level fashion instead of using ioutil. This isn't ideal, but I don't see how else I could get the flags through.

Example:

func TempFile(dir, prefix string) (f *os.File, err error) {
	if dir == "" {
		dir = os.TempDir()
	}

	access := uint32(syscall.GENERIC_READ | syscall.GENERIC_WRITE)
	creation := uint32(syscall.CREATE_NEW)
	flags := uint32(FILE_ATTRIBUTE_TEMPORARY | FILE_FLAG_DELETE_ON_CLOSE)
	
	for i := 0; i < 10000; i++ {
		path := filepath.Join(dir, prefix+nextRandom())

		h, err := syscall.CreateFile(syscall.StringToUTF16Ptr(path), access, 0, nil, creation, flags, 0)
		if err == nil {
			return os.NewFile(uintptr(h), path), nil
		}
	}
	
	// Proper error handling is still to do
	return nil, os.ErrExist
}

Did restic help you today? Did it make you happy in any way?

Reading the discussions here makes me happy to see a welcoming community!