-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Race condition in Clear() function in file restic/internal/cache/file.go #4760
Description
Output of restic version
restic 0.16.4 compiled with go1.21.6 on linux/amd64
What backend/service did you use to store the repository?
rest-server version rest-server 0.12.1 compiled with go1.20.5 on linux/amd64
Problem description / Steps to reproduce
When performing backups on each host, I backup per filesystem separately. This is to make it easier to restore based on mount point. So for example, I backup / and /home separately.
I schedule these backups using cron and have them all start at the same time.
Sometimes, I get the following error (but not always):
error clearing pack files in cache: remove /home/<username>/.cache/restic/<hash>/data/1b/1b<hash>: no such file or directoryor
error clearing index files in cache: remove /home/<username>/.cache/restic/<hash>/index/79/79<hash>: no such file or directoryI suspect that there is a race condition when clearing the local cache of packs and indexes that no longer exist in the repository between different restic backup processes from the same user on the same host. The race is occurring in the Clear method for file /internal/cache/file.go at line 152
Expected behavior
If an obsolete index or pack file in the cache is found, and upon attempting to delete it, the operating system reports that it does not exist - this error should be ignored.
Or alternately, local locking semantics are used so that only one restic process is able to clear the cache of obsolete entries at a time.
The latter would be the more 'correct' solution. But admittedly, it is an edge case and ignoring os.ErrNotExist would probably suffice.
Actual behavior
A race condition can occur where multiple restic processes are building a list of obsolete entries, and then attempting to delete them at the same time, resulting in the above unpredictable outcomes. The process that encounters a missing entry immediately stops processing its list of files, and if the list of files is ordered, would mean that ultimately, the files would get removed. However, the error that results reports a harmless happenstance, and is unnecessary.
Did restic help you today? Did it make you happy in any way?
Restic is backing up a dozen computers for me now, and is a joy to work with, providing very efficient and reliable backups of my personal files.