Skip to content

Race condition in Clear() function in file restic/internal/cache/file.go #4760

@damoclark

Description

@damoclark

Output of restic version

restic 0.16.4 compiled with go1.21.6 on linux/amd64

What backend/service did you use to store the repository?

rest-server version rest-server 0.12.1 compiled with go1.20.5 on linux/amd64

Problem description / Steps to reproduce

When performing backups on each host, I backup per filesystem separately. This is to make it easier to restore based on mount point. So for example, I backup / and /home separately.

I schedule these backups using cron and have them all start at the same time.

Sometimes, I get the following error (but not always):

error clearing pack files in cache: remove /home/<username>/.cache/restic/<hash>/data/1b/1b<hash>: no such file or directory

or

error clearing index files in cache: remove /home/<username>/.cache/restic/<hash>/index/79/79<hash>: no such file or directory

I suspect that there is a race condition when clearing the local cache of packs and indexes that no longer exist in the repository between different restic backup processes from the same user on the same host. The race is occurring in the Clear method for file /internal/cache/file.go at line 152

Expected behavior

If an obsolete index or pack file in the cache is found, and upon attempting to delete it, the operating system reports that it does not exist - this error should be ignored.

Or alternately, local locking semantics are used so that only one restic process is able to clear the cache of obsolete entries at a time.

The latter would be the more 'correct' solution. But admittedly, it is an edge case and ignoring os.ErrNotExist would probably suffice.

Actual behavior

A race condition can occur where multiple restic processes are building a list of obsolete entries, and then attempting to delete them at the same time, resulting in the above unpredictable outcomes. The process that encounters a missing entry immediately stops processing its list of files, and if the list of files is ordered, would mean that ultimately, the files would get removed. However, the error that results reports a harmless happenstance, and is unnecessary.

Did restic help you today? Did it make you happy in any way?

Restic is backing up a dozen computers for me now, and is a joy to work with, providing very efficient and reliable backups of my personal files.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions