Skip to content

prune: improve automatic handling of small pack files #5293

@MichaelEischer

Description

@MichaelEischer

Output of restic version

0.17.3

What should restic do differently? Which functionality do you think we should add?

prune should clean up small files more aggressively by default. Currently only pack files smaller than 4% of the default pack size are cleaned up. Setting --repack-small increases that threshold to 80%. The low default of 4% was chosen to not accidentally repack the whole repository when users switch from the 4MB default pack size of old restic versions to the new maximum of 128MB. However, this results in every backup run effectively leaving a small pack file behind that is not cleaned up by default.

In parallel to #5183, @rawtaz and I discussed options to improve the current default behavior. What should work is the following:

Retrieve the size of all pack files. Then let prune consider everything smaller than 80% of the 1st percentile (or maybe the 3rd or 5th percentile) as "small". That variant will work independently of the packsize with which the repository was created initially. This will completely solve the use case of getting rid of small files for normal repositories.

Then the only remaining use case is migrating a repository to a higher pack size. There it is up to the users to decide which file sizes to upgrade. The most common scenario will still be upgrading everything to the new pack size (although a lot of users probably won't even do that). This can be handled by the new --repack-smaller-than option. --repack-small would then be deprecated.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions