Output of restic version
$ restic version
restic 0.18.0 compiled with go1.24.4 on linux/amd64
What should restic do differently? Which functionality do you think we should add?
restic copy takes a --pack-size argument which is very helpful, and provides a hint to restic of the target pack size in the remote. restic copy takes either a list of snapshots or a --from-repo argument. When restic copy takes a long list of snapshots or a --from-repo with many small diffs it treats each snapshot separately and is unable to respect the --pack-size argument.
Feature request: Spool packs across snapshots during restic copy. Rather than treating each snapshot as an individual entity, go through the snapshot list tracking needed data until hitting the requested pack size. Then send the data in a large pack, before sending the metadata for all of the snapshots that now have their required data on the remote.
I'm not yet familiar with this section of Restic's source code, but it would be great to get some thoughts on whether this is possible. With minor searches it appears safe as the worst case is you spend longer copying a larger pack that fails and that data remains unreferenced - this seems to be the expected behaviour when requesting a large pack size.
What are you trying to do? What problem would this solve?
I have many small backup sources that backup text files that change very little regularly. This creates many small packs in the repo when restic backup is used - this is expected behaviour, and when restic forget --prune --repack-small comes around they get grouped up into easier to manage files. All is well so far.
When I use restic copy from this repo with many snapshots and many large packs, it appears to be splitting things back into small packs because it moves the data each snapshot is responsible for at once. My expected behaviour is that it works to generate packs of the requested --pack-size instead.
For write once/read rarely (or expensive read) remote repos, this makes rsync or rclone more effectively than restic copy. That's because I can repack the local repo to get the pack size I want then rsync this somewhere and maintain that pack size. I cannot see a way to achieve this with restic copy without repacking on the remote.
Did restic help you today? Did it make you happy in any way?
Always :) I'm moving a bunch of my backups around, found the new --pack-size and flawlessly updated my core repos. Really nice stuff.
Output of
restic versionWhat should restic do differently? Which functionality do you think we should add?
restic copytakes a--pack-sizeargument which is very helpful, and provides a hint to restic of the target pack size in the remote.restic copytakes either a list of snapshots or a--from-repoargument. When restic copy takes a long list of snapshots or a--from-repowith many small diffs it treats each snapshot separately and is unable to respect the--pack-sizeargument.Feature request: Spool packs across snapshots during
restic copy. Rather than treating each snapshot as an individual entity, go through the snapshot list tracking needed data until hitting the requested pack size. Then send the data in a large pack, before sending the metadata for all of the snapshots that now have their required data on the remote.I'm not yet familiar with this section of Restic's source code, but it would be great to get some thoughts on whether this is possible. With minor searches it appears safe as the worst case is you spend longer copying a larger pack that fails and that data remains unreferenced - this seems to be the expected behaviour when requesting a large pack size.
What are you trying to do? What problem would this solve?
I have many small backup sources that backup text files that change very little regularly. This creates many small packs in the repo when
restic backupis used - this is expected behaviour, and whenrestic forget --prune --repack-smallcomes around they get grouped up into easier to manage files. All is well so far.When I use
restic copyfrom this repo with many snapshots and many large packs, it appears to be splitting things back into small packs because it moves the data each snapshot is responsible for at once. My expected behaviour is that it works to generate packs of the requested--pack-sizeinstead.For write once/read rarely (or expensive read) remote repos, this makes
rsyncorrclonemore effectively thanrestic copy. That's because I can repack the local repo to get the pack size I want thenrsyncthis somewhere and maintain that pack size. I cannot see a way to achieve this withrestic copywithout repacking on the remote.Did restic help you today? Did it make you happy in any way?
Always :) I'm moving a bunch of my backups around, found the new
--pack-sizeand flawlessly updated my core repos. Really nice stuff.