large prune fails #725
Description
Output of restic version
restic 0.3.3 (v0.3.3-4-g9f1f174)
compiled with go1.7.4 on linux/amd64
Expected behavior
I ran backups followed by restic forget for several months before noticing that it was eating a ton of diskspace. They I tried running restic prune to reclaim the space, but it failed
The repository is on another machine as accessed via NFS. This failure is running the prune over NFS. If I run it directly on the server it runs out of memory very quickly. (very whimpy machine)
Actual behavior
$ restic prune
counting files in repo
building new index for repo
[28:47] 100.00% 332112 / 332112 packs
repository contains 332112 packs (5251506 blobs) with 1.499 TiB bytes
processed 13790865 blobs: 8539359 duplicate blobs, 891.007 GiB duplicate
load all snapshots
find data that is still in use for 32 snapshots
[1:11:45] 100.00% 32 / 32 snapshots
found 2704802 of 13790865 data blobs still in use, removing 11086063 blobs
will delete 51619 packs and rewrite 271265 packs, this frees 284.798 GiB
header is larger than file
restic/pack.readHeader
/tmp/restic-build-068254688/src/restic/pack/pack.go:201
restic/pack.List
/tmp/restic-build-068254688/src/restic/pack/pack.go:223
restic/repository.Repack
/tmp/restic-build-068254688/src/restic/repository/repack.go:38
main.runPrune
/tmp/restic-build-068254688/src/cmds/restic/cmd_prune.go:211
main.glob..func11
/tmp/restic-build-068254688/src/cmds/restic/cmd_prune.go:26
github.com/spf13/cobra.(*Command).execute
/tmp/restic-build-068254688/src/github.com/spf13/cobra/command.go:599
github.com/spf13/cobra.(*Command).ExecuteC
/tmp/restic-build-068254688/src/github.com/spf13/cobra/command.go:689
github.com/spf13/cobra.(*Command).Execute
/tmp/restic-build-068254688/src/github.com/spf13/cobra/command.go:648
main.main
/tmp/restic-build-068254688/src/cmds/restic/main.go:40
runtime.main
/usr/local/go/src/runtime/proc.go:183
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:2086
Steps to reproduce the behavior
Rerunning has the same failure.