Output of restic version
I typically use:
restic 0.8.3
compiled with go1.10 on linux/amd64
but I also tried with:
restic 0.9.2 compiled with go1.10.3 on linux/amd64
How did you run restic exactly?
restic -r <repo_path> forget --keep-last 5 --prune
What backend/server/service did you use to store the repository?
This may probably be very relevant: the source of the data is on a USB hard disk, which is mounted always in the same mount point. The repository is instead a SMB share mounted on a specific constant path via cifs. SMB share is stored on a ext4 filesystem.
I've seen this failure reported other times, but I created a new issue anyway to explain this particular situation.
Expected behavior
Prune should succeed.
Actual behavior
Backing up succeeds. Then, I run the above command, which should remove exactly one snapshot. Prune fails returning exit code 1. Running prune once again instead seems to work and report 5 available snapshots. The error reported by the first prune command is:
enter password for repository:
repository 2b913e7d opened successfully, password is correct
Applying Policy: keep the last 5 snapshots snapshots
snapshots for ([...]):
keep 5 snapshots:
ID Date Host Tags Directory
----------------------------------------------------------------------
[...]
----------------------------------------------------------------------
5 snapshots
remove 1 snapshots:
ID Date Host Tags Directory
----------------------------------------------------------------------
[...]
----------------------------------------------------------------------
1 snapshots
1 snapshots have been removed, running prune
counting files in repo
building new index for repo
[8:32] 100.00% 30451 / 30451 packs
incomplete pack file (will be removed): 0419682275c07ffa260e5369a3a1d14c750a8e58ef4e1a88d69339717233874b
incomplete pack file (will be removed): 052ae43adbf9501b7acee6ff25ace7a5f57ebc254428c5c4d5e350390f0161e4
incomplete pack file (will be removed): 06ffe477f19fc9f0857897ce98170951400694d155f4efcc0d93cca29dcf0153
incomplete pack file (will be removed): 08af490a1f8debbca69808314b24bc61f45be5e12c084ecd304518b8875a2bde
incomplete pack file (will be removed): 098c7c1c13f5070c7265aeb16aa2f2e3c10fd53147114e2d5dc5434ea29dd9ba
incomplete pack file (will be removed): 0d0db3810b0be0fbc3582521c34b4cf5729418c23853e47f08021717a30967a7
incomplete pack file (will be removed): 11c7e8b95ed3eff46b2819bc3608a0bbe2afa658df1e5a5a94745f76cfba0304
incomplete pack file (will be removed): 11ef82a4fe5f62fc37ad432fb9ffc1611fa1fbc060c28b06160b8f7b0e05d095
incomplete pack file (will be removed): 1281ca112f53a1a1944b10cf258c3fabd36209b394f837a6afea5922c4c41697
incomplete pack file (will be removed): 1c5afa76417bcbc932499b000ab4fbc5c1fabf604d006883c454e6fdad9b3836
incomplete pack file (will be removed): 1c6606767eb76b658e554d8189a468a1db418df1dc448905428b2b087703932c
incomplete pack file (will be removed): 28a3b8b4c8d466fa4998d34412bee15ba6e05e4623a0fee0f39823411a1eca12
incomplete pack file (will be removed): 2bae1122447e8de3001327596b3af9983414307856d71da12db1ede6b871166f
incomplete pack file (will be removed): 3182fea28f3095ed9db5c3e41ff5cd34856c881f0f797004ff7ce3c66705525a
incomplete pack file (will be removed): 31f069689a117e800522e5568ff3df49883a85d9d1a29da3fb66c758a7caf10d
incomplete pack file (will be removed): 3279f70d1f19b400df47acc858c9967ba1845fc84f85e6a0e880ea19dea4c3d4
incomplete pack file (will be removed): 362358d67c70d287b35f149245f7d17faa274c86286779397b6c0a50d958d3e2
incomplete pack file (will be removed): 3aa916f478c9b53c59d2e9a4b23f81acf3dc2612c43612f14e2c6c7edee6f5d5
incomplete pack file (will be removed): 3ac13a9630f020b88a00d43211141126a0f83aa11bcd9cfa74acd2c30c5eb7d0
incomplete pack file (will be removed): 3f67a59f7fb7f86ebb3baea412fef3690dc3422d743430f411c2d89275d7436c
incomplete pack file (will be removed): 3fc8f32d65c08e9d94954bd25faa886eeffb782a53ac66f4e606509539fd5786
incomplete pack file (will be removed): 4db38e486f34070801bf13e71d468c8927c66bff4fb55f53fa1c48ce605b6cf9
incomplete pack file (will be removed): 4f1380d6ce36a11eb8158ebfa331f9fbb657fc57d3e683b738cf31bc55ef87da
incomplete pack file (will be removed): 4ff210a088d4a93ef8dadfbd5548ad243988b084cfbe4f320d1eb5f7826890cb
incomplete pack file (will be removed): 5853d3e8d5bebb3ffd4157ca6c9bdd62302486d14b09df2f60027422484fe483
incomplete pack file (will be removed): 64cbd81bcb3c0a3839400bf51c256f5ef48cf55ebdc947c9542a2157e6dbcb45
incomplete pack file (will be removed): 67080490cf05f4f4c2b2a79a329f7ae12d42803199a1a218a06329d96eeb8f48
incomplete pack file (will be removed): 701b3d47ed15afd38b4a54f4dfef8dac39adfd7fa5c220b4a27fc08f663ffba6
incomplete pack file (will be removed): 744f4a5f609cc8e0d32d6db97d41e442f64a3b4f375385f07dd9d28ebcb2733a
incomplete pack file (will be removed): 78fc31910055722c4f30d4dc98a8a102b62717169b4017463af808a64e5a271b
incomplete pack file (will be removed): 7ce924782de235bd51417e2f4b7cda9ba769d4c059eea12a016482ed8a18a7ee
incomplete pack file (will be removed): 7fefd7b9bdf8b271087996b781a1555aab0b7f87e4776e15657c2cff8c8abf20
incomplete pack file (will be removed): 80f7cae4759579cc2d24404bdc4d1ed1734c13957b825b228080d436b1f6d799
incomplete pack file (will be removed): 84d6087c5930c270f52f1f328aae66b9a1f5c37d8e950dbf6552957fb7ecce5b
incomplete pack file (will be removed): 869c67459a7d996706896a1d1a4c2ae31be1a40a4daa9d422f1e0befa3d1dcea
incomplete pack file (will be removed): 8aa1cb77f0e799835ff635b1fbb15c67c6ced46da705df7e13d89577d4f5f441
incomplete pack file (will be removed): 97b80c967d5649794deb32cc6915c1b14e537e0d1b8004a198860bf7fd5dd941
incomplete pack file (will be removed): 9a83cd05dfdcd5cfdeb32c52aa629b53e0f66d1cacbb9c9a36c89c75a7b0367a
incomplete pack file (will be removed): 9b34ae7836789b440715ee337721d56a0e4f874319b4a9f9d7bef76553cdc4b5
incomplete pack file (will be removed): 9f2a40348559b9d8e9b8323aeb39752e92beb3df06b243c7ab7ef9cdc1d27c69
incomplete pack file (will be removed): 9f715e1df6d580914abcd1fb565bca83ec862128c71f1a0b5a0ca8e1497a86df
incomplete pack file (will be removed): ac312cb7d3163de46f4a1a17ee18d3584426f08d2cc77b763baa08dcba27eea6
incomplete pack file (will be removed): ada80341e6fac08e2b4cba6c5d3ccdf4b6d55337598ae327f67e3a08a1dc167d
incomplete pack file (will be removed): b473443d7609fc9680d298ce9b498e0bf8155d96cc36b0a3d1cdfda007629856
incomplete pack file (will be removed): b4939cbed564a8e32c25d94eb83d3b95cbcfd1d7b3ff70efe991f6fdf4c9e33b
incomplete pack file (will be removed): b604e614ed8a4cd19aba44feb1f9608151a81c02eb1beef8484a2f56db3d2b6b
incomplete pack file (will be removed): b64921a9827653f311c38f77622188fe8aa59b28f3083fd12b1884a62ff235a7
incomplete pack file (will be removed): ba6c08b1bb7c9f717ce6b4dcd27d45e22fd6b8dd7cd9ba324af1ee051e72f2bd
incomplete pack file (will be removed): be1e7a1489b837859b23f6b28ff4de93f894d4cb01651c633d05487836e5d8eb
incomplete pack file (will be removed): be7d36159084ba965e2773ea331496a453df42d4cf5db940d925e13ea8ac5809
incomplete pack file (will be removed): c1290a18a7cb37b071749629d3780153c5e750e461d8f9c823dcf975b9817a45
incomplete pack file (will be removed): c65e2b45b9d5e2e7bf4413f67ac9b4ba2fab16ce799656ccd025ebc013dcae77
incomplete pack file (will be removed): c6d6eb2b00e8fad066abaa9f3495ad772e3beb74b5152775d524bf49a861fc58
incomplete pack file (will be removed): c73c3cb9a2794ef6a8960c530d515c1a250f11b4bc5d90a8230a24ce67a5e551
incomplete pack file (will be removed): c988a36ddd335b5b8e4b76ba63235632f0015ce8c8f70fcc85f8db1e7f1b185d
incomplete pack file (will be removed): daaed644c6cfb078f6211df1c5ef121fd62eb1b0f81a6d542b0388d5bfa928cf
incomplete pack file (will be removed): dd3d09a8062d80aa2c97759005f9729a88391af54f214c0895caadb34aeb571e
incomplete pack file (will be removed): e5952a97b5da1234beb15bcec2e8b31231834328b44ab2ee7cbcfa90c98f90d4
incomplete pack file (will be removed): e8c6d9c2794945503574853b5d36291d92503d6a259b51823b38f714fc178a4e
incomplete pack file (will be removed): e8da886b00b552c6bb95e9656b69366c568f778136f3d4ce49cbc0978ca13156
incomplete pack file (will be removed): e96b280db2ded87eeddfde558a72d1756e01dc430e1706a1a540586b71606623
incomplete pack file (will be removed): f00ceebe50cd5062abb52b4b73f19f95cbeed29e674b04ec3dce24251888635e
incomplete pack file (will be removed): fa1649e4c66455325e9e320bf06fbfd2e038a70d49346ab28c87b21cf36bc14e
incomplete pack file (will be removed): ff96a7f6ed64b19412c5bd69503eec2ad76cd2a82bb1d16dc7ec1ab1ffee45c3
repository contains 30387 packs (117851 blobs) with 147.154 GiB
processed 117851 blobs: 24540 duplicate blobs, 29.393 GiB duplicate
load all snapshots
find data that is still in use for 5 snapshots
[0:00] 100.00% 5 / 5 snapshots
found 93121 of 117851 data blobs still in use, removing 24730 blobs
will remove 64 invalid files
will delete 67 packs and rewrite 8017 packs, this frees 29.468 GiB
hash does not match id: want ad4b41a807fef9716a60f92f63334d3b28fd5a8a744a43069455fcd42f1db352, got 2ac3712c12e1b363c94fd6637fa8c09e8cffbc6d0bfc13699d83b8121e45c085
github.com/restic/restic/internal/repository.Repack
src/github.com/restic/restic/internal/repository/repack.go:35
main.pruneRepository
src/github.com/restic/restic/cmd/restic/cmd_prune.go:278
main.runForget
src/github.com/restic/restic/cmd/restic/cmd_forget.go:240
main.glob..func10
src/github.com/restic/restic/cmd/restic/cmd_forget.go:24
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).execute
src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:762
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).ExecuteC
src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:852
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).Execute
src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:800
main.main
src/github.com/restic/restic/cmd/restic/main.go:86
runtime.main
/usr/local/go/src/runtime/proc.go:198
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:2361
Steps to reproduce the behavior
Backup once. Prune with the above command. Prune fails. Second prune command succeeds.
This seems to be always reproducible.
Do you have any idea what may have caused this?
Unfortunately no idea.
Do you have an idea how to solve the issue?
I tried to rebuild the index. It succeeds, but the the same behaviour can be reproduced: backup, prune, failure. I also removed restic cache from the home directory, nothing changed.
I created this repo for development purposes, I'm not much interested in the data it contains. I can freely experiment on it to fix this issue if someone is interested. I cannot provide the repo itself as it contains a private data set.
Did restic help you or made you happy in any way?
Of course. Seems to be a good backup solution, but still needs a few critical features.
Output of
restic versionI typically use:
but I also tried with:
restic 0.9.2 compiled with go1.10.3 on linux/amd64How did you run restic exactly?
restic -r <repo_path> forget --keep-last 5 --prune
What backend/server/service did you use to store the repository?
This may probably be very relevant: the source of the data is on a USB hard disk, which is mounted always in the same mount point. The repository is instead a SMB share mounted on a specific constant path via cifs. SMB share is stored on a ext4 filesystem.
I've seen this failure reported other times, but I created a new issue anyway to explain this particular situation.
Expected behavior
Prune should succeed.
Actual behavior
Backing up succeeds. Then, I run the above command, which should remove exactly one snapshot. Prune fails returning exit code 1. Running prune once again instead seems to work and report 5 available snapshots. The error reported by the first prune command is:
Steps to reproduce the behavior
Backup once. Prune with the above command. Prune fails. Second prune command succeeds.
This seems to be always reproducible.
Do you have any idea what may have caused this?
Unfortunately no idea.
Do you have an idea how to solve the issue?
I tried to rebuild the index. It succeeds, but the the same behaviour can be reproduced: backup, prune, failure. I also removed restic cache from the home directory, nothing changed.
I created this repo for development purposes, I'm not much interested in the data it contains. I can freely experiment on it to fix this issue if someone is interested. I cannot provide the repo itself as it contains a private data set.
Did restic help you or made you happy in any way?
Of course. Seems to be a good backup solution, but still needs a few critical features.