-
Notifications
You must be signed in to change notification settings - Fork 1.7k
[backend-error-redesign] "context canceled" issues due to timeouts #4970
Description
Output of restic version
Built from master on 29th of July:
restic 0.17.0-dev (compiled manually) compiled with go1.22.0 on linux/amd64
What backend/service did you use to store the repository?
s3 (served via standalone minio deployment)
Problem description / Steps to reproduce
Getting "context canceled" while doing prune operations lately.
Expected behavior
Operations should have a chance to finish, even on extremely slow endpoints.
Actual behavior
Depending on the IO status on the hardware (which is a busy one with multiple IO-heavy processes are running in parallel, albeit sometimes slow), some operations gets canceled. E.g.
List(data) returned error, retrying after 1s: Get "http://myhost:8888/mybucket/?continuation-token=ZGF0YS83Ni83NmQ4OTg2MzUzOGMxMjMxOTU2MDI2Mzg2M2M4NmM0ZTM3NzNjMjU2Y2I2MjBiZWJjYzVhZDQzY2E3ZWQ0NjIwW21pbmlvX2NhY2hlOnYyLGlkOmFiZjQ5YjFkLTVjYTItNDY3MS1hNzM4LWIyMjFkZDIwNTMzOSxwOjAsczowXQ%3D%3D&delimiter=&encoding-type=url&fetch-owner=true&list-type=2&prefix=data%2F": context canceled
Do you have any idea what may have caused this?
A while ago I asked the question on the forum and turned out changing one value was solving the issue (or disabling the backend-error-redesign feature flag).
Wanted to open this as an issue to make it traceable, since this is a blocking thing on slow endpoints. Plus the flag has been enabled as beta on release version also.
Even though the endpoint is a lot slower than expected, I don't think restic should require a "minimum speed" from endpoints. Maybe the timeout can be configured with a special argument (sorry if there is already one, I could not find digging the code).
Did restic help you today? Did it make you happy in any way?
Still the best! 🚀