-
-
Notifications
You must be signed in to change notification settings - Fork 910
Feature requests: --ignore-warning= mode (for WARNINGS) and --max-retries= / --wait= #819
Description
First off, this is a terrific utility and I've written a script to perform a weekly backup on my server. I have two suggestions for features that would be of immense benefit.
I am running s3cmd version 1.6.1, so not sure if some of these issues have been resolved in a more current release, but based on my review of the code I don't think so.
- I have probably 30 GB of user-uploads, which I have an automated cron script to run to back these up on a routine basis. Due to the number of files, it just takes a single bad connection to cause a failure with these kinds of warning messages. Then, I get a cron notification and have to restart the process manually.
Cannot retrieve any response status before encountering an EPIPE or ECONNRESET exception
WARNING: Upload failed: /srv/html/uploads/548/88919fd4bec2bf.pdf ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.00)
WARNING: Waiting 3 sec...
It would be helpful to hardcode in some kind of retries so that it will not just bail after a single upload. Perhaps even an increasing wait period between retries? There could be a retry limit per file and a total retry limit as well, which can be tuned based on network performance.
- When running this as a cron job, warnings about symbolic links, etc. are piped to STDERR causing it to fail with exit status.
Some of these warnings (for example, about symbolic links) should be ignored. Currently, I can achieve this with some advanced bash scripting, piped through grep, but it would be better to handle this differently. Specifying which warnings to silence would be very useful.