s3cmd sync fails with RemoteDisconnected #960
Description
Looks like it's not able to fetch second part of listing the bucket content
DEBUG: Canonical Request:
GET
/
marker=Foto%2F2001%2FGemini%2FSoftball%2FDownload%2FHow%20To%20Play%20Softball%285%29.htm&prefix=Foto%2F
host:pirkojm-backup.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180306T100045Z
host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-date': '20180306T100045Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=****/20180306/eu-west-1/s3/aws4_request,SignedHeaders=host;x-am
z-content-sha256;x-amz-date,Signature=****', 'x-amz-content-sha256': '****'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(pirkojm-backup): pirkojm-backup.s3.amazonaws.com
DEBUG: ConnMan.get(): re-using connection: https://pirkojm-backup.s3.amazonaws.com#1
DEBUG: format_uri(): /?marker=Foto%2F2001%2FGemini%2FSoftball%2FDownload%2FHow%20To%20Play%20Softball%285%29.htm&prefix=Foto%2F
DEBUG: Sending request method_string='GET', uri='/?marker=Foto%2F2001%2FGemini%2FSoftball%2FDownload%2FHow%20To%20Play%20Softball%285%29.htm&prefix=Foto%2F', headers={'x-amz-date': '2018030
6T100045Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=****/20180306/eu-west-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=****', 'x-amz-content-sha256': '****'}, body=(0 bytes)
DEBUG: Response:
{}
Here is the trace:
Problem: RemoteDisconnected: Remote end closed connection without response
S3cmd: 2.0.1
python: 3.6.4 (default, Jan 25 2018, 15:20:29)
[GCC 7.2.1 20180116]
environment LANG=cs_CZ.UTF-8
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 3073, in <module>
rc = main()
File "/usr/bin/s3cmd", line 2989, in main
rc = cmd_func(args)
File "/usr/bin/s3cmd", line 1873, in cmd_sync
return cmd_sync_local2remote(args)
File "/usr/bin/s3cmd", line 1846, in cmd_sync_local2remote
ret = _single_process(args[:-1])
File "/usr/bin/s3cmd", line 1605, in _single_process
return _child(destination_base, source_args)
File "/usr/bin/s3cmd", line 1709, in _child
remote_list, dst_exclude_list, remote_total_size = fetch_remote_list(destbase_with_source_list, recursive = True, require_attribs = True)
File "/usr/lib/python3.6/site-packages/S3/FileLists.py", line 460, in fetch_remote_list
objectlist, tmp_total_size = _get_filelist_remote(uri, recursive = True)
File "/usr/lib/python3.6/site-packages/S3/FileLists.py", line 397, in _get_filelist_remote
recursive = recursive, uri_params = uri_params)
File "/usr/lib/python3.6/site-packages/S3/S3.py", line 308, in bucket_list
for truncated, dirs, objects in self.bucket_list_streaming(bucket, prefix, recursive, uri_params, limit):
File "/usr/lib/python3.6/site-packages/S3/S3.py", line 342, in bucket_list_streaming
response = self.bucket_list_noparse(bucket, prefix, recursive, uri_params, max_keys)
File "/usr/lib/python3.6/site-packages/S3/S3.py", line 373, in bucket_list_noparse
response = self.send_request(request)
File "/usr/lib/python3.6/site-packages/S3/S3.py", line 1256, in send_request
http_response = conn.c.getresponse()
File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib/python3.6/site-packages/S3/Custom_httplib3x.py", line 53, in httpresponse_patched_begin
version, status, reason = self._read_status()
File "/usr/lib/python3.6/http/client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response