Skip to content

Fix http.client.RemoteDisconnected errors#1014

Merged
fviard merged 1 commit intos3tools:masterfrom
Quiq:master
Mar 21, 2020
Merged

Fix http.client.RemoteDisconnected errors#1014
fviard merged 1 commit intos3tools:masterfrom
Quiq:master

Conversation

@rhuddleston
Copy link
Copy Markdown

@rhuddleston rhuddleston commented Nov 4, 2018

With this commit 4 years ago OSError was caught:

889c5a7

Looks like there wasn't a good reason to include it give the comment. Also now with python 3 http.client.RemoteDisconnected is getting caught by OSError.

I see @fviard tried to make a fix for RemoteDisconnected here:

b4f1424

but the problem persisted until I removed these lines. There may be other places in the code that have the same issue but this fixed my specific issues with recv_file():

  File "/usr/bin/s3cmd", line 1316, in _download
    response = s3.object_get(uri, dst_stream, dst_file, extra_label = seq_label)
            http_response = conn.c.getresponse()
  File "/usr/lib/python3/dist-packages/S3/S3.py", line 715, in object_get
    response = self.recv_file(request, stream, labels, start_position)
  File "/usr/lib/python3/dist-packages/S3/S3.py", line 1612, in recv_file
    http_response = conn.c.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/lib/python3/dist-packages/S3/Custom_httplib3x.py", line 53, in httpresponse_patched_begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response```

I deployed this fix and verified it fixed the errors across our servers.

With this commit 4 years ago OSError was caught:
6a43907 
s3tools@889c5a7

Looks like there wasn't a good reason to include it give the comment. Also now with python 3 http.client.RemoteDisconnected is getting caught by OSError.
@rhuddleston
Copy link
Copy Markdown
Author

rhuddleston commented Feb 10, 2019

@fviard What do you think, can we merge this? Also maybe it's time for a 2.0.3 release :)

@nadiahpk
Copy link
Copy Markdown

nadiahpk commented May 18, 2019

@rhuddleston Thank you! I only started having the problem this week for some reason, and I couldn't figure it out. But this fixed my problem.

@rogierslag
Copy link
Copy Markdown

rogierslag commented Jun 13, 2019

Would be awesome to get this to a stable release :)

jkroepke added a commit to adorsys-containers/s3-tools that referenced this pull request Feb 20, 2020
@jkroepke
Added patch for connection lost
9a3b4ea 
s3tools/s3cmd#1014
@fviard fviard merged commit 7690455 into s3tools:master Mar 21, 2020
@fviard
Copy link
Copy Markdown
Contributor

fviard commented Mar 21, 2020

Finally merged, thanks for the investigation and the fix!

@jkroepke
Copy link
Copy Markdown

jkroepke commented Mar 25, 2020

Nice, when we can expect a stable release?

@fviard
Copy link
Copy Markdown
Contributor

fviard commented Mar 25, 2020

@jkroepke In the coming days.

@fviard
Copy link
Copy Markdown
Contributor

fviard commented Apr 7, 2020

Fyi, the new release v2.1.0 is out and available in github and in Pypi.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@rhuddleston @nadiahpk @rogierslag @fviard @jkroepke