Timeout in session chunked requests lead to subsequent request timeout #4894
Description
Expected Result
Requests in same session don't interfere with one another.
Actual Result
A slow response from a chunked request causes the subsequent request to fail with a timeout even if the server responded before the timeout was reached.
Reproduction Steps
Run a test server that accepts GET and POST requests. When responding to POST requests it waits 10 seconds before responding:
import time
from http.server import BaseHTTPRequestHandler, HTTPServer
time_sleep = 10
class HandlerSlowResponse(BaseHTTPRequestHandler):
def do_POST(self):
# Hangs
time.sleep(time_sleep)
self.send_response(200)
self.end_headers()
def do_GET(self):
# OK
self.send_response(200)
self.end_headers()
if __name__ == '__main__':
print("Time sleep %s" % time_sleep)
httpd = HTTPServer(('', 8000), HandlerSlowResponse)
print('Listening on :8000')
httpd.serve_forever()Then, in another REPL, verify non-session requests don't interfere with one another:
import requests
import time
timeout = 2
def norm(i):
print("GET (quick response) %s" % i)
try:
requests.get('http://localhost:8000', timeout=timeout, data='hello'.encode('utf-8'))
except Exception as e:
print("-> EXC Norm %s %s" % (i, e))
def chunked(i):
print("POST chunked (slow response) %s" % i)
# Does not fail if exceeds timeout
def gen():
yield 'hello'.encode('utf-8')
try:
requests.post('http://localhost:8000', timeout=timeout, data=gen())
except Exception as e:
print("-> EXC Chunked %s %s" % (i, e))
def cycle():
# requests don't interfere with one another
for i in range(1, 10):
time.sleep(1)
if i % 2 == 0:
norm(i)
else:
chunked(i)
if __name__ == "__main__":
print("Timeout %s" % timeout)
cycle()Output:
Note that chunked requests never time out (ticket #4402).
Timeout 2
POST chunked (slow response) 1
GET (quick response) 2
POST chunked (slow response) 3
GET (quick response) 4
POST chunked (slow response) 5
GET (quick response) 6
POST chunked (slow response) 7
GET (quick response) 8
POST chunked (slow response) 9
Next, do the same thing but now using a Session:
import requests
import time
timeout = 2
session = requests.Session()
def norm_session(i):
print("GET norm session (quick response) %s" % i)
try:
session.get('http://localhost:8000', timeout=timeout, data='hello'.encode('utf-8'))
except Exception as e:
print("-> EXC Norm session %s %s" % (i, e))
def chunked_session(i):
print("POST chunked session (slow response) %s" % i)
def gen():
yield 'hello'.encode('utf-8')
try:
session.post('http://localhost:8000', timeout=timeout, data=gen())
except Exception as e:
print("-> EXC Chunked session %s %s" % (i, e))
def cycle_session():
# A normal request always times out after a chunked timeout request
# even if the server responded in time
for i in range(1, 10):
time.sleep(1)
if i % 2 == 0:
norm_session(i)
else:
chunked_session(i)
if __name__ == "__main__":
print("Timeout %s" % timeout)
cycle_session()Output:
Note that chunked requests do time out but cause the subsequent GET to fail.
Timeout 2
POST chunked session (slow response) 1
GET norm session (quick response) 2
POST chunked session (slow response) 3
-> EXC Chunked session 3 timed out
GET norm session (quick response) 4
-> EXC Norm session 4 HTTPConnectionPool(host='localhost', port=8000): Read timed out. (read timeout=2)
POST chunked session (slow response) 5
GET norm session (quick response) 6
POST chunked session (slow response) 7
-> EXC Chunked session 7 timed out
GET norm session (quick response) 8
-> EXC Norm session 8 HTTPConnectionPool(host='localhost', port=8000): Read timed out. (read timeout=2)
POST chunked session (slow response) 9
System Information
$ python -m requests.help
{
"chardet": {
"version": "3.0.4"
},
"cryptography": {
"version": ""
},
"idna": {
"version": "2.6"
},
"implementation": {
"name": "CPython",
"version": "3.6.5"
},
"platform": {
"release": "16.7.0",
"system": "Darwin"
},
"pyOpenSSL": {
"openssl_version": "",
"version": null
},
"requests": {
"version": "2.18.4"
},
"system_ssl": {
"version": "1000210f"
},
"urllib3": {
"version": "1.22"
},
"using_pyopenssl": false
}
This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).