Streamed request with preset Host header sends two Host headers #4392
Description
The following code sends two host headers:
import requests
def gen(): yield b'binary data'
requests.post(
'http://localhost:3000/',
data=gen(),
headers={'Host': 'example.com'}
)(I tested with nc -vvl 3000 on OS X).
Headers received:
POST / HTTP/1.1
Host: localhost:3000
User-Agent: python-requests/2.18.1
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Host: example.com
Transfer-Encoding: chunked
This is because when chunking a request, we don't tell the httplib / http.client library to skip the host header if one is already set. From the adapters.HTTPAdapter.send() implementation:
low_conn.putrequest(request.method,
url,
skip_accept_encoding=True)which leaves the skip_host parameter set to False. It probably should use skip_host='host' in request.headers here.
System Information
>>> requests.help.main()
{
"chardet": {
"version": "3.0.4"
},
"cryptography": {
"version": "2.0.3"
},
"implementation": {
"name": "CPython",
"version": "3.6.3"
},
"platform": {
"release": "16.7.0",
"system": "Darwin"
},
"pyOpenSSL": {
"openssl_version": "1010006f",
"version": "17.3.0"
},
"requests": {
"version": "2.18.1"
},
"system_ssl": {
"version": "100020cf"
},
"urllib3": {
"version": "1.22"
},
"using_pyopenssl": true
}