Invalid calculation for max header size when headers are sent in at least two chunks #371
Description
Waitress incorrectly calculates header_bytes_received when headers are sent in more than one chunk. Issue is related to following code in parser.py
if index >= 0:
# If the headers have ended, and we also have part of the body
# message in data we still want to validate we aren't going
# over our limit for received headers.
self.header_bytes_received += index
consumed = datalen - (len(s) - index)
else:
self.header_bytes_received += datalen
consumed = datalenLength of data from chunk first to n - 1 (n is last) is added to header_bytes_received and when finally end of headers is is found in last chunk the header_bytes_received is incremented by size of all headers. This means that length of chunks from 1 to n - 1 is counted twice in header_bytes_received.
I think that if end of headers is found, header_bytes_received should be set to position of end of headers (index).
To reproduce this issue following code can be used.
Server:
import waitress
def application(environ, start_response):
start_response(
"200 OK", [("Content-Type", "text/plain")]
)
yield b"OK\r\n"
if __name__ == "__main__":
waitress.serve(
application,
max_request_header_size = 200,
)Client:
import socket
import time
header_data = b'1' * 100
s = socket.create_connection(('127.0.0.1', 8080), timeout = 60)
to_send = (
b"GET /long_header HTTP/1.0\r\n"
b"x-header: " + header_data + b"\r\n"
)
to_send2 = (
b"Content-Length: 0\r\n"
b"\r\n"
)
s.send(to_send)
time.sleep(2)
s.send(to_send2)
print("Headers size: ", len(to_send) + len(to_send2))
with s.makefile('rb', 0) as fp:
data = fp.read()
print(data.decode())
assert data.splitlines()[0] == b"HTTP/1.0 200 OK"
s.close()Currently this code produces 431 HTTP error when it should work because sent headers are smaller than 200 (161 bytes)
HTTP/1.0 431 Request Header Fields Too Large
Connection: close
Content-Length: 91
Content-Type: text/plain
Date: Tue, 22 Mar 2022 12:01:39 GMT
Server: waitress
Request Header Fields Too Large
exceeds max_header of 200
(generated by waitress)