Changelog category (leave one):

• Experimental Feature

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Refactor HTTP server. Separate HTTP/1.x implementation. Support HTTP/2 using nghttp2

Documentation entry for user-facing changes

• Documentation is written (mandatory for new features)

Technical details

Move changes from #80141.

HTTPRequestHandler now has a single virtual method handleRequest that takes HTTPServerRequest and HTTPServerResponseBase.

HTTPServerRequest stores the request method, path, and header fields. It also provides a ReadBuffer to read the body from. This ReadBuffer is passed to HTTPServerRequest's constructor, and handleRequest knows nothing about its implementation.

HTTPServerResponseBase provides storage for the response status and header fields. It also has an abstract makeUniqueStream method that returns a WriteBufferFromHTTPServerResponseBase for the response body. Now, this is the only way to send a response to the client. Previously, there were two ways that duplicated each other's logic: HTTPServerResponse::send() and WriteBufferFromHTTPServerResponse. They are now unified.

HTTP/2 support is enabled via the <http2_server></http2_server> config section.

If HTTP/2 is enabled, HTTPServerConnectionFactory can decide whether the connection should be handled as HTTP/2. If the connection is secure, it checks ALPN. Otherwise, it checks if the client has sent an HTTP/2 preface.

Now, there are two connection handlers: HTTP1ServerConnection and HTTP2ServerConnection. The first one works as before. It creates an HTTPServerRequest with the ReadBuffer from the HTTP/1.x stream (fixed length/chunked) and an HTTPServerResponseBase with the WriteBufferFromHTTPServerResponseBase implementation based on the old WriteBufferFromHTTPServerResponse class.

HTTP2ServerConnection handles an HTTP/2 connection. It has a main IO thread that exchanges HTTP/2 frames with the client. For each new stream, it creates a new thread that runs handleRequest. The HTTP/2 stream provides its own implementations of ReadBuffer and WriteBufferFromHTTPServerResponseBase.

One important thing to mention: the HTTP/1.x server had a feature for sending X-ClickHouse-Progress headers to the client over time to report query progress. For HTTP/2, we can't implement the same thing easily (at least because HEADERS (+CONTINUATION) frames must be sent together, without any other frame in between (RFC 9113 section 4.3.)). So, WriteBufferFromHTTPServerResponseBase::onProgress implementation for HTTP/1.x remains the same as before, while its HTTP/2 implementation is a no-op. I believe the correct way to implement progress reporting for HTTP/2 is via 1xx informational responses.