-
Notifications
You must be signed in to change notification settings - Fork 5.3k
Description
Description:
There is an early return in ConnPoolImplBase::checkForDrained that prevents closeIdleConnections from being called in the case where there are no drained callbacks registered for the pool:
void ConnPoolImplBase::checkForDrained() {
if (drained_callbacks_.empty()) {
return;
}
closeIdleConnections();In TcpTunnelingIntegrationTest.InvalidResponseHeaders inside test/integration/tcp_tunneling_integration_test.cc, the behaviour expected by the test changes based on the presence of a callback.
In this InvalidResponseHeaders test, an invalid response header is sent from the upstream fake server. The test expects that Envoy will send an HTTP/2 reset frame, but if the connection pool has a drained callback, the connection closes at the transport layer instead.
The relevant sequence is as follows:
- The server sends a non-200 response code
- Inside
TcpProxy::HttpUpstream::DecoderShim::decodeHeaders,TcpProxy::HttpUpstream::resetEncoderis called. - After a few more intermediate function calls,
Http::CodecClient::deleteRequestis called and the request is removed from the client'sactive_requests_list. - After this,
codec_client_callbacks_->onStreamDestroy()is called, which ends up with a call toHttp::Http2::ConnPoolImpl::onStreamDestroy. - In
onStreamDestroy,ConnPoolImplBase::checkForDrainedis called.
In the current code, when checkForDrained is called at step (5), the early exit is hit so no idle connections are affected. The connection is then allowed to send out a HTTP/2 reset frame and the upstream server endpoint closes the connection.
However, if you add a callback to the connection pool, the early exit is skipped and closeIdleConnections is executed. Because the request was removed in step (3) above, the ActiveClient associated with the connection is considered idle and is closed, which in turn closes the connection on the Envoy upstream client side. This prevents the HTTP/2 reset frame from being sent.
I have a few questions:
- Is it correct that
checkForDrainedshould have different observable behaviour depending on the presence of an idle callback? (I'm guessing 'no')
** If we wanted to do this, we could movecloseIdleConnectionsabove the early return, but that affects question (2) - The
Http2::ConnPoolImpl::onStreamDestroycode greedily attempts to reap idle connections (which closes the socket at the transport layer), while the integration test asserts that the HTTP/2 layer should be allowed to sent a reset frame. Which is correct?
This was found while I was testing requested changes for #13061 so if there's a good solution to this problem I might be able to wrap it up in that PR.
Repro steps:
Add a drained callback to the constructor of ConnectionPool::ConnPoolImplBase and run the bazel target //test/integration:tcp_tunneling_integration_test.
Logs:
The test log after the repro steps above (with -l trace) invalid_response_headers.log