Skip to content

[http1] fix H/1 response pipelining#13983

Merged
asraa merged 8 commits intoenvoyproxy:masterfrom
asraa:alloc-headers
Nov 18, 2020
Merged

[http1] fix H/1 response pipelining#13983
asraa merged 8 commits intoenvoyproxy:masterfrom
asraa:alloc-headers

Conversation

@asraa
Copy link
Copy Markdown
Contributor

@asraa asraa commented Nov 11, 2020

Signed-off-by: Asra Ali asraa@google.com

Commit Message: Fix H/1 response pipelining. Pipelined responses would continue to process through http_parser and cause failures in double header allocations. Instead, pause http_parser after a response is complete, and add support to redispatch in the codec client if there is more data and the connection is still open.

Risk Level: Medium, changes some H/1 flow.
Testing: Added an integration test, regression tests from fuzzer that caught the problem.
Fixes: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26838

Signed-off-by: Asra Ali <asraa@google.com>
Signed-off-by: Asra Ali <asraa@google.com>
Copy link
Copy Markdown
Contributor

@antoniovicente antoniovicente left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: I'm not sure why conditioning on active requests to redispatch breaks integration tests, it made sense to me, and would like to understand why not (or maybe it is right, and something else needs to be changed)

Thanks for the fix. redispatch breaking integration tests sounds weird and interesting. Your mention of 204 responses makes me think that the issue may be related to an upstream generating a response body for a response that can't have a body (e.i. 204, no content, or possibly some HEAD requests). I'm curious about which tests fail.

// The HTTP/1 codec will pause dispatch after a single response is complete. We redispatch if
// there is more data.
// TODO(asraa): Check that there are more active requests? This causes a failure in a response
// with 204 integration test.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible for data to be non-empty at this point when proxying H2 or H3 requests? I think the answer is no.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the answer is no as well. If all of the data was not consumed it means codec had encountered an error.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add an ASSERT and see if fuzzing turns something up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's correct re H2. even if the connection closes, nghttp2 will still consume any unconsumed data.

about H3, I hope it also does this. It is opaque to me, but I can guess that it does because it currently loops through all available data until there is none. i don't know about connection closing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are some conditions about remaining data. A lot of tests fail when I add the ASSERT. Looking into it.

@yanavlasov
Copy link
Copy Markdown
Contributor

/wait

Signed-off-by: Asra Ali <asraa@google.com>
@asraa
Copy link
Copy Markdown
Contributor Author

asraa commented Nov 12, 2020

Thanks Antonio for pushing me to dig into why a 204 test (

upstream_request_->encodeData(512, true);
) would fail. It indicated I was mistaken about seeing multiple responses like you and Matt pointed out. The message is complete after the headers and the body should be dropped and cause a connection close instead of redispatching.

So, no more re-dispatch. Draining and closing it is.

Signed-off-by: Asra Ali <asraa@google.com>
Signed-off-by: Asra Ali <asraa@google.com>
Signed-off-by: Asra Ali <asraa@google.com>
Copy link
Copy Markdown
Contributor

@antoniovicente antoniovicente left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just some very minor nits.

Signed-off-by: Asra Ali <asraa@google.com>
Signed-off-by: Asra Ali <asraa@google.com>
@asraa asraa merged commit 5290844 into envoyproxy:master Nov 18, 2020
mpuncel added a commit to mpuncel/envoy that referenced this pull request Nov 18, 2020
* master: (117 commits)
  vrp: allow supervisord to open its log file (envoyproxy#14066)
  [http1] fix H/1 response pipelining (envoyproxy#13983)
  wasm: make dependency clearer (envoyproxy#14062)
  docs: updating 100-continue docs (envoyproxy#14040)
  quiche: fix stream trailer decoding issue (envoyproxy#13871)
  tidy: use last_github_commit script instead of target branch (envoyproxy#14052)
  stats: use RE2 and a better pattern to accelerate a single stats tag-extraction RE (envoyproxy#8831)
  wasm: use static registration for runtimes (envoyproxy#14014)
  grpc-json-transcoder: Add support for configuring unescaping behavior (envoyproxy#14009)
  ci: fix CodeQL-build by removing deprecated set-env command (envoyproxy#14046)
  config: fix crash when type URL doesn't match proto. (envoyproxy#14031)
  Build: Propagate user-supplied tags to external headers library. (envoyproxy#14016)
  [test host utils] use make_shared to avoid memory leaks (envoyproxy#14042)
  jwt_authn: update to jwt_verify_lib with 1 minute clock skew (envoyproxy#13872)
  quiche: update QUICHE tar (envoyproxy#13949)
  sds: improve watched directory documentation. (envoyproxy#14029)
  log the internal error message from *SSL when the cert and private key doesn't match (envoyproxy#14023)
  wasm: fix CPE for Wasmtime. (envoyproxy#14024)
  docs: Bump sphinxext-rediraffe version (envoyproxy#13996)
  CDS: remove warming cluster if CDS response desired (envoyproxy#13997)
  ...
andreyprezotto pushed a commit to andreyprezotto/envoy that referenced this pull request Nov 24, 2020
* Fix HTTP/1 response pipelining. Extraneous data after a response complete when the connection is still open should not be processed.

Signed-off-by: Asra Ali <asraa@google.com>
qqustc pushed a commit to qqustc/envoy that referenced this pull request Nov 24, 2020
* Fix HTTP/1 response pipelining. Extraneous data after a response complete when the connection is still open should not be processed.

Signed-off-by: Asra Ali <asraa@google.com>
Signed-off-by: Qin Qin <qqin@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants