Skip to content

buffer filter: Populate content-length header#7848

Merged
mattklein123 merged 3 commits intoenvoyproxy:masterfrom
euroelessar:buffer-content-length
Aug 7, 2019
Merged

buffer filter: Populate content-length header#7848
mattklein123 merged 3 commits intoenvoyproxy:masterfrom
euroelessar:buffer-content-length

Conversation

@euroelessar
Copy link
Copy Markdown
Contributor

Description: Make buffer filter populate content-length header if it is not present in the request already. As buffer has to fetch the whole request in order to proceed, calculation of the header value involves mostly no extra cost.
Risk Level: MEDIUM
Testing: added unit and integration tests
Docs Changes: added section about new behavior around content-length header
Release Notes: updated
Fixes #7844

Ruslan Nigmatullin added 2 commits August 7, 2019 00:30
Signed-off-by: Ruslan Nigmatullin <elessar@dropbox.com>
Signed-off-by: Ruslan Nigmatullin <elessar@dropbox.com>
@mattklein123 mattklein123 self-assigned this Aug 7, 2019
Copy link
Copy Markdown
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM with some small comments.

/wait

* access log: added :ref:`buffering <envoy_api_field_config.accesslog.v2.CommonGrpcAccessLogConfig.buffer_size_bytes>` and :ref:`periodical flushing <envoy_api_field_config.accesslog.v2.CommonGrpcAccessLogConfig.buffer_flush_interval>` support to gRPC access logger. Defaults to 16KB buffer and flushing every 1 second.
* admin: added ability to configure listener :ref:`socket options <envoy_api_field_config.bootstrap.v2.Admin.socket_options>`.
* admin: added config dump support for Secret Discovery Service :ref:`SecretConfigDump <envoy_api_msg_admin.v2alpha.SecretsConfigDump>`.
* buffer filter: the buffer filter populates content-length header if not present, behavior can be disabled using the runtime feature `envoy.reloadable_features.buffer_populate_content_length`.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would call this envoy.reloadable_features.buffer_filter_populate_content_length. Also, can you refer to this in the fitler docs also?

Http::FilterDataStatus BufferFilter::decodeData(Buffer::Instance& data, bool end_stream) {
content_length_ += data.length();
if (end_stream || settings_->disabled()) {
if (request_headers_ != nullptr && request_headers_->ContentLength() == nullptr) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that request_headers_ being non-null is guarded by disabled so the if statement above is OK, but can you add a comment about this? It's a little non-obvious. You could potentially also ASSERT(!settings_->disabled())

cr
Signed-off-by: Ruslan Nigmatullin <elessar@dropbox.com>
Copy link
Copy Markdown
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mattklein123 mattklein123 merged commit 4c08c00 into envoyproxy:master Aug 7, 2019
@htuch
Copy link
Copy Markdown
Member

htuch commented Aug 9, 2019

@euroelessar we suspect this may have introduced a regression in the execution time for buffer_filter_integration_test; could you please take a look? CC @jmarantz

@jmarantz
Copy link
Copy Markdown
Contributor

jmarantz commented Aug 9, 2019

Specifically, Protocols/BufferIntegrationTest.RouterRequestPopulateContentLength/IPv4_HttpDownstream_HttpUpstream takes 15+ seconds and most of it I think is in response->waitForEndStream()

Running with --gtest_filter=\*BufferIntegrationTest.RouterRequestPopulateContentLength\* executes 8 variants of the same test and consequently takes 120 seconds.

@mattklein123
Copy link
Copy Markdown
Member

Possibly related to #7872

@jmarantz
Copy link
Copy Markdown
Contributor

jmarantz commented Aug 9, 2019

Appears to be fixed by #7885

junr03 added a commit that referenced this pull request Oct 22, 2019
…8609)

Description: #7848 introduced content-length population in the buffer filter. However, it is only populated if the client stream ends on a data frame. This PR adds content-length population when closing the stream via trailers.
Risk Level: low - expanded functionality runtime guarded.
Testing: unit and integration tests.
Release Notes: I believe the current release notes added by #7848 are still adequate.

Signed-off-by: Jose Nino <jnino@lyft.com>
derekargueta pushed a commit to derekargueta/envoy that referenced this pull request Oct 24, 2019
…nvoyproxy#8609)

Description: envoyproxy#7848 introduced content-length population in the buffer filter. However, it is only populated if the client stream ends on a data frame. This PR adds content-length population when closing the stream via trailers.
Risk Level: low - expanded functionality runtime guarded.
Testing: unit and integration tests.
Release Notes: I believe the current release notes added by envoyproxy#7848 are still adequate.

Signed-off-by: Jose Nino <jnino@lyft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

buffer: Synthesize content-length header in case of request buffering

4 participants