Skip to content

gzip: make use of generalized compression filter#10306

Merged
mattklein123 merged 4 commits intoenvoyproxy:masterfrom
rojkov:gzip
Mar 10, 2020
Merged

gzip: make use of generalized compression filter#10306
mattklein123 merged 4 commits intoenvoyproxy:masterfrom
rojkov:gzip

Conversation

@rojkov
Copy link
Copy Markdown
Member

@rojkov rojkov commented Mar 9, 2020

Description:
This is the second part of the refactoring started in #7057. The GzipFilter class is removed completely and generalized CompressionFilter is instantiated instead by GzipFilterFactory. Instances of `CompressionFilter are configured to use "gzip" compressors to deflate HTTP streams.

Risk Level: Medium
Testing: unit tests, manual tests
Docs Changes: updated docs/root/configuration/http_filters/gzip_filter.rst
Release Notes: N/A

Deprecated: the statistics counter <stat_prefix>.gzip.header_gzip is deprecated in favor of <stat_prefix>.gzip.header_compressor_used provided by the generalized compression filter (the meaning stays the same). Also a new counter <stat_prefix>.gzip.header_compressor_overshadowed is added which contains a number of requests skipped by this filter instance because they were handled by another compression filter in the same filter chain.

Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
@rojkov rojkov requested a review from dio as a code owner March 9, 2020 15:01
@repokitteh-read-only
Copy link
Copy Markdown

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to api/.

🐱

Caused by: #10306 was opened by rojkov.

see: more, trace.

@mattklein123 mattklein123 self-assigned this Mar 9, 2020
Copy link
Copy Markdown
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with some small comments. Thank you!

// .. attention:
//
// **This field is deprecated**. Set the `compressor` field instead.
google.protobuf.UInt32Value content_length = 2 [deprecated = true];
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update https://www.envoyproxy.io/docs/envoy/latest/intro/deprecated. Also are we going to deprecate the standalone gzip filter entirely? Perhaps that should be done in a different change since I think doing the warning/stat dance for that deprecation will not be built-in?cc @zuercher who has been working on this recently.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also are we going to deprecate the standalone gzip filter entirely?

The filter itself isn't deprecated. Our users shouldn't notice any changes in its functionality except the new deprecation warning on using content_length and other deprecated config fields.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But why would we keep the gzip filter when there is the generic compression filter? Should we deprecate the gzip filter and have users migrate to using the generic compression filter?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generic filter is not exposed to users directly. The idea was to expose it through a new config field of the existing "gzip" filter. Then to introduce a new filter - "brotli" - exposing the same field.

...
  http_filters:
  - name: envoy.filters.http.gzip
    config:
      compression_level: speed
      compressor:
         remove_accept_encoding_header: true
  - name: envoy.filters.http.brotli
    config:
      quality: 12
      compressor:
         remove_accept_encoding_header: true

This way users would do the transition seamlessly.

If I understood correctly you wanted to keep "gzip" as it was and to expose the generic filter as a normal HTTP filter parameterized with a compression lib config:

...
  http_filters:
  - name: envoy.filters.http.compress
    config:
      remove_accept_encoding_header: true
      compressor_library:
      - name: gzip
        config:
          compression_level: speed
  - name: envoy.filters.http.compress
    config:
      remove_accept_encoding_header: true
      compressor_library:
      - name: brotli
        config:
          quality: 12

This way users may experience unexpected results when both the old gzip and the new generic filter end up in the same filter chain for some reason.

But since this PR has been merged already the gzip filter is capable of cooperation with other compression filters now and users won't get unexpected results.

Still I need to know which way is preferable.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I would definitely prefer the 2nd option, as it's more extensible for the future. I think we should add that, allow the generic compression filter to support gzip, and then deprecate the gzip filter?

- text/html
- application/json
disable_on_etag_header: true
compressor:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a DEPRECATED_FEATURE_TEST that tests the old path.

@dio
Copy link
Copy Markdown
Member

dio commented Mar 10, 2020

This is awesome!

Dmitry Rozhkov added 2 commits March 10, 2020 13:28
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
@rojkov
Copy link
Copy Markdown
Member Author

rojkov commented Mar 10, 2020

Thanks! I've updated the doc and the integration test.

Copy link
Copy Markdown
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! We can ship and iterate on the full gzip filter deprecation.

@mattklein123 mattklein123 merged commit c85bf32 into envoyproxy:master Mar 10, 2020
@rojkov rojkov deleted the gzip branch March 11, 2020 09:23
@rojkov
Copy link
Copy Markdown
Member Author

rojkov commented Mar 11, 2020 via email

@rgs1
Copy link
Copy Markdown
Member

rgs1 commented Mar 18, 2020

This broke the gzip.filter_enabled runtime flag for disabling the gzip filter. Shall I fix that or just move to the compression filter (although not sure it's ready)?

@rgs1
Copy link
Copy Markdown
Member

rgs1 commented Mar 18, 2020

I guess the answer depends on how long it'll take to deprecate the gzip filter...

@rgs1
Copy link
Copy Markdown
Member

rgs1 commented Mar 18, 2020

Nevermind, noticed a compressor field was added so I can fix it via config as pointed out offline by Matt (it's still breakage, but very minor).

@rgs1
Copy link
Copy Markdown
Member

rgs1 commented Mar 18, 2020

Related fix while I was reading that code:

#10446

cc: @rojkov

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants