-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Enable gzip for broker web service #234
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
We had this long time back. Eventually we turned off compression because compressing stats was generating a log GC pressures. I'd be ok to put this with a config switch to disable it. |
|
Can we keep the default to off? |
|
@merlimat have you ever considered moving the WebService to a netty backed http server implementation? |
|
@sschepens Yes, I actually thought of that long time back :). Main problem is that we have a big number of REST handlers already in place and most of them are not really perf sensitive or called that much frequent. So it would be a huge effort to port them all. |
Fixes apache#208 This PR ports Kafka's `SaslServerAuthenticator` into KoP with some limits: - Don't handle authentication corner cases for the Kafka Client 0.9.0.x, it's too troublesome and KoP doesn't support Kafka Client 0.9.0.x now. - Don't handle `SaslHandshakeRequest` v0 because this request has no header that it takes a lot of efforts to refactor current code and test. - Keep the old authentication way, i.e. user pass a `username` field to represent the namespace and a `data` field to represent the token, then Pulsar broker will use JWT authentication to do the actual authentication. See `docs/security.md` for details. - Add a unit test to cover the case that client doesn't configure authentication. More works to do: - Currently it just does a simple check for permissions in the authentication step. The authorization should be performed before each request is processed. - The authenticator exposes a getter for `AuthenticationState`, the `isExpired` method should be called to check if the token is expired and trigger the reconnection for clients.
|
This PR is evidently stale or abandoned. Reopen if this is not so. |
|
@sschepens:Thanks for your contribution. For this PR, do we need to update docs? |
…flows Fixes: apache#24790 PIP: apache#234 Motivation Authentication implementations such as AuthenticationOAuth2 create their own thread pools, which prevents resource sharing across multiple Pulsar client instances. Since PIP-234 adds support for sharing resources like Netty EventLoopGroup, we need to extend this capability to authentication flows to avoid unnecessary resource duplication. Modifications Added AuthenticationInitContext interface in the public API to pass shared resources to authentication providers Implemented AuthenticationInitContextImpl with service registry for shared resources Extended Authentication interface with new start(AuthenticationInitContext) method (backward compatible) Modified PulsarClientImpl to create context and register shared resources (EventLoopGroup, Timer) Updated AuthenticationOAuth2 to pass context to OAuth2 flows Extended Flow interface in OAuth2 with context-aware initialization Modified FlowBase to use shared EventLoopGroup and Timer from context for AsyncHttpClient Key changes enable OAuth2 flows to reuse: Shared Netty EventLoopGroup Shared Timer Verifying this change This change is already covered by existing tests, such as: OAuth2 authentication tests Client connection tests Added AuthenticationResourceSharingTest for new context functionality Does this pull request potentially affect one of the following parts: The public API (adds AuthenticationInitContext interface and extends Authentication interface) Dependencies (add or upgrade a dependency) The schema The default values of configurations The threading model (enables sharing, doesn't change model) The binary protocol The REST endpoints The admin CLI options The metrics Anything that affects deployment Documentation doc-required Future Work Potential extension to other authentication methods Additional resource types can be added to the context as needed Shared DNS resolver/cache
Enable gzip compression on broker WebService, this would mostly benefit stats requests as they are very heavy and JSON is really compressible.