HTTP/2: Prevent memory leak when trying to create new streams on a connection that received a GOAWAY.#9674
Merged
normanmaurer merged 3 commits intonetty:4.1from Oct 17, 2019
Conversation
…nnection that received a GOAWAY. Motivation: In netty#8692, `Http2FrameCodec` was updated to keep track of all "being initialized" streams, allocating memory before initialization begins, and releasing memory after initialization completes successfully. In some instances where stream initialization fails (e.g. because this connection has received a GOAWAY frame), this memory is never released. Modifications: This change updates the `Http2FrameCodec` to use a separate promise for monitoring the success of sending HTTP2 headers. When sending of headers fails, we now make sure to release memory allocated for stream initialization. Result: After this change, failures in writing HTTP2 Headers (e.g. because this connection has received a GOAWAY frame) will no longer leak memory.
|
Can one of the admins verify this patch? |
normanmaurer
requested changes
Oct 16, 2019
Member
normanmaurer
left a comment
There was a problem hiding this comment.
A few nits but generally LGTM!
codec-http2/src/test/java/io/netty/handler/codec/http2/Http2FrameCodecTest.java
Outdated
Show resolved
Hide resolved
codec-http2/src/main/java/io/netty/handler/codec/http2/Http2FrameCodec.java
Outdated
Show resolved
Hide resolved
codec-http2/src/main/java/io/netty/handler/codec/http2/Http2FrameCodec.java
Outdated
Show resolved
Hide resolved
codec-http2/src/main/java/io/netty/handler/codec/http2/Http2FrameCodec.java
Show resolved
Hide resolved
Member
|
@millems can you please also sign our icla: https://netty.io/s/icla |
Member
|
@netty-bot test this please |
Member
|
@millems probably I'm missing something but why is a separate promise needed? Couldn't we just add a listener to the existing promise? |
Contributor
Author
|
@njhill Using a separate promise was intended to be defensive, and only performing the cleanup if the send-headers failed. I was concerned about the following scenario:
Let me know if this concern is unfounded, and I can remove the additional future. |
Contributor
Author
|
@normanmaurer Will review comments and address. CLA was reviewed and signed yesterday. Let me know if it's not on record and I can go again. |
Member
Member
|
@millems all good... I somehow missed it 🤦♂ |
1. Assert channel.finishAndReleaseAll() returns false. 2. Attach failure detection to original future, instead of a separate future. 3. Use ChannelListener instead of GenericFutureListener 4. Note that numInitializingStreams is for testing.
Contributor
Author
|
Addressed change requests. |
normanmaurer
requested changes
Oct 16, 2019
codec-http2/src/main/java/io/netty/handler/codec/http2/Http2FrameCodec.java
Outdated
Show resolved
Hide resolved
Member
|
@netty-bot test this please |
Member
|
@netty-bot test this please |
normanmaurer
approved these changes
Oct 17, 2019
Member
normanmaurer
left a comment
There was a problem hiding this comment.
@millems thanks a lot for your work... :)
normanmaurer
pushed a commit
that referenced
this pull request
Oct 17, 2019
…nnection that received a GOAWAY. (#9674) Motivation: In #8692, `Http2FrameCodec` was updated to keep track of all "being initialized" streams, allocating memory before initialization begins, and releasing memory after initialization completes successfully. In some instances where stream initialization fails (e.g. because this connection has received a GOAWAY frame), this memory is never released. Modifications: This change updates the `Http2FrameCodec` to use a separate promise for monitoring the success of sending HTTP2 headers. When sending of headers fails, we now make sure to release memory allocated for stream initialization. Result: After this change, failures in writing HTTP2 Headers (e.g. because this connection has received a GOAWAY frame) will no longer leak memory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation:
In #8692,
Http2FrameCodecwas updated to keep track of all "being initialized" streams, allocating memory before initialization begins, and releasing memory after initialization completes successfully.In some instances where stream initialization fails (e.g. because this connection has received a GOAWAY frame), this memory is never released.
Modifications:
This change updates the
Http2FrameCodecto use a separate promise for monitoring the success of sending HTTP2 headers. When sending of headers fails, we now make sure to release memory allocated for stream initialization.Result:
After this change, failures in writing HTTP2 Headers (e.g. because this connection has received a GOAWAY frame) will no longer leak memory.