Skip to content

Use allocator when constructing ByteBufHolder sub-types or use Unpool…#9377

Merged
normanmaurer merged 3 commits into4.1from
alloc_buffer
Jul 18, 2019
Merged

Use allocator when constructing ByteBufHolder sub-types or use Unpool…#9377
normanmaurer merged 3 commits into4.1from
alloc_buffer

Conversation

@normanmaurer
Copy link
Copy Markdown
Member

…ed.EMPTY_BUFFER where possible

Motivation:

In many places Netty uses Unpooled.buffer(0) while should use EMPTY_BUFFER. We can't change this due to back compatibility in the constructors but can use Unpooled.EMPTY_BUFFER in some cases to ensure we not allocate at all. In others we can directly use the allocator either from the Channel / ChannelHandlerContext or the request / response.

Modification:

  • Use Unpooled.EMPTY_BUFFER where possible
  • Use allocator where possible

Result:

Fixes #9345 for websockets and http package

…ed.EMPTY_BUFFER where possible

Motivation:

In many places Netty uses Unpooled.buffer(0) while should use EMPTY_BUFFER. We can't change this due to back compatibility in the constructors but can use Unpooled.EMPTY_BUFFER in some cases to ensure we not allocate at all. In others we can directly use the allocator either from the Channel / ChannelHandlerContext or the request / response.

Modification:

- Use Unpooled.EMPTY_BUFFER where possible
- Use allocator where possible

Result:

Fixes #9345 for websockets and http package
@normanmaurer
Copy link
Copy Markdown
Member Author

@doom369 PTAL as an alternative fix.

* @param data The data to encode
* @return An encoded string containing the data
*/
@SuppressJava6Requirement(reason = "Guarded with java version check")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow, I didn't know you can do that :)

@doom369
Copy link
Copy Markdown
Contributor

doom369 commented Jul 16, 2019

@normanmaurer as cleanup it is ok. However, it doesn't fully fixes #9345. For example, WebSocketUtil.base64 during handshake uses wrapped buffer so for java < 8 it will be still the issue. Or creating any of WebSocketFrame instance, for example PingWebSocketFrame will cause the same.

@normanmaurer
Copy link
Copy Markdown
Member Author

@doom369 I agree it doesn't 100 % fix all of it but I think for a patch release it is an acceptable tradeoff... I am not sure I understand why using a wrapped buffer is a problem tho.

@doom369
Copy link
Copy Markdown
Contributor

doom369 commented Jul 16, 2019

Hm, you are right. Not sure why I dig into it.

@normanmaurer
Copy link
Copy Markdown
Member Author

@netty-bot test this please

@normanmaurer
Copy link
Copy Markdown
Member Author

@doom369 so what you think about this as "first step" ?

@doom369
Copy link
Copy Markdown
Contributor

doom369 commented Jul 17, 2019

@normanmaurer looks good. This PR should fix heap allocation for our flow. If I didn't miss anything.

@normanmaurer normanmaurer added this to the 4.1.38.Final milestone Jul 17, 2019
@normanmaurer
Copy link
Copy Markdown
Member Author

@doom369 so can you approve ?

@normanmaurer normanmaurer merged commit 1e8c0c5 into 4.1 Jul 18, 2019
@normanmaurer normanmaurer deleted the alloc_buffer branch July 18, 2019 08:29
normanmaurer added a commit that referenced this pull request Jul 18, 2019
#9377)

Motivation:

In many places Netty uses Unpooled.buffer(0) while should use EMPTY_BUFFER. We can't change this due to back compatibility in the constructors but can use Unpooled.EMPTY_BUFFER in some cases to ensure we not allocate at all. In others we can directly use the allocator either from the Channel / ChannelHandlerContext or the request / response.

Modification:

- Use Unpooled.EMPTY_BUFFER where possible
- Use allocator where possible

Result:

Fixes #9345 for websockets and http package
Copy link
Copy Markdown
Member

@njhill njhill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

buffer.release();

ByteBuf buffer = ctx.alloc().buffer(buf.length());
buffer.writeCharSequence(buf.toString(), CharsetUtil.UTF_8);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can pass StringBuilder directly

@amizurov amizurov mentioned this pull request Nov 5, 2019
normanmaurer pushed a commit that referenced this pull request Nov 6, 2019
Motivation:

After fix #9377 some websocket examples work incorrect

Modification:

Replace `Unpooled.EMPTY_BUFFER` to `ctx.alloc().buffer(0)` for responses with possible content

Result:

Examples work
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Netty still uses heap buffers while preferDirect is true

3 participants