Skip to content

Correctly return NEED_WRAP if we produced some data even if we could …#10396

Merged
normanmaurer merged 1 commit into4.1from
buffer_overflow_fix
Jul 9, 2020
Merged

Correctly return NEED_WRAP if we produced some data even if we could …#10396
normanmaurer merged 1 commit into4.1from
buffer_overflow_fix

Conversation

@normanmaurer
Copy link
Copy Markdown
Member

…not consume any during SSLEngine.wrap(...)

Motivation:

At the moment we may report BUFFER_OVERFLOW when wrap(...) fails to consume data but still prodce some. This is not correct and we should better report NEED_WRAP as we already have produced some data (for example tickets). This way the user will try again without increasing the buffer size which is more correct and may reduce the number of allocations

Modifications:

Return NEED_WRAP when we produced some data but not consumed any.

Result:

Fix ReferenceCountedOpenSslEngine.wrap(...) state machine

…not consume any during SSLEngine.wrap(...)

Motivation:

At the moment we may report BUFFER_OVERFLOW when wrap(...) fails to consume data but still prodce some. This is not correct and we should better report NEED_WRAP as we already have produced some data (for example tickets). This way the user will try again without increasing the buffer size which is more correct and may reduce the number of allocations

Modifications:

Return NEED_WRAP when we produced some data but not consumed any.

Result:

Fix ReferenceCountedOpenSslEngine.wrap(...) state machine
@normanmaurer
Copy link
Copy Markdown
Member Author

Unfortunally I was only able to trigger this with my work for session resumption and TLS1.3 + tickets. That said this bug exists also without this work so I thought we should better do a separate PR to allow users to better understand what was fixed and why.

@normanmaurer
Copy link
Copy Markdown
Member Author

This was observed while working on #10331

@normanmaurer normanmaurer requested review from chrisvest and ejona86 July 8, 2020 20:14
Copy link
Copy Markdown
Member

@carl-mastrangelo carl-mastrangelo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@normanmaurer normanmaurer merged commit cb9d4a1 into 4.1 Jul 9, 2020
@normanmaurer normanmaurer deleted the buffer_overflow_fix branch July 9, 2020 06:52
normanmaurer added a commit that referenced this pull request Jul 9, 2020
…not consume any during SSLEngine.wrap(...) (#10396)

Motivation:

At the moment we may report BUFFER_OVERFLOW when wrap(...) fails to consume data but still prodce some. This is not correct and we should better report NEED_WRAP as we already have produced some data (for example tickets). This way the user will try again without increasing the buffer size which is more correct and may reduce the number of allocations

Modifications:

Return NEED_WRAP when we produced some data but not consumed any.

Result:

Fix ReferenceCountedOpenSslEngine.wrap(...) state machine
@normanmaurer normanmaurer added this to the 4.1.51.Final milestone Jul 9, 2020
Kvicii pushed a commit to Kvicii/netty that referenced this pull request Jul 10, 2020
* '4.1' of github.com:netty/netty:
  [maven-release-plugin] prepare for next development iteration
  [maven-release-plugin] prepare release netty-4.1.51.Final
  Correctly return NEED_WRAP if we produced some data even if we could not consume any during SSLEngine.wrap(...) (netty#10396)
  Modify OpenSSL native library loading to accommodate GraalVM (netty#10395)
Kvicii pushed a commit to Kvicii/netty that referenced this pull request Jul 20, 2020
* '4.1-read' of github.com:Kvicii/netty: (43 commits)
  Make the TLSv1.3 check more robust and not depend on the Java version… (netty#10409)
  Reduce the scope of synchronized block in PoolArena (netty#10410)
  Add IndexOutOfBoundsException error message (netty#10405)
  Add default handling for switch statement (netty#10408)
  Review PooledByteBufAllocator in respect of jemalloc 4.x changes and update allocate algorithm.(netty#10267)
  Support session cache for client and server when using native SSLEngine implementation (netty#10331)
  Simple fix typo (netty#10403)
  Eliminate a redundant method call in HpackDynamicTable.add(...) (netty#10399)
  jdk.tls.client.enableSessionTicketExtension must be respected by OPENSSL and OPENSSL_REFCNT SslProviders (netty#10401)
  重新编译4.1分支
  [maven-release-plugin] prepare for next development iteration
  [maven-release-plugin] prepare release netty-4.1.51.Final
  Correctly return NEED_WRAP if we produced some data even if we could not consume any during SSLEngine.wrap(...) (netty#10396)
  Modify OpenSSL native library loading to accommodate GraalVM (netty#10395)
  Update to netty-tcnative 2.0.31.Final and make SslErrorTest more robust (netty#10392)
  Add option to HttpObjectDecoder to allow duplicate Content-Lengths (netty#10349)
  Add detailed error message corresponding to the IndexOutOfBoundsException while calling getEntry(...) (netty#10386)
  Do not report ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify as blocking call (netty#10387)
  Add unit test for HpackDynamicTable. (netty#10389)
  netty用于测试的内嵌Channel | ChannelInboundHandler主要方法
  ...
ihanyong pushed a commit to ihanyong/netty that referenced this pull request Jul 31, 2020
…not consume any during SSLEngine.wrap(...) (netty#10396)

Motivation:

At the moment we may report BUFFER_OVERFLOW when wrap(...) fails to consume data but still prodce some. This is not correct and we should better report NEED_WRAP as we already have produced some data (for example tickets). This way the user will try again without increasing the buffer size which is more correct and may reduce the number of allocations

Modifications:

Return NEED_WRAP when we produced some data but not consumed any.

Result:

Fix ReferenceCountedOpenSslEngine.wrap(...) state machine
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants