Auto-port 4.1: Add maxWindowLog parameter to ZstdDecoder to bound memory allocation#16894
Merged
Conversation
…16850) Motivation: The ZstdDecoder does not currently constrain the memory the underlying zstd decoder may allocate per stream. RFC 8478 ("Security Considerations") states: An attacker may provide correctly formed compressed frames with unreasonable memory requirements. A decoder must always control memory requirements and enforce some (system-specific) limits in order to protect memory usage from such scenarios. The most common instance of this attack is a tiny (a few hundred bytes) Zstandard frame whose header declares a very large Window_Size — for example Window_Log = 31, a 2 GiB sliding window. When such a frame is fed to libzstd, the declared window is allocated as native memory before any actual content is decoded, so a handful of concurrent connections is enough to drive a server into OOM. This is exactly the class of "unreasonable memory requirements" calls out. The libzstd manual exposes a parameter (ZSTD_d_windowLogMax) specifically to bound this, and zstd-jni surfaces it via ZstdInputStreamNoFinalizer.setLongMax(int). The fix is to wire it up in ZstdDecoder with a sensible default. Note that the existing maximumAllocationSize parameter only caps the Netty-side output buffer handed to the next handler; it does not bound the native window memory libzstd allocates, so the attack surface remained open prior to this change. Modification: Add a new public constant DEFAULT_MAX_WINDOW_LOG = 27 (128 MiB window), a reasonable default for general-purpose server use that still leaves significant headroom for typical zstd CLI output (whose default Window_Log is ≤ 23). Add a new constructor ZstdDecoder(int maximumAllocationSize, int maxWindowLog). maxWindowLog is validated to be in [10, 31] per RFC 8478. The existing ZstdDecoder(int maximumAllocationSize) constructor now delegates to the new one with DEFAULT_MAX_WINDOW_LOG, so the default behavior of existing call sites is preserved at the API level while no longer being vulnerable to the attack above. In handlerAdded, after the existing setContinuous(true), call zstdIs.setLongMax(maxWindowLog). Frames whose Window_Log exceeds the configured cap are rejected by libzstd with a ZstdIOException("Frame requires too much memory for decoding"), which the existing catch (Exception e) path wraps into a DecompressionException and transitions the handler to CORRUPTED. Result: Fixes #. ZstdDecoder now controls the per-stream native window memory libzstd will allocate, addressing the "unreasonable memory requirements" scenario in RFC 8478. Users who legitimately need to accept frames with very large windows can opt in via the new constructor with a larger maxWindowLog. --------- Co-authored-by: Chris Vest <christianvest_hansen@apple.com> (cherry picked from commit 0bd1657)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-port of #16850 to 4.1
Cherry-picked commit: 0bd1657
Motivation:
The ZstdDecoder does not currently constrain the memory the underlying zstd decoder may allocate per stream. RFC 8478 ("Security Considerations") states:
An attacker may provide correctly formed compressed frames with unreasonable memory requirements. A decoder must always control memory requirements and enforce some (system-specific) limits in order to protect memory usage from such scenarios.
The most common instance of this attack is a tiny (a few hundred bytes) Zstandard frame whose header declares a very large Window_Size — for example Window_Log = 31, a 2 GiB sliding window. When such a frame is fed to libzstd, the declared window is allocated as native memory before any actual content is decoded, so a handful of concurrent connections is enough to drive a server into OOM. This is exactly the class of "unreasonable memory requirements" calls out.
The libzstd manual exposes a parameter (ZSTD_d_windowLogMax) specifically to bound this, and zstd-jni surfaces it via ZstdInputStreamNoFinalizer.setLongMax(int). The fix is to wire it up in ZstdDecoder with a sensible default. Note that the existing maximumAllocationSize parameter only caps the Netty-side output buffer handed to the next handler; it does not bound the native window memory libzstd allocates, so the attack surface remained open prior to this change.
Modification:
Add a new public constant DEFAULT_MAX_WINDOW_LOG = 27 (128 MiB window), a reasonable default for general-purpose server use that still leaves significant headroom for typical zstd CLI output (whose default Window_Log is ≤ 23).
Add a new constructor ZstdDecoder(int maximumAllocationSize, int maxWindowLog). maxWindowLog is validated to be in [10, 31] per RFC 8478.
The existing ZstdDecoder(int maximumAllocationSize) constructor now delegates to the new one with DEFAULT_MAX_WINDOW_LOG, so the default behavior of existing call sites is preserved at the API level while no longer being vulnerable to the attack above.
In handlerAdded, after the existing setContinuous(true), call zstdIs.setLongMax(maxWindowLog). Frames whose Window_Log exceeds the configured cap are rejected by libzstd with a ZstdIOException("Frame requires too much memory for decoding"), which the existing catch (Exception e) path wraps into a DecompressionException and transitions the handler to CORRUPTED.
Result:
Fixes #.
ZstdDecoder now controls the per-stream native window memory libzstd will allocate, addressing the "unreasonable memory requirements" scenario in RFC 8478. Users who legitimately need to accept frames with very large windows can opt in via the new constructor with a larger maxWindowLog.