Skip to content

Adaptive: Fix concurrency issue in adaptive allocator (#16767)#16778

Merged
chrisvest merged 1 commit into
netty:4.1from
chrisvest:4.1-adaptive-fix
May 9, 2026
Merged

Adaptive: Fix concurrency issue in adaptive allocator (#16767)#16778
chrisvest merged 1 commit into
netty:4.1from
chrisvest:4.1-adaptive-fix

Conversation

@chrisvest

Copy link
Copy Markdown
Member

Motivation:
We have a few code paths where Chunk.remainingCapacity() is called without regard for exclusive access to the given chunk. The BuddyChunk implementation unfortunately drained the freeList in this call, leading to racy modifications of the buddies array, that in turn can manifest as unpredictable bugs.

Modification:
Make the BuddyChunk.remainingCapacity non-modifying. Instead of draining the freeList, we sum up the capacity held by it and add it to the capacity accounted for in the Chunk super class.

This operation needs a weakPeekReduce method on the MpscIntQueue. This method performs a reduction operation on a given range of entries, but without removing entries or incrementing the consumer index.

A regression test is added, which successfully captured the issue before it was fixed.

Also did a few test cleanups.

Result:
No more corruptions in the buddies array.

This fixes the rare occurrences of assertion errors like the following:

java.lang.AssertionError
	at io.netty.buffer.AdaptivePoolingAllocator$Magazine.allocate(AdaptivePoolingAllocator.java:916)
	at io.netty.buffer.AdaptivePoolingAllocator$Magazine.tryAllocate(AdaptivePoolingAllocator.java:854)
	at io.netty.buffer.AdaptivePoolingAllocator$MagazineGroup.allocate(AdaptivePoolingAllocator.java:421)
	at io.netty.buffer.AdaptivePoolingAllocator.allocate(AdaptivePoolingAllocator.java:271)

(cherry picked from commit bd866c3)

Motivation:
We have a few code paths where `Chunk.remainingCapacity()` is called
without regard for exclusive access to the given chunk. The `BuddyChunk`
implementation unfortunately drained the `freeList` in this call,
leading to racy modifications of the `buddies` array, that in turn can
manifest as unpredictable bugs.

Modification:
Make the `BuddyChunk.remainingCapacity` non-modifying. Instead of
draining the `freeList`, we sum up the capacity held by it and add it to
the capacity accounted for in the `Chunk` super class.

This operation needs a `weakPeekReduce` method on the `MpscIntQueue`.
This method performs a reduction operation on a given range of entries,
but without removing entries or incrementing the consumer index.

A regression test is added, which successfully captured the issue before
it was fixed.

Also did a few test cleanups.

Result:
No more corruptions in the buddies array.

This fixes the rare occurrences of assertion errors like the following:

```
java.lang.AssertionError
	at io.netty.buffer.AdaptivePoolingAllocator$Magazine.allocate(AdaptivePoolingAllocator.java:916)
	at io.netty.buffer.AdaptivePoolingAllocator$Magazine.tryAllocate(AdaptivePoolingAllocator.java:854)
	at io.netty.buffer.AdaptivePoolingAllocator$MagazineGroup.allocate(AdaptivePoolingAllocator.java:421)
	at io.netty.buffer.AdaptivePoolingAllocator.allocate(AdaptivePoolingAllocator.java:271)
```

(cherry picked from commit bd866c3)
@chrisvest chrisvest added this to the 4.1.134.Final milestone May 8, 2026
@chrisvest chrisvest enabled auto-merge (squash) May 8, 2026 22:56
@chrisvest chrisvest disabled auto-merge May 9, 2026 21:53
@chrisvest chrisvest merged commit 7fc2dcb into netty:4.1 May 9, 2026
17 of 18 checks passed
@chrisvest chrisvest deleted the 4.1-adaptive-fix branch May 9, 2026 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant