ByteBufAllocatorMetric.usedDirectMemory reports low values #16059

BaurzhanSakhariev · 2025-12-17T11:15:22Z

BaurzhanSakhariev
Dec 17, 2025

Hi, I have simple test that writes some bytes to a direct buffer.
Idea is to use allocator.metric().usedDirectMemory() and try to short circuit the operation if used direct memory is about to exceed the limit.

Netty version: 4.2.9.Final

// Run it with -Xmx80m (direct will be similar, Runtime.getRuntime().maxMemory())
// PooledByteBufAllocator allocator = new PooledByteBufAllocator(true);
AdaptiveByteBufAllocator allocator = new AdaptiveByteBufAllocator(true);
ByteBuf buf = allocator.buffer();
assertThat(buf.isDirect()).isTrue();

// I run this with MaxDirectMemorySize not specified, 
// see https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/jdk/internal/misc/VM.java#L138
long directMemoryLimit = Runtime.getRuntime().maxMemory();

byte[] data = new byte[5000];
Arrays.fill(data, (byte) 1);

while (true) {
    if (allocator.metric().usedDirectMemory() + data.length > directMemoryLimit) {
        throw new RuntimeException("OOM has been prevented, used memory reporting is precise.");
    }
    logger.info(
        "used direct :{}, used heap: {}, writing: {}, limit: {}",
        allocator.metric().usedDirectMemory(),
        allocator.metric().usedHeapMemory(),
        data.length,
        directMemoryLimit
    );
    buf.writeBytes(data);
}

I'm getting OOM-s with both allocators

PooledByteBufAllocator

java.lang.OutOfMemoryError: Cannot reserve 37748736 bytes of direct buffer memory (allocated: 50331649, limit: 81133568)

	at __randomizedtesting.SeedInfo.seed([4EA583D51E5D8AD3:9A07F15EAC7E0677]:0)
	at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:108)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:367)
	at io.netty.util.internal.CleanerJava9.allocate(CleanerJava9.java:86)
	at io.netty.util.internal.PlatformDependent.allocateDirect(PlatformDependent.java:600)
	at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:767)
	at io.netty.buffer.PoolArena$DirectArena.newUnpooledChunk(PoolArena.java:754)
	at io.netty.buffer.PoolArena.allocateHuge(PoolArena.java:230)
	at io.netty.buffer.PoolArena.allocate(PoolArena.java:146)
	at io.netty.buffer.PoolArena.reallocate(PoolArena.java:328)
	at io.netty.buffer.PooledByteBuf.capacity(PooledByteBuf.java:126)
	at io.netty.buffer.AbstractByteBuf.ensureWritable0(AbstractByteBuf.java:305)
	at io.netty.buffer.AbstractByteBuf.ensureWritable(AbstractByteBuf.java:280)
	at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1080)
	at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1088)
	at io.netty.buffer.WrappedByteBuf.writeBytes(WrappedByteBuf.java:803)
	at io.netty.buffer.AdvancedLeakAwareByteBuf.writeBytes(AdvancedLeakAwareByteBuf.java:618)

PooledByteBufAllocator logs

 used direct :8388608, used heap: 0, writing: 5000, limit: 81133568
.. many same entries 

 used direct :12582912, used heap: 0, writing: 5000, limit: 81133568
 ...many similar entries with "used direct" increasing to 4MB from time to time.
 ...
 
// last reported before OOM
used direct :37748736, used heap: 0, writing: 5000, limit: 81133568

Note that allocated: 50331649 in the OOM error is 12MB higher then last reported in logs allocator.metric().usedDirectMemory()

AdaptiveByteBufAllocator

java.lang.OutOfMemoryError: Cannot reserve 37748736 bytes of direct buffer memory (allocated: 50200577, limit: 81133568)

	at __randomizedtesting.SeedInfo.seed([4EA583D51E5D8AD3:9A07F15EAC7E0677]:0)
	at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:108)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:367)
	at io.netty.util.internal.CleanerJava9.allocate(CleanerJava9.java:86)
	at io.netty.util.internal.PlatformDependent.allocateDirect(PlatformDependent.java:600)
	at io.netty.buffer.UnpooledDirectByteBuf.allocateDirectBuffer(UnpooledDirectByteBuf.java:138)
	at io.netty.buffer.UnpooledDirectByteBuf.<init>(UnpooledDirectByteBuf.java:81)
	at io.netty.buffer.UnpooledUnsafeDirectByteBuf.<init>(UnpooledUnsafeDirectByteBuf.java:48)
	at io.netty.buffer.UnsafeByteBufUtil.newUnsafeDirectByteBuf(UnsafeByteBufUtil.java:698)
	at io.netty.buffer.AdaptiveByteBufAllocator$DirectChunkAllocator.allocate(AdaptiveByteBufAllocator.java:115)
	at io.netty.buffer.AdaptivePoolingAllocator.allocateFallback(AdaptivePoolingAllocator.java:309)
	at io.netty.buffer.AdaptivePoolingAllocator.allocate(AdaptivePoolingAllocator.java:273)
	at io.netty.buffer.AdaptivePoolingAllocator.reallocate(AdaptivePoolingAllocator.java:332)
	at io.netty.buffer.AdaptivePoolingAllocator$AdaptiveByteBuf.capacity(AdaptivePoolingAllocator.java:1471)
	at io.netty.buffer.AbstractByteBuf.ensureWritable0(AbstractByteBuf.java:305)
	at io.netty.buffer.AbstractByteBuf.ensureWritable(AbstractByteBuf.java:280)
	at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1080)
	at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1088)
	at io.netty.buffer.WrappedByteBuf.writeBytes(WrappedByteBuf.java:803)
	at io.netty.buffer.AdvancedLeakAwareByteBuf.writeBytes(AdvancedLeakAwareByteBuf.java:618)

Logs - same picture, 4MB increment, heap is also 0 (expected, just logged it for completeness), last reported before OOM is
37617664 which is also around 12MB lower then allocated: 50200577 from the error message

BaurzhanSakhariev · 2025-12-18T13:07:25Z

BaurzhanSakhariev
Dec 18, 2025
Author

on heap OOM part is resolved in comments of #16058

Created #16068 for reproducible part

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ByteBufAllocatorMetric.usedDirectMemory reports low values #16059

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Uh oh!

ByteBufAllocatorMetric.usedDirectMemory reports low values #16059

Uh oh!

Uh oh!

BaurzhanSakhariev Dec 17, 2025

Replies: 1 comment

Uh oh!

Uh oh!

BaurzhanSakhariev Dec 18, 2025 Author

BaurzhanSakhariev
Dec 17, 2025

BaurzhanSakhariev
Dec 18, 2025
Author