-
Notifications
You must be signed in to change notification settings - Fork 38.9k
Multipart upload leak on client abort (ByteBuf.release() not called) #36262
Description
Description
I see Netty leak reports during multipart uploads when the client aborts the request mid‑upload. Leak detector logs point to the inbound HTTP pipeline / multipart parsing path. This reproduces quickly when the client timeout is small and the
uploaded files are large.
Environment
- OS: Linux
- JDK: 21
- Spring Boot: 3.5.10
- Spring Framework: 6.x (from Boot 3.5.10 BOM)
- Reactor Netty: 1.2.14
- Netty: 4.1.130.Final
- Leak detection:
-Dio.netty.leakDetection.level=advanced (also tried paranoid)
-Dio.netty.leakDetection.targetRecords=32
-Dio.netty.leakDetection.samplingInterval=64
Dependencies (relevant)
org.springframework.boot:spring-boot-starter-reactor-netty:3.5.10
└─ io.projectreactor.netty:reactor-netty-http:1.2.14
├─ io.netty:netty-codec-http:4.1.130.Final
├─ io.netty:netty-buffer:4.1.130.Final
├─ io.netty:netty-transport:4.1.130.Final
├─ io.netty:netty-handler:4.1.130.Final
└─ io.projectreactor.netty:reactor-netty-core:1.2.14
Reproduction
Minimal repro project:
https://github.com/dmittriy13/webflux-multipart-leak-repro
Summary of steps:
- Start the app.
- Run k6 load test with a short client timeout so requests are aborted mid‑upload.
- Observe leak logs.
Expected behavior
No ByteBuf leak reports on client abort. Inbound buffers should be released even if the request is cancelled.
Actual behavior
Leak logs appear immediately after client abort. Example:
Full leak stack trace
2026-02-05T09:56:34.932Z ERROR 1 --- [r-http-epoll-13] io.netty.util.ResourceLeakDetector : LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more
information.
app-1 | Recent access records:
app-1 | #1:
app-1 | io.netty.handler.codec.http.DefaultHttpContent.release(DefaultHttpContent.java:92)
app-1 | io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:90)
app-1 | reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:296)
app-1 | reactor.netty.channel.FluxReceive.lambda$request$1(FluxReceive.java:136)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | #2:
app-1 | io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:677)
app-1 | org.springframework.core.io.buffer.NettyDataBuffer.forEachByte(NettyDataBuffer.java:134)
app-1 | org.springframework.core.io.buffer.DataBufferUtils$AbstractNestedMatcher.match(DataBufferUtils.java:882)
app-1 | org.springframework.http.codec.multipart.MultipartParser$BodyState.onNext(MultipartParser.java:522)
app-1 | org.springframework.http.codec.multipart.MultipartParser.hookOnNext(MultipartParser.java:123)
app-1 | org.springframework.http.codec.multipart.MultipartParser.hookOnNext(MultipartParser.java:52)
app-1 | reactor.core.publisher.BaseSubscriber.onNext(BaseSubscriber.java:160)
app-1 | reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
app-1 | reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200)
app-1 | reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
app-1 | reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:292)
app-1 | reactor.netty.channel.FluxReceive.lambda$request$1(FluxReceive.java:136)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | #3:
app-1 | org.springframework.core.io.buffer.NettyDataBufferFactory.wrap(NettyDataBufferFactory.java:94)
app-1 | reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106)
app-1 | reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200)
app-1 | reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
app-1 | reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:292)
app-1 | reactor.netty.channel.FluxReceive.lambda$request$1(FluxReceive.java:136)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | #4:
app-1 | reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:185)
app-1 | reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
app-1 | reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:292)
app-1 | reactor.netty.channel.FluxReceive.lambda$request$1(FluxReceive.java:136)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | #5:
app-1 | Hint: 'reactor.right.reactiveBridge' will handle the message from this point.
app-1 | io.netty.handler.codec.http.DefaultHttpContent.touch(DefaultHttpContent.java:86)
app-1 | io.netty.handler.codec.http.DefaultHttpContent.touch(DefaultHttpContent.java:25)
app-1 | io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:115)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:417)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | reactor.netty.http.server.HttpTrafficHandler.channelRead(HttpTrafficHandler.java:326)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:361)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:325)
app-1 | io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
app-1 | io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:805)
app-1 | io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | #6:
app-1 | Hint: 'reactor.left.httpTrafficHandler' will handle the message from this point.
app-1 | io.netty.handler.codec.http.DefaultHttpContent.touch(DefaultHttpContent.java:86)
app-1 | io.netty.handler.codec.http.DefaultHttpContent.touch(DefaultHttpContent.java:25)
app-1 | io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:115)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:417)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:361)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:325)
app-1 | io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
app-1 | io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:805)
app-1 | io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | Created at:
app-1 | io.netty.buffer.SimpleLeakAwareByteBuf.unwrappedDerived(SimpleLeakAwareByteBuf.java:144)
app-1 | io.netty.buffer.SimpleLeakAwareByteBuf.readRetainedSlice(SimpleLeakAwareByteBuf.java:67)
app-1 | io.netty.buffer.AdvancedLeakAwareByteBuf.readRetainedSlice(AdvancedLeakAwareByteBuf.java:108)
app-1 | io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:459)
app-1 | io.netty.handler.codec.http.HttpServerCodec$HttpServerRequestDecoder.decode(HttpServerCodec.java:167)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:545)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:484)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
app-1 | io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
app-1 | io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:805)
app-1 | io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
Additional notes
- The issue reproduces only for multipart. A raw (non‑multipart) upload endpoint does not leak under the same abort conditions.
- The leak appears even if application logic is effectively a no‑op (no downstream calls, no further processing).
Questions
- Is this a known issue in multipart parsing under client abort?
- Is there a recommended way to ensure inbound buffers are released in this scenario?