Skip to content

IllegalReferenceCountException: refCnt: 0, decrement: 1 on large payloads #1374

Closed
Closed
IllegalReferenceCountException: refCnt: 0, decrement: 1 on large payloads#1374
Labels
for/springframeworkThis belongs to the Spring Framework projecttype/bugA general bug
@dazito

Description

@dazito
dazito
opened on Nov 9, 2020

I am using a Socks5 proxy but this issue also occurs with a direct connection (no proxy).
This issue seems to be related with netty/netty#9882 for in this case it happens all the time for the same URL.
I upgraded spring boot webflux version from 2.3.3.RELEASE to 2.3.5.RELEASE but I am still getting this issue.

Expected Behavior

Every time I make a GET request to a specific URL that generates a big text file (around 130 000 lines) I expect to get the text file parsed into a string.

Actual Behavior

Netty for some reason is throwing io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1

Steps to Reproduce

I am afraid I cannot share the url from where to get this text file as it is a private URL and I have no control over it.

This is the code to send the GET request:

httpReactiveClient.get()
                .uri(URI.create(BASE_URL + url)) 
                .retrieve()
                .bodyToMono(String.class)
                .map(s -> cleanResponseBody(url, s));

// Later call subscribe

Using exchange().flatMap() insteand of bodyToMono() and with debug mode on I can see that the server returns 200 OK response code so I believe the issue occurs while dealing the big response body (as also hinted here netty/netty#9882)

Your Environment

  • Reactor version(s) used: Using spring-boot-starter-webflux 2.3.5.RELEASE which is using reactor-netty 0.9.13.RELEASE
  • Other relevant libraries versions (eg. netty, ...):
  • JVM version (java -version): OpenJDK 11
  • OS and version (eg. uname -a): Ubuntu 20.04.1 LTS

Here is the debug log ouput from my request:

2020-11-10 00:20:24.834 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba] Created a new pooled channel, now 1 active connections and 0 inactive connections
2020-11-10 00:20:24.834 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.tcp.SslProvider            : [id: 0x22cf32ba] SSL enabled using engine SSLEngineImpl and SNI subdomain.upstream-server.com/<unresolved>:443
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.channel.BootstrapHandlers  : [id: 0x22cf32ba] Initialized pipeline DefaultChannelPipeline{(Socks5InitialResponseDecoder#0 = io.netty.handler.codec.socksx.v5.Socks5InitialResponseDecoder), (Socks5InitialResponseDecoder#0.encoder = io.netty.handler.codec.socksx.v5.Socks5ClientEncoder), (reactor.left.proxyHandler = io.netty.handler.proxy.Socks5ProxyHandler), (reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.sslReader = reactor.netty.tcp.SslProvider$SslReadHandler), (BootstrapHandlers$BootstrapInitializerHandler#0 = reactor.netty.channel.BootstrapHandlers$BootstrapInitializerHandler), (PooledConnectionProvider$PooledConnectionAllocator$PooledConnectionInitializer#0 = reactor.netty.resources.PooledConnectionProvider$PooledConnectionAllocator$PooledConnectionInitializer), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/files/6FfTNtIl/, connection=PooledConnection{channel=[id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050]}}, [response_completed])
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] Removed handler: ReadTimeoutHandler, pipeline: DefaultChannelPipeline{(reactor.left.proxyHandler = io.netty.handler.proxy.Socks5ProxyHandler), (reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (WriteTimeoutHandler = io.netty.handler.timeout.WriteTimeoutHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] Removed handler: WriteTimeoutHandler, pipeline: DefaultChannelPipeline{(reactor.left.proxyHandler = io.netty.handler.proxy.Socks5ProxyHandler), (reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/files/6FfTNtIl/, connection=PooledConnection{channel=[id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050]}}, [disconnecting])
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] Releasing channel
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] Channel cleaned, now 0 active connections and 1 inactive connections
2020-11-10 00:20:24.836 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Registering pool release on close event for channel
2020-11-10 00:20:24.836 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Channel connected, now 1 active connections and 0 inactive connections
2020-11-10 00:20:24.844 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x84c6a868, L:/127.0.0.1:47946 ! R:localhost/127.0.0.1:9050] onStateChange(PooledConnection{channel=[id: 0x84c6a868, L:/127.0.0.1:47946 ! R:localhost/127.0.0.1:9050]}, [disconnecting])
2020-11-10 00:20:25.196 DEBUG 19156 --- [or-http-epoll-4] io.netty.handler.ssl.SslHandler          : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] HANDSHAKEN: protocol:TLSv1.3 cipher suite:TLS_AES_256_GCM_SHA384
2020-11-10 00:20:25.196 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] onStateChange(PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050]}, [connected])
2020-11-10 00:20:25.196 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/, connection=PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050]}}, [configured])
2020-11-10 00:20:25.196 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Added decoder [ReadTimeoutHandler] at the end of the user pipeline, full pipeline: [reactor.left.proxyHandler, reactor.left.sslHandler, reactor.left.httpCodec, reactor.left.decompressor, ReadTimeoutHandler, reactor.right.reactiveBridge, DefaultChannelPipeline$TailContext#0]
2020-11-10 00:20:25.197 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Added decoder [WriteTimeoutHandler] at the end of the user pipeline, full pipeline: [reactor.left.proxyHandler, reactor.left.sslHandler, reactor.left.httpCodec, reactor.left.decompressor, ReadTimeoutHandler, WriteTimeoutHandler, reactor.right.reactiveBridge, DefaultChannelPipeline$TailContext#0]
2020-11-10 00:20:25.197 DEBUG 19156 --- [or-http-epoll-4] r.netty.http.client.HttpClientConnect    : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Handler is being applied: {uri=https://subdomain.upstream-server.com/feed/yjb05.dat, method=GET}
2020-11-10 00:20:25.197 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/feed/yjb05.dat, connection=PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050]}}, [request_prepared])
2020-11-10 00:20:25.197 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/feed/yjb05.dat, connection=PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050]}}, [request_sent])
2020-11-10 00:20:25.334 DEBUG 19156 --- [or-http-epoll-4] io.netty.handler.ssl.SslHandler          : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] HANDSHAKEN: protocol:TLSv1.3 cipher suite:TLS_AES_256_GCM_SHA384
2020-11-10 00:20:25.334 DEBUG 19156 --- [or-http-epoll-4] io.netty.handler.ssl.SslHandler          : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] HANDSHAKEN: protocol:TLSv1.3 cipher suite:TLS_AES_256_GCM_SHA384
2020-11-10 00:20:25.382 DEBUG 19156 --- [or-http-epoll-4] r.n.http.client.HttpClientOperations     : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Received response (auto-read:false) : [Server=nginx, Date=Mon, 09 Nov 2020 23:20:25 GMT, Content-Type=text/javascript;charset=UTF-8, Connection=keep-alive, Expires=Mon, 09 Nov 2020 23:20:05 GMT, Pragma=no-cache, Vary=Accept-Encoding, Cache-Control=public, Last-Modified=Mon, 09 Nov 2020 23:19:55 GMT, Accept-Ranges=bytes, Age=29, X-Deliver=Mon, 09 Nov 2020 23:20:25 GMT, transfer-encoding=chunked]
2020-11-10 00:20:25.382 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/feed/yjb05.dat, connection=PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050]}}, [response_received])
2020-11-10 00:20:25.383 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.channel.FluxReceive        : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] FluxReceive{pending=0, cancelled=false, inboundDone=false, inboundError=null}: subscribing inbound receiver
2020-11-10 00:20:25.487 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] Channel closed, now 0 active connections and 0 inactive connections
2020-11-10 00:20:25.487 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] Non Removed handler: ReadTimeoutHandler, context: ChannelHandlerContext(ReadTimeoutHandler, [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050]), pipeline: DefaultChannelPipeline{(reactor.left.proxyHandler = io.netty.handler.proxy.Socks5ProxyHandler), (reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (ReadTimeoutHandler = io.netty.handler.timeout.ReadTimeoutHandler), (WriteTimeoutHandler = io.netty.handler.timeout.WriteTimeoutHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-11-10 00:20:25.487 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] Non Removed handler: WriteTimeoutHandler, context: ChannelHandlerContext(WriteTimeoutHandler, [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050]), pipeline: DefaultChannelPipeline{(reactor.left.proxyHandler = io.netty.handler.proxy.Socks5ProxyHandler), (reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (ReadTimeoutHandler = io.netty.handler.timeout.ReadTimeoutHandler), (WriteTimeoutHandler = io.netty.handler.timeout.WriteTimeoutHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-11-10 00:20:25.488 ERROR 19156 --- [or-http-epoll-4] r.n.channel.ChannelOperationsHandler     : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] Error was received while reading the incoming data. The connection will be closed.

io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
	at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100) ~[netty-buffer-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.http.DefaultHttpContent.release(DefaultHttpContent.java:92) ~[netty-codec-http-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:340) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
	at reactor.netty.channel.ChannelOperations.onInboundNext(ChannelOperations.java:358) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
	at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:677) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
	at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:96) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1282) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1329) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.proxy.ProxyHandler.channelRead(ProxyHandler.java:253) ~[netty-handler-proxy-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at java.base/java.lang.Thread.run(Thread.java:832) ~[na:na]

2020-11-10 00:20:25.489  WARN 19156 --- [or-http-epoll-4] reactor.netty.channel.FluxReceive        : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] An exception has been observed post termination

io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
	at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100) ~[netty-buffer-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.http.DefaultHttpContent.release(DefaultHttpContent.java:92) ~[netty-codec-http-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:340) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
	at reactor.netty.channel.ChannelOperations.onInboundNext(ChannelOperations.java:358) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
	at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:677) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
	at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:96) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1282) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1329) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.handler.proxy.ProxyHandler.channelRead(ProxyHandler.java:253) ~[netty-handler-proxy-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
	at java.base/java.lang.Thread.run(Thread.java:832) ~[na:na]

2020-11-10 00:20:25.489 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/feed/yjb05.dat, connection=PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050]}}, [disconnecting])

Metadata

Metadata

Assignees

No one assigned

    Labels

    for/springframeworkThis belongs to the Spring Framework projecttype/bugA general bug

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions