Skip to content

Fix handler name on message not fully read#21478

Merged
jasontedor merged 1 commit intoelastic:masterfrom
jasontedor:message-not-fully-read-wrapper
Nov 10, 2016
Merged

Fix handler name on message not fully read#21478
jasontedor merged 1 commit intoelastic:masterfrom
jasontedor:message-not-fully-read-wrapper

Conversation

@jasontedor
Copy link
Copy Markdown
Member

@jasontedor jasontedor commented Nov 10, 2016 •
edited
Loading

Today when a message is not fully read on a response, we log (among
other details) the handler name. Unfortunately, if the handler is a
wrapper, all that we see is

org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler@7446ba18

completely losing the offending handler. This commit adds an override
for TransportService$ContextRestoreResponseHandler#toString so that the
underlying offender can be discovered.

@jasontedor jasontedor added :Distributed/Network Http and internode communication implementations review v5.0.1 v5.1.1 v6.0.0-alpha1 labels Nov 10, 2016
@jasontedor
Fix handler name on message not fully read
f04a404 
Today when a message is not fully read on a response, we log (among
other details) the handler name. Unfortunately, if the handler is a
wrapper, all that we see is

o.e.t.TransportService$ContextRestoreResponseHandler@7446ba18

completely losing the offending handler. This commit adds an override
for TransportService$ContextRestoreResponseHandler#toString so that the
underlying offender can be discovered.
@jasontedor jasontedor force-pushed the message-not-fully-read-wrapper branch from 1c7a12c to f04a404 Compare November 10, 2016 19:46
@rjernst
Copy link
Copy Markdown
Member

rjernst commented Nov 10, 2016

LGTM

@jasontedor jasontedor merged commit 0a06a0c into elastic:master Nov 10, 2016
jasontedor added a commit that referenced this pull request Nov 10, 2016
@jasontedor
Fix handler name on message not fully read
54275e1 
Today when a message is not fully read on a response, we log (among
other details) the handler name. Unfortunately, if the handler is a
wrapper, all that we see is

o.e.t.TransportService$ContextRestoreResponseHandler@7446ba18

completely losing the offending handler. This commit adds an override
for TransportService$ContextRestoreResponseHandler#toString so that the
underlying offender can be discovered.

Relates #21478
jasontedor added a commit that referenced this pull request Nov 10, 2016
@jasontedor
Fix handler name on message not fully read
66058f2 
Today when a message is not fully read on a response, we log (among
other details) the handler name. Unfortunately, if the handler is a
wrapper, all that we see is

o.e.t.TransportService$ContextRestoreResponseHandler@7446ba18

completely losing the offending handler. This commit adds an override
for TransportService$ContextRestoreResponseHandler#toString so that the
underlying offender can be discovered.

Relates #21478
@jasontedor
Copy link
Copy Markdown
Member Author

jasontedor commented Nov 10, 2016

Thanks @nik9000 and @rjernst.

@jasontedor jasontedor deleted the message-not-fully-read-wrapper branch November 10, 2016 19:57
@hujing101
Copy link
Copy Markdown

hujing101 commented Dec 3, 2016

Elasticsearch version: 5.0.2
Plugins installed: []
JVM version: 1.8.0_60
OS version: Red Hat Enterprise Linux release 6.3 (Santiago)
ES_Cluster_Nodes: 3
The datanode log file has an exception:
java.lang.IllegalStateException: Message not fully read (response) for requestId [42250], handler [org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction$1@1c53da10], error [false]; resetting at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1220) ~[elasticsearch-5.0.2.jar:5.0.2] at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[transport-netty4-5.0.2.jar:5.0.2] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293) [netty-codec-4.1.5.Final.jar:4.1.5.Final] at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:280) [netty-codec-4.1.5.Final.jar:4.1.5.Final] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:396) [netty-codec-4.1.5.Final.jar:4.1.5.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248) [netty-codec-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:610) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:513) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:467) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:437) [netty-transport-4.1.5.Final.jar:4.1.5.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-common-4.1.5.Final.jar:4.1.5.Final] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]

@jasontedor jasontedor mentioned this pull request Dec 20, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nik9000 nik9000 nik9000 approved these changes

Assignees

No one assigned

Labels

:Distributed/Network Http and internode communication implementations >enhancement v5.0.1 v5.1.1 v6.0.0-alpha1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@jasontedor @rjernst @hujing101 @nik9000 @clintongormley