Skip to content

Report close connection exceptions at INFO#81768

Merged
DaveCTurner merged 1 commit intoelastic:masterfrom
DaveCTurner:2021-12-15-tcp-transport-info-logs
Dec 16, 2021
Merged

Report close connection exceptions at INFO#81768
DaveCTurner merged 1 commit intoelastic:masterfrom
DaveCTurner:2021-12-15-tcp-transport-info-logs

Conversation

@DaveCTurner
Copy link
Copy Markdown
Member

In #51612 we promoted certain log messages regarding unexpected network
exceptions from TRACE to DEBUG. In fact it's often useful to see
these exceptions by default, so with this commit we show the message
(but not the stack trace) at INFO level. This commit also adds some
commentary about what each of the exceptions means.

Closes #66473

In elastic#51612 we promoted certain log messages regarding unexpected network
exceptions from `TRACE` to `DEBUG`. In fact it's often useful to see
these exceptions by default, so with this commit we show the message
(but not the stack trace) at `INFO` level. This commit also adds some
commentary about what each of the exceptions means.

Closes elastic#66473
@DaveCTurner DaveCTurner added >enhancement :Distributed/Network Http and internode communication implementations v8.1.0 labels Dec 15, 2021
@elasticmachine elasticmachine added the Team:Distributed Meta label for distributed team. label Dec 15, 2021
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

logger.debug(() -> new ParameterizedMessage("send message failed [channel: {}]", channel), e);
final Level closeConnectionExceptionLevel = NetworkExceptionHelper.getCloseConnectionExceptionLevel(e);
if (closeConnectionExceptionLevel == Level.OFF) {
logger.warn(new ParameterizedMessage("send message failed [channel: {}]", channel), e);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the condition above be closeConnectionExceptionLevel == Level.WARN?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the WARN case is handled in the final else branch. This branch is handling the case that the exception doesn't appear to be network-related, and preserves the earlier behaviour that such exceptions are logged as warnings.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I was confused with Level.OFF actually representing a non network related.
I wonder if that could be changed to Level.WARN then, the first branch could be merged with the last one, unless I miss something.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind of, but I think that risks future bugs. The intention is to distinguish "this exception is network-related and deserves a warning" from "this exception is not network-related". An earlier iteration actually returned WARN sometimes but then I moved them all to INFO because I found that all the Linux ones might be benign and I didn't want to emit warnings for benign problems. I didn't investigate the Windows behaviour and we might decide to move them back up to WARN in future.

I could have used an Optional<Level> or a null or similar but Level#OFF seemed like a simpler way to say the same thing.

@DaveCTurner DaveCTurner merged commit a9d68bc into elastic:master Dec 16, 2021
@DaveCTurner
Copy link
Copy Markdown
Member Author

Thanks Ievgen!

@DaveCTurner DaveCTurner deleted the 2021-12-15-tcp-transport-info-logs branch December 16, 2021 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Distributed/Network Http and internode communication implementations >enhancement Team:Distributed Meta label for distributed team. v8.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Increase log levels of definitely-broken network disconnects

3 participants