[refactor][broker] Suppress error logging when message expiration fails#19778
Merged
Conversation
|
@massakam Please add the following content to your PR description and select a checkbox: |
efbec10 to
7b37028
Compare
7b37028 to
22b4b8d
Compare
22b4b8d to
9a1c906
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #19778 +/- ##
============================================
- Coverage 72.89% 72.85% -0.05%
+ Complexity 31619 31600 -19
============================================
Files 1861 1861
Lines 137356 137376 +20
Branches 15117 15124 +7
============================================
- Hits 100131 100080 -51
- Misses 29271 29349 +78
+ Partials 7954 7947 -7
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
There are some cases where an error is returned when trying to expire messages using the pulsar-admin command. For example, when there are no messages in the backlog.
$ ./bin/pulsar-admin topics expire-messages -s sub1 -t 1m persistent://public/default/massakam 2023-03-09T16:07:01,485+0900 [AsyncHttpClient-7-1] WARN org.apache.pulsar.client.admin.internal.BaseResource - [http://localhost:8080/admin/v2/persistent/public/default/massakam/subscription/sub1/expireMessages/60] Failed to perform http post request: javax.ws.rs.ClientErrorException: HTTP 409 {"reason":"Expire message by timestamp not issued on topic persistent://public/default/massakam for subscription sub1 due to ongoing message expiration not finished or subscription almost catch up. If it's performed on a partitioned topic operation might succeeded on other partitions, please check stats of individual partition."} Expire message by timestamp not issued on topic persistent://public/default/massakam for subscription sub1 due to ongoing message expiration not finished or subscription almost catch up. If it's performed on a partitioned topic operation might succeeded on other partitions, please check stats of individual partition. Reason: Expire message by timestamp not issued on topic persistent://public/default/massakam for subscription sub1 due to ongoing message expiration not finished or subscription almost catch up. If it's performed on a partitioned topic operation might succeeded on other partitions, please check stats of individual partition. 2023-03-09T16:07:01,529+0900 [main] WARN org.apache.pulsar.common.util.ShutdownUtil - Triggering immediate shutdown of current process with status 1 java.lang.Exception: Stacktrace for immediate shutdown at org.apache.pulsar.common.util.ShutdownUtil.triggerImmediateForcefulShutdown(ShutdownUtil.java:52) ~[pulsar-common.jar:3.0.0-SNAPSHOT] at org.apache.pulsar.admin.cli.PulsarAdminTool.exit(PulsarAdminTool.java:315) ~[pulsar-client-tools.jar:3.0.0-SNAPSHOT] at org.apache.pulsar.admin.cli.PulsarAdminTool.execute(PulsarAdminTool.java:305) ~[pulsar-client-tools.jar:3.0.0-SNAPSHOT] at org.apache.pulsar.admin.cli.PulsarAdminTool.main(PulsarAdminTool.java:280) ~[pulsar-client-tools.jar:3.0.0-SNAPSHOT]At this time, the broker also prints an error log with a very long stacktrace of the exception that caused it. If we are monitoring error logs, this is very noisy and unnecessarily increases the size of the log files.
click to expand
In many cases, it is caused by incorrect requests from users, so I think the log level should be warn instead of error.
Modifications
In the admin APIs to expire messages, if the exception thrown is
RestException, log a warn, otherwise log an error with the stacktrace of the exception.Verifying this change
Documentation
docdoc-requireddoc-not-neededdoc-complete