-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[improve][broker] Optimize message expiration rate repeated update issues #24073
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[improve][broker] Optimize message expiration rate repeated update issues #24073
Conversation
|
@zjxxzjwang Please add the following content to your PR description and select a checkbox: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the time period for calculating the rate is the interval between two tasks. If the time interval between two tasks is very small, the calculation speed will be very large.
Good observations, @zjxxzjwang. It seems that this PR is not only an optimization, but targets fixing invalid rate stats.
However, the current change would make "topic stats" inaccurate since the rate wouldn't get updated in that case. For consistency reasons, updating the rate should happen in the same way for all code paths.
One possible solution would be to make updateRates method private in PersistentMessageExpiryMonitor and call updateRates in the implementation of getMessageExpiryRate method, before the updated rate is returned.
Hello, the rate will continue to be updated ,because the updateRates method of the PersistentTopic class will also be executed at regular intervals, and the time period used to calculate the rate will be fixed (the timing period of the updateRates method). The calculated rate can be explained more strongly. @lhotari |
@zjxxzjwang Please address this review comment. The solution would be somewhat similar as there is in #24067. |
Good suggestion, I have made the modifications according to your suggestion. Please help me review it。 @lhotari |
|
@lhotari Hello, please help me review it |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #24073 +/- ##
============================================
+ Coverage 73.57% 74.17% +0.59%
+ Complexity 32624 32426 -198
============================================
Files 1877 1863 -14
Lines 139502 144311 +4809
Branches 15299 16457 +1158
============================================
+ Hits 102638 107041 +4403
+ Misses 28908 28812 -96
- Partials 7956 8458 +502
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…sues (apache#24073) Co-authored-by: zjxxzjwang <zjxxzjwang@tencent.com> (cherry picked from commit b936f46) (cherry picked from commit c51b9ca)
…sues (apache#24073) Co-authored-by: zjxxzjwang <zjxxzjwang@tencent.com> (cherry picked from commit b936f46)
…sues (apache#24073) Co-authored-by: zjxxzjwang <zjxxzjwang@tencent.com> (cherry picked from commit b936f46) (cherry picked from commit c51b9ca)
…sues (apache#24073) Co-authored-by: zjxxzjwang <zjxxzjwang@tencent.com> (cherry picked from commit b936f46) (cherry picked from commit c51b9ca)
…sues (apache#24073) Co-authored-by: zjxxzjwang <zjxxzjwang@tencent.com> (cherry picked from commit b936f46) (cherry picked from commit f3368b4)
…sues (apache#24073) Co-authored-by: zjxxzjwang <zjxxzjwang@tencent.com> (cherry picked from commit b936f46) (cherry picked from commit f3368b4)
…sues (apache#24073) Co-authored-by: zjxxzjwang <zjxxzjwang@tencent.com> (cherry picked from commit b936f46) (cherry picked from commit f3368b4)
…sues (apache#24073) Co-authored-by: zjxxzjwang <zjxxzjwang@tencent.com>
Motivation
The message expiration rate is updated in both tasks, as follows:


the time period for calculating the rate is the interval between two tasks. If the time interval between two tasks is very small, the calculation speed will be very large.
Modifications
Delete the update method in the message expiration task, and retain the unified update method in the topic
Documentation
docdoc-requireddoc-not-neededdoc-complete