Fix broker connection retry attempt counter in the error log #9911
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #9911 +/- ##
==========================================
- Coverage 78.67% 78.67% -0.01%
==========================================
Files 153 153
Lines 19291 19295 +4
Branches 2568 2569 +1
==========================================
+ Hits 15177 15180 +3
Misses 3816 3816
- Partials 298 299 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
9897610 to
36e41f6
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR fixes an issue where the broker connection retry attempt counter in error logs becomes stalled after some retry attempts. The problem was that the retry counter was calculated from the connection interval, which plateaus at a maximum value, making the displayed retry count inaccurate.
- Adds a dedicated counter to track actual retry attempts instead of deriving from interval
- Updates error logging to use the new counter for accurate retry count display
- Resets the counter when connection is successfully established
Reviewed Changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| celery/worker/consumer/consumer.py | Adds broker_connection_retry_attempt counter and updates retry logic to use it instead of interval-based calculation |
| t/unit/worker/test_consumer.py | Updates test to reflect corrected retry interval behavior in error messages |
50a0785 to
48a2897
Compare
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
e143136 to
0a197b7
Compare
Note: Before submitting this pull request, please review our contributing
guidelines.
Description
When the consumer fails to connect to broker, the error logs retry counter stalls after some time. More can be found in this issue : #9340
Design
Maintains a counter in the consumer to track retry attempts, this is because the interval returned from kombu is not sufficient indicator for the retry count and will be interval_max after a few retry attempts.
Changes