-
-
Notifications
You must be signed in to change notification settings - Fork 4.9k
Fix pending_result memory leak #9806
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
This was the smallest change I could come up with, but I believe the result backends could be updated to take care of this on the call to forget as well. In that case, I can create a new PR for that. Though I'm fine with this approach. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9806 +/- ##
==========================================
- Coverage 78.62% 78.61% -0.01%
==========================================
Files 153 153
Lines 19199 19200 +1
Branches 2547 2547
==========================================
Hits 15095 15095
+ Misses 3811 3809 -2
- Partials 293 296 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
auvipy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we also add unit and integration tests? or they are already covered?
I think so. A sanity check seems appropriate for this issue. |
|
lets do it |
|
I added this unit test, but it feels a bit redundant. |
|
self = <pytest_celery.api.worker.CeleryTestWorker object at 0x7f42339a6b10>
E AssertionError: Worker container 'peaceful_kowalevski' did not log -> Starting long running task within 60 seconds |
|
t/integration/test_canvas.py::test_stamping_mechanism::test_all_tasks_of_canvas_are_stamped [Assert all tasks are stamped] SUBPASS::debug::Code: null |
|
test failures are seems to be unrelated |
|
so how should we move forward with this? #9797 (comment) |
I don't see how it could be related at the moment., and I had the same issues with multiple previous releases. I'm hoping it's a an issue with my local setup. We can go ahead. |
9ddb04c to
306dad6
Compare
306dad6 to
a761b28
Compare
Description
Async results create subscriptions and they get stored in the backend's _pending_results store. The docs clearly warns about the mandatory use of either get or forget to clean up resources. Still these subscriptions aren't cleaned up when only forget is called for example.
See #9797 for more details.
Changes(Fixes #9797)
Added a call to backend.remove_pending_result when forget is called to release any subscriptions and references stored in the backend.