-
-
Notifications
You must be signed in to change notification settings - Fork 757
Description
In tests that raise an exception, the DequeHandler prevents garbage collection by maintaining a reference to the raised exception and its traceback:

(This reference chain was created with objgraph.find_backref_chain https://mg.pov.lt/objgraph/#memory-leak-example)
Keeping those references alive prevents garbage collection of otherwise released objects and therefore testing for proper garbage collection of Scheduler, TaskState and Worker instances as requested by #6250. For example, in #6344, the failures of distributed/tests/test_worker.py::test_heartbeat_comm_closed and several other tests are caused by this.
One possible solution to enable garbage collection in tests would be to call DequeHandler.clear_all_instances() in check_instances() before checking for proper garbage collection, though there might be superior options that generally improve our logging.