-
-
Notifications
You must be signed in to change notification settings - Fork 757
Description
In tests that raise an exception, the LogCaptureHandler from pytest prevents garbage collection by maintaining a reference to the raised exception and its traceback:

(This reference chain was created with objgraph.find_backref_chain https://mg.pov.lt/objgraph/#memory-leak-example)
Keeping those references alive prevents garbage collection of otherwise released objects and therefore testing for proper garbage collection of Scheduler, TaskState and Worker instances as requested by #6250. For example, in #6344, the failures of distributed/tests/test_scheduler.py::test_rebalance_workers_and_keys and several other tests are caused by this.
One possible for the failures in #6344 would be to manually clear the handlers during check_instances(), though they seem to be hard to access, which results in a rather brittle solution:
handlers = logging.getLogger().handlers
for handler in handlers:
if hasattr(handler, "records"):
handler.records.clear()