Skip to content

pytest'sLogCaptureHandler prevents garbage collection of Worker instances (and others) #6352

@hendrikmakait

Description

@hendrikmakait

In tests that raise an exception, the LogCaptureHandler from pytest prevents garbage collection by maintaining a reference to the raised exception and its traceback:
worker_chain

(This reference chain was created with objgraph.find_backref_chain https://mg.pov.lt/objgraph/#memory-leak-example)

Keeping those references alive prevents garbage collection of otherwise released objects and therefore testing for proper garbage collection of Scheduler, TaskState and Worker instances as requested by #6250. For example, in #6344, the failures of distributed/tests/test_scheduler.py::test_rebalance_workers_and_keys and several other tests are caused by this.

One possible for the failures in #6344 would be to manually clear the handlers during check_instances(), though they seem to be hard to access, which results in a rather brittle solution:

handlers = logging.getLogger().handlers
for handler in handlers:
    if hasattr(handler, "records"):
        handler.records.clear()

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions