Flaky test_slow_terminate #6373
Description
test_slow_terminate failed unexpectedly in #6372 with a standard 30s test timeout
) from None
E asyncio.exceptions.TimeoutError: Test timeout after 30s.
E ========== Test stack trace starts here ==========
E Stack for <Task pending name='Task-172714' coro=<test_slow_terminate() running at /home/runner/work/distributed/distributed/distributed/tests/test_worker_memory.py:788> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f802626fd90>()]>> (most recent call last):
E File "/home/runner/work/distributed/distributed/distributed/tests/test_worker_memory.py", line 788, in test_slow_terminate
E await leak_until_restart(c, s, a)
https://github.com/dask/distributed/runs/6503645519?check_suite_focus=true
There are tons of warnings like
2022-05-19 09:01:54,693 - distributed.worker_memory - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker-memory.html#memory-not-released-back-to-the-os for more information. -- Unmanaged memory: 6.14 GiB -- Worker memory limit: 400.00 MiB
cc @crusaderky