Mock process memory readings in test_worker.py (v2)#5878
Mock process memory readings in test_worker.py (v2)#5878crusaderky merged 20 commits intodask:mainfrom
Conversation
This reverts commit ef0e44b.
fjetter
left a comment
There was a problem hiding this comment.
I think these tests are much more readable and easier to maintain, thank you!
| while not a.data.disk or len(a.data.disk) > prev_n: | ||
| prev_n = len(a.data.disk) | ||
| await asyncio.sleep(0) |
There was a problem hiding this comment.
This is a bit fragile. As soon as the while loop in the memory monitor changes anything about how it awaits, e.g. another async call to evict data, this condition will break since we rely on skipping exactly one loop iteration with sleep(0).
I can't come up with a nicer way other than sleeping for a longer time so this is fine for me. I just wanted to raise this in case somebody else thinks of something
There was a problem hiding this comment.
Another ugly thing we can do but I think I dislike this even more is to wait for _memory_monitoring to toggle since then we know for sure the memory_monitor is done. I think I prefer sticking with sleep(0) and deal with a test failure/test rewrite if that ever happens
In scope
test_spill_hysteresisflaky on ubuntu #5848Out of scope
test_scheduler.py::test_memorytest_active_memory_manager.py, which should be carried out by adding a switch to use managed memoryrebalance(): postponed indefinitely due to the whole function to be reimplemented on top of the Active Memory Manager