Subsecond cluster startup time #6413
Description
We used to be able to start up clusters in tens of milliseconds. Now it takes a second or two.
In [1]: from dask.distributed import LocalCluster
In [2]: %%time
...: with LocalCluster(processes=False, n_workers=0):
...: pass
...:
CPU times: user 751 ms, sys: 68.5 ms, total: 819 ms
Wall time: 2.79 s
In [3]: %%time
...: with LocalCluster(processes=False, n_workers=0):
...: pass
...:
CPU times: user 73.4 ms, sys: 28 µs, total: 73.4 ms
Wall time: 1.08 sThis would make testing nicer.