Disable HTTP API by default (dask#6420)

Not entirely sure why, but `@functools.wraps` on the test function makes them show up. This is convenient when VSCode users want to debug a `@gen_test` test.

Co-authored-by: Matthew Rocklin <mrocklin@gmail.com>

Co-authored-by: Florian Jetter <fjetter@users.noreply.github.com>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…ask#6421)

Adds a Lock to distributed.profile to enable better concurrency control. In particular, it allows running garbage collection without a profiling thread holding references to objects, which is necessary for dask#6250.

…#6444)

the PeriodicCallbacks in self.periodic_callbacks need to be started and stopped on the same loop as self.loop because they bind to IOLoop.current() when PeriodicCallback.start is called

When running a synchronous Cluster the loop will be running in a different thread to where cluster._ipython_widget_ is called from

…k#6434)

This causes problems with certain kinds of views. So just pass the
`memoryview` as-is even though `PickleBuffer` will chain the underlying
`memoryview`s instead of referencing the original `obj`. Maybe this can
be fixed upstream.

…#6450)

Tests like dask#6395 will fail (timeout) because a log statement doesn't get printed, but since you never get to see what _was_ printed, CI failuers are hard to debug.

Adds a `wait_for_log_line` helper that tees the output to stdout, so you can at least see what happened.

Closes dask#6164

Fixes dask#6454

refs dask#6163 

Passing anything other than `IOLoop.current()` will fail when you try to use the Scheduler anyway:
self._lock will be bound to `IOLoop.current().asyncio_loop` aka `asyncio.get_running_loop()`:
https://github.com/dask/distributed/blob/fc42dbd89afa4ab6709b197db5c2a14c284e99bf/distributed/scheduler.py#L2923
and start_http_server runs on the IOLoop.current() also:
https://github.com/dask/distributed/blob/fc42dbd89afa4ab6709b197db5c2a14c284e99bf/distributed/scheduler.py#L2963

)

distributed/tests/test_profile.py fails occasionally (e.g., see dask#6444 (comment)). This test restructures the test to avoid timing-based flakes.

…sk#6923)

)

So that the progress bar diagnostic doesn't interfere with other print
statements or logging by the user.

dask#6898)