I can't reproduce this issue using any tests, yet. maybe #5910 was already a fix.

However, our old friend test_dashboard_port_zero is failing. It picks proper ports, the scheduler comes up but judging from the timestamps, the scheduler closes pretty soon again

2022-06-23 10:48:33,378 - distributed.scheduler - INFO - -----------------------------------------------
2022-06-23 10:48:35,201 - distributed.http.proxy - INFO - To route to workers diagnostics web server please install jupyter-server-proxy: python -m pip install jupyter-server-proxy
2022-06-23 10:48:35,324 - distributed.scheduler - INFO - State start
2022-06-23 10:48:35,329 - distributed.scheduler - INFO - -----------------------------------------------
2022-06-23 10:48:35,329 - distributed.scheduler - INFO - Clear task state
2022-06-23 10:48:35,330 - distributed.scheduler - INFO -   Scheduler at:     tcp://127.0.0.1:49760
2022-06-23 10:48:35,330 - distributed.scheduler - INFO -   dashboard at:           127.0.0.1:49773
2022-06-23 10:48:35,825 - distributed.scheduler - INFO - Scheduler closing...
2022-06-23 10:48:35,825 - distributed.scheduler - INFO - Scheduler closing all comms
2022-06-23 10:48:35,826 - distributed.scheduler - INFO - Stopped scheduler at 'tcp://127.0.0.1:49760'
2022-06-23 10:48:35,827 - distributed.scheduler - INFO - End scheduler

We can see that almost exactly 500ms from the start 35,324, the scheduler closes at 35,825.

Is anybody aware of an interval we set to 500ms that could cause a scheduler to close?

Another confusing part is that the CI run https://github.com/dask/distributed/runs/7021261881?check_suite_focus=true

shows the following code for the test

deftest_dashboard_port_zero(loop):
        pytest.importorskip("bokeh")
        port = open_port()
with popen(
            [
"dask-scheduler",
"--host",
f"127.0.0.1:{port}",
"--dashboard-address",
":0",
            ],
        ):
>           with Client(f"tcp://127.0.0.1:{port}", loop=loop) as c:

which is the new version of this test after #6591

but my branch should still include the old version of the test, see https://github.com/fjetter/distributed/blob/eb679cc3912fa33cd6f24575973367ce99d71fcd/distributed/cli/tests/test_dask_scheduler.py#L217-L226

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       15 files  ±0         15 suites  ±0   10h 21m 30s ⏱️ - 37m 50s
  2 890 tests ±0    2 805 ✔️ +2    84 💤 +1  1 ❌  - 2 
21 408 runs  ±0  20 437 ✔️  - 1  970 💤 +6  1 ❌  - 4 

For more details on these failures, see this check.

Results for commit d22cbbb. ± Comparison against base commit 4779f84.

♻️ This comment has been updated with latest results.

well, I rebased and cherry-picked 9f9e1c2 onto it
Let's see what CI thinks

Another confusing part is that the CI run [...] shows [...] the new version of this test after #6591 but my branch should still include the old version of the test

When a github action that contains

      - name: Checkout source
        uses: actions/checkout@v2

is triggered by a pull_request event, what's checked out is a copy of main with your branch merged in.
The tests in https://github.com/fjetter/distributed/actions, which are instead triggerd by the push event, will contain the old code.