Skip to content

WorkerState duration counters on prefix level#7288

Merged
fjetter merged 1 commit intodask:mainfrom
fjetter:duration_counter_prefix
Nov 10, 2022
Merged

WorkerState duration counters on prefix level#7288
fjetter merged 1 commit intodask:mainfrom
fjetter:duration_counter_prefix

Conversation

@fjetter
Copy link
Member

@fjetter fjetter commented Nov 10, 2022

Counting tasks on group level was always unnecessary. Group level counter may be a good tool to replace occupancy eventually in scheduling decisions but we're not there yet. For now, counters on prefix level provide the exact same result but it requires orders of magnitude fewer iterations (particularly for total_occupancy; once we get rid of this, we can go back to TG level counters)

Closes #7256

@github-actions
Copy link
Contributor

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       15 files  ±0         15 suites  ±0   6h 47m 43s ⏱️ + 2m 54s
  3 175 tests ±0    3 091 ✔️ ±0    83 💤  - 1  1 +1 
23 492 runs  ±0  22 588 ✔️ +2  903 💤  - 3  1 +1 

For more details on these failures, see this check.

Results for commit d786a95. ± Comparison against base commit 61cb719.

@fjetter fjetter merged commit 74d09fe into dask:main Nov 10, 2022
@fjetter fjetter deleted the duration_counter_prefix branch November 10, 2022 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler.total_occupancy is significant runtime cost

1 participant