Add Client.log_on_scheduler method to help correlating cluster logs with client-side events.#7179
Add Client.log_on_scheduler method to help correlating cluster logs with client-side events.#7179hendrikmakait wants to merge 2 commits intodask:mainfrom
Client.log_on_scheduler method to help correlating cluster logs with client-side events.#7179Conversation
Unit Test ResultsSee test report for an extended history of previous test failures. This is useful for diagnosing flaky tests. 15 files ± 0 15 suites ±0 6h 19m 9s ⏱️ - 15m 42s For more details on these failures, see this check. Results for commit a7b1540. ± Comparison against base commit 8f25111. |
jacobtomlinson
left a comment
There was a problem hiding this comment.
This seems like a reasonable change to me if you want to ensure that a log message from the client is inserted at a specific point in the scheduler log (obviously the comm may delay this a little).
I wonder if we want to think a bit more about how logs are aggregated and viewed from the client, scheduler and workers though. Right now cluster objects have the ability to grab logs and show them next to each other, but perhaps we need to go further and store logs in a more modern timestamped format that can be interlaced accurately down the line. This could either be both in a logging platform outside of Dask or within the widgets we have now.
|
@jacobtomlinson, totally agreed. I feel like |
|
Yeah, I totally agree this is a reasonable addition, but it definitely feels like a workaround. |
|
@hendrikmakait seems this is out of date now, do we still want to do this? |
I can resolve the merge conflicts but given a lack of interest and the fact that this is more of a workaround than a long-term improvement I'm also fine with closing. |
|
-1 from me too. I'd rather have a nice way of putting client, scheduler and worker logs side by side. |
This PR adds a
Client.log_on_schedulermethod that logs the provided message in the scheduler's logs. I implemented a similar helper for coiled/benchmarks/pull/473 and figured this might be generally helpful to allow users to inject additional timestamped information into the cluster logs.pre-commit run --all-files