Skip to content

Use of new dask CLI#6735

Merged
jrbourbeau merged 25 commits intodask:mainfrom
douglasdavis:use-new-dask-cli
Oct 14, 2022
Merged

Use of new dask CLI#6735
jrbourbeau merged 25 commits intodask:mainfrom
douglasdavis:use-new-dask-cli

Conversation

@douglasdavis
Copy link
Copy Markdown
Member

@douglasdavis douglasdavis commented Jul 16, 2022

This transitions existing CLI tooling to use the new dask CLI tool from dask/dask#9283

  • Moves tests to use dask foo instead of dask-foo
  • Removes some deprecated options and tests associated with those depcrecations.
  • Adds a warning to the dask-foo flavor single executables
  • Updates all occurrences in the documentation of the single executables

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 16, 2022

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       15 files  ±  0         15 suites  ±0   6h 14m 26s ⏱️ - 9m 10s
  3 143 tests  - 10    3 052 ✔️  - 17    85 💤 +1  6 +6 
23 251 runs   - 84  22 332 ✔️  - 98  913 💤 +8  6 +6 

For more details on these failures, see this check.

Results for commit 2cbf0b4. ± Comparison against base commit 044225a.

♻️ This comment has been updated with latest results.

@mrocklin
Copy link
Copy Markdown
Member

Two requests:

  1. Verify that the previous commands still work (seems like tests pass, so we should be good)
  2. Add a simple test showing that the new ones do the same thing?

@douglasdavis
Copy link
Copy Markdown
Member Author

Closing in favor of #6738

@mrocklin
Copy link
Copy Markdown
Member

mrocklin commented Jul 19, 2022 via email

@fjetter
Copy link
Copy Markdown
Member

fjetter commented Aug 18, 2022

We should probably use this as an opportunity to enforce a couple of deprecations. For instance, the following arguments are no longer supported. it would be great if the new CLI would not even show them.

  • --reconnect
  • --nprocs
  • --bokeh
  • `--bokeh-port

@douglasdavis
Copy link
Copy Markdown
Member Author

douglasdavis commented Aug 23, 2022

This should be ready to go upon reversing the dependence on my douglasdavis/dask@new-cli branch (I'll leave it as draft until that is reversed), see my comment at the partner dask/dask PR

@douglasdavis douglasdavis marked this pull request as ready for review September 1, 2022 15:07
Copy link
Copy Markdown
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the delay here @douglasdavis. I'm taking a look at this now. In the meantime, would you mind resolving the current merge conflict?

@douglasdavis
Copy link
Copy Markdown
Member Author

douglasdavis commented Oct 4, 2022

@jrbourbeau no worries! fixed the conflict and merged main in both dask/dask#9283 and here
(PS the merge conflict really had to do with the temporary overriding of the dask version requirement to the branch in dask/dask#9283)

Copy link
Copy Markdown
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @douglasdavis!

douglasdavis and others added 5 commits October 4, 2022 11:45
Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>
Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>
Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>
Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>
@douglasdavis
Copy link
Copy Markdown
Member Author

Thanks for the review @jrbourbeau! I've incorporated suggestions/responded to your comments-- happy to keep iterating.

@douglasdavis
Copy link
Copy Markdown
Member Author

Looks like test_deprecated_single_executable is hitting a 30 second time out sometimes. Should we adjust a timeout argument?

@jrbourbeau jrbourbeau mentioned this pull request Oct 12, 2022
7 tasks
@jrbourbeau
Copy link
Copy Markdown
Member

Pushed an update which resolves the test_deprecated_single_executable issue

Copy link
Copy Markdown
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @douglasdavis

@consideRatio
Copy link
Copy Markdown
Contributor

#6735 (comment)

We should probably use this as an opportunity to enforce a couple of deprecations. For instance, the following arguments are no longer supported. it would be great if the new CLI would not even show them.

  • --reconnect
  • --nprocs
  • --bokeh
  • --bokeh-port

Note that this became a breaking change with no entry in the changelog and entirely unreflected in the PR title.

@jrbourbeau
Copy link
Copy Markdown
Member

Thanks for reporting @consideRatio. I'm not sure I follow as I thought we had already been emitting warnings about these options (for example, this sort of thing). Which option(s) change was problematic for you?

@consideRatio
Copy link
Copy Markdown
Contributor

consideRatio commented Oct 24, 2022

Its nothing problematic about making the breaking change for me, but its a plus if its made a bit more visible via the changelog, PR title, PR description.

dask/helm-chart#338

I didnt see warnings in this case due to the nature of how dask-scheduler was used :/

@douglasdavis
Copy link
Copy Markdown
Member Author

I should have included something in the PR about removing the deprecations, apologies for that. It makes sense that automation tools would indeed make FutureWarnings harder to catch

@jrbourbeau
Copy link
Copy Markdown
Member

Yeah, that's a fair point. See #7178 for an update to the changelog

gjoseph92 pushed a commit to gjoseph92/distributed that referenced this pull request Oct 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants