Add pass-through arguments for scheduler/worker `--preload` modules. by asford · Pull Request #1634 · dask/distributed

asford · 2017-12-14T17:50:08Z

Adds support for pass-through command line arguments to worker and scheduler preload modules via an argv pattern similar to that used for compiler subcommand arguments. Intended to provide support "generic" preload modules with run-time configuration.

~~Extension of dask_setup interface will likely break existing preload implementations.~~ Extends preload interface with an optional dask_command module attribute.
Preload argument handling is ambiguous in the case of multiple modules. Only support a single preload module exposing the dask_command interface. Applications requiring multiple preload modules may declare a single "top-level" module chaining the required subcommands.
Test coverage and test updates.

mrocklin · 2017-12-14T18:01:30Z

Thanks for taking this on. First question is "is there a convention or common practice for doing this with the click library?" It seems like the sort of thing that has probably come up before.

…

On Thu, Dec 14, 2017 at 11:50 AM, Alexander Ford ***@***.***> wrote: *Work in progress and untested, this PR is intended to provide a forum for feedback on this implementation.* Adds support for pass-through command line arguments to worker and scheduler preload modules via an argv pattern similar to that used for compiler subcommand arguments. Intended to provide support "generic" preload modules with run-time configuration. - Extension of dask_setup interface will likely break existing preload implementations. - Preload argument handling is *ambiguous* in the case of multiple modules. Should the full argv set be passed to each module? Should the argv be specified on a per-module basis? - Test coverage and test updates. ------------------------------ You can view, comment on, or merge this pull request online at: #1634 Commit Summary - Add pass-through arguments for scheduler/worker `--preload` modules. File Changes - *M* distributed/cli/dask_scheduler.py <https://github.com/dask/distributed/pull/1634/files#diff-0> (6) - *M* distributed/cli/dask_worker.py <https://github.com/dask/distributed/pull/1634/files#diff-1> (6) - *M* distributed/nanny.py <https://github.com/dask/distributed/pull/1634/files#diff-2> (5) - *M* distributed/preloading.py <https://github.com/dask/distributed/pull/1634/files#diff-3> (11) - *M* distributed/worker.py <https://github.com/dask/distributed/pull/1634/files#diff-4> (5) Patch Links: - https://github.com/dask/distributed/pull/1634.patch - https://github.com/dask/distributed/pull/1634.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1634>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszJyaQfDMR-t7AW9DPyw_X-MnpLviks5tAV_QgaJpZM4RCbxv> .

asford · 2017-12-14T18:47:13Z

I've looked through the click documentation and think this would be possible. The "clang-like" interface seems harder to support via click, but I'm not tied to that concept. Would a model that passes any "unrecognized" arguments into the preloaded module be preferable to the --preload_args interface? This would result in:

dask-scheduler --preload scheduler_preload.py --flags-for-preloads=flag_value args-for-preloads

This interface seems better, but may complicate (a) unknown argument error handling and (b) the use of multiple preload modules.

My main concern with building the "preload" logic around click is that it ties the preload implementation to a specific command-line parser, but I can see an argument for standardization. I'll explore this more and prototype a click-based system.

jakirkham · 2017-12-14T19:31:48Z

The AppVeyor build appeared to have hung, but didn't terminate itself. So took the liberty of canceling it.

Edit: Should add the previous build for this PR did the exact same thing. Probably worth investigating if there is something here causing it.

asford · 2017-12-14T20:33:12Z

@jakirkham Sorry, my intention was to skip CI for this pull until we've landed on an implementation. Would you mind checking if the [skip ci] flag I've added is now working?

asford · 2017-12-14T20:36:09Z

@mrocklin It turned out that a click based implementation was easier than I expected. I've updated this interface to support delegation into a click.Command exposed in the preload module attribute dask_command.

This obviously needs testing and documentation updates, but unfortunately I may not be able to push this to a mergeable state until early next week. Would you mind commenting on the interface as implemented?

mrocklin · 2017-12-14T20:37:52Z

Will do, although probably not until later tonight or early tomorrow. I'm out for the day. Thanks again for pursuing this and looking into click conventions.

…

On Thu, Dec 14, 2017 at 2:36 PM, Alexander Ford ***@***.***> wrote: @mrocklin <https://github.com/mrocklin> It turned out that a click based implementation was easier than I expected. I've updated this interface to support delegation into a click.Command exposed in the preload module attribute dask_command. This obviously needs testing and documentation updates, but unfortunately I may not be able to push this to a mergeable state until early next week. Would you mind commenting on the interface as implemented? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1634 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszKqnGcCcZqTHxYzS0FhZ2jaxYpCBks5tAYa5gaJpZM4RCbxv> .

jakirkham · 2017-12-14T21:02:07Z

Sorry, my intention was to skip CI for this pull until we've landed on an implementation.

No worries. Just figured I'd mention it in case this was unexpected or you were trying to debug the CI.

Would you mind checking if the [skip ci] flag I've added is now working?

Sorry. Don't seem to be seeing it. If you add [ci skip] to the commit message, that usually works.

mrocklin · 2017-12-15T01:14:45Z

In reviewing this it would be useful to see an example of how it works. Perhaps add a test following the pattern in distributed/cli/tests/test_dask_scheduler.py::test_preload_file?

mrocklin · 2017-12-18T18:07:09Z

I'm finding review this to be a bit of a challenge due to my own lack of familiarity with the click library.

CC'ing @danielfrg in case cd he has time and has experience with common conventions on how to pass through keyword arguments to downstream commands. @danielfrg , in brief the dask-scheduler and dask-worker commands can specify user-defined scripts to run at a certain point in the setup process. This is often used to set up logging, add additional services, etc.. We would like to pass keywords down from the main dask-scheduler call to these scripts. Here is an example.

dask-scheduler ... --preload my-script.py --key1 value --key2 value

Where --key1 and --key2 are intended for my-script.py

danielfrg · 2017-12-18T20:39:50Z

I kinda like what wait-for-it does you pass a -- and after that the extra command and flags.

It will look like:

dask-scheduler --port 8080 -- python my-script.py --key1 value1 --key2 value2

mrocklin · 2017-12-21T20:43:45Z

I kinda like what wait-for-it does you pass a -- and after that the extra command and flags.

This would presumably go outside of standard click approaches though, yes? I'm not a good judge of how fancy this is or what conventions are these days, but I'm a little hesitant about this.

I think what I'm looking for is a way to pass keywords down to subcommands. I want the click equivalent of passing **kwargs

def f(x=1, y=2, **kwargs):
    tmp = g(**kwargs)
    return x + y + tmp

def g(z=3, **kwargs):
    ...

asford · 2017-12-22T00:16:05Z

Yes, the "--" delimited approach is hard to do with click. I believe you'd need to layer ad additional parsing layer on to the click system. This implementation just passed any args that aren't recognised by the dask parser into a *single* dynamically loaded click command.

On Thu, Dec 21, 2017, 12:43 Matthew Rocklin ***@***.***> wrote: I kinda like what wait-for-it does you pass a -- and after that the extra command and flags. This would presumably go outside of standard click approaches though, yes? I'm not a good judge of how fancy this is or what conventions are these days, but I'm a little hesitant about this. I think what I'm looking for is a way to pass keywords down to subcommands. I want the click equivalent of passing **kwargs def f(x=1, y=2, **kwargs): tmp = g(**kwargs) return x + y + tmp def g(z=3, **kwargs): ... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1634 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARQqJrn6VSjglsjP2YL2zL7WgAPxYEvks5tCsMDgaJpZM4RCbxv> .

-- Alex Ford a.sewall.ford@gmail.com 206.659.6559

mrocklin · 2017-12-22T01:02:39Z

Any thoughts on the implementation in this PR @danielfrg ? (again, assuming you have the time for review) On Thu, Dec 21, 2017 at 7:16 PM, Alexander Ford <notifications@github.com> wrote:

…

Yes, the "--" delimited approach is hard to do with click. I believe you'd need to layer ad additional parsing layer on to the click system. This implementation just passed any args that aren't recognised by the dask parser into a *single* dynamically loaded click command. On Thu, Dec 21, 2017, 12:43 Matthew Rocklin ***@***.***> wrote: > I kinda like what wait-for-it does you pass a -- and after that the extra > command and flags. > > This would presumably go outside of standard click approaches though, > yes? I'm not a good judge of how fancy this is or what conventions are > these days, but I'm a little hesitant about this. > > I think what I'm looking for is a way to pass keywords down to > subcommands. I want the click equivalent of passing **kwargs > > def f(x=1, y=2, **kwargs): > tmp = g(**kwargs) > return x + y + tmp > def g(z=3, **kwargs): > ... > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#1634 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ AARQqJrn6VSjglsjP2YL2zL7WgAPxYEvks5tCsMDgaJpZM4RCbxv> > . > -- Alex Ford ***@***.*** 206.659.6559 <(206)%20659-6559> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1634 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszAV-R34js5bwAHNAhfqw9ULtVbJ8ks5tCvTGgaJpZM4RCbxv> .

mrocklin · 2017-12-24T17:21:46Z

OK, I gave this a shot with the following example:

# click_test.py
@click.command(context_settings={'ignore_unknown_options': True})
@click.option('--foo', type=str)
def dask_setup(foo):
    print(foo)

if __name__ == '__main__':
    dask_setup()

$ dask-scheduler --preload click_test.py --foo 5
distributed.utils - INFO - Reload module click_test from .py file
Usage: dask-scheduler [OPTIONS] [PRELOAD_ARGV]...

Error: Additional preload arguments specified, but --preload target did not expose command.

It's not clear to me how to use this work properly yet. I appreciate the informative error messages.
They may not be enough for new users though. Perhaps this needs documentation? Perhaps an example or two?

asford · 2017-12-29T04:12:32Z

Rebased, added initial test coverage and documented. This should now be ready for review.

asford · 2017-12-29T10:11:01Z

It looks like the CI error was py2.7 distributed/tests/test_worker.py::test_statistical_profiling_cycle. I haven't been able to repro locally in a conda-based python 2.7 development environment. Is it possible that this test is flaky?

mrocklin · 2017-12-29T16:28:35Z

Yes, that test is known to be flaky. I have a fix in a local branch. Feel free to disregard for now (there are a few more of these lingering around as well).

…

On Fri, Dec 29, 2017 at 2:11 AM, Alexander Ford ***@***.***> wrote: It looks like the CI error was py2.7 distributed/tests/test_worker. py::test_statistical_profiling_cycle <https://travis-ci.org/dask/distributed/jobs/322776454#L2209>. I haven't been able to repro locally in a conda-based python 2.7 development environment. Is it possible that this test is flaky? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1634 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszCOLFUY4DRX0ojL9-SFrOiBe3SvAks5tFLq1gaJpZM4RCbxv> .

asford · 2018-01-16T18:27:42Z

@mrocklin @danielfrg Friendly ping on this pull. Happy to revise the implementation, but would like to have a +1/-1 and plan for a merge.

mrocklin

In principle the approach looks good to me. I've raised a couple of points below.

mrocklin · 2018-01-16T18:33:05Z

+@click.command()
+@click.option("--passthrough", type=str, default="default")
+def dask_command(passthrough):
+    _config["passthrough"] = passthrough


I'm inclined to just put the command bits on dask_setup rather than add a new special function. Presumably these run at the same time anyway?

Understood, fixed below.

mrocklin · 2018-01-16T18:34:31Z

+        return value
+
+    if value and not ctx.params.get("preload", None):
+        raise click.UsageError("Additional preload arguments specified without --preload target.")


This might be valuable to include in a test. We want to make ensure that operations like dask-scheduler --bad-keyword do genuinely raise informative arguments. In these cases we probably don't want to talk about --preload but rather just mention that there are unknown keywords (and perhaps list what they are).

Updated in commit below to match click error messages for unknown options and extra arguments.

Adds support for optional command-line parsing to worker and scheduler --preload modules. Modules *may* support an additional `dask_command` module attribute, which must provide a click.Command to be used in parsing command-line options from the root command-line interface. `dask_command` will be invoked with any "additional" arguments specified on the command line interface, and is expected to configure the preload module. Following calls to `dask_setup` and `dask_teardown` provide the target worker or scheduler objects.

Adds initial test coverage for click-based preload arguments. Fixup errors in preload argument specification and preload main handling.

From review, convert to parsing options on dask_setup if provided as a click command. Fixup error messages from unknown/extra options to match click conventions if preload argment module isn't provided.

mrocklin · 2018-01-26T14:20:53Z

An experiment and results:

# foo.py

import click

@click.command()
@click.option("--cmd", type=str, default='default')
def dask_setup(scheduler, cmd):
    print("Hello")
    print(cmd)

print("running")

$ dask-scheduler --preload foo.py --cmd bar
distributed.utils - INFO - Reload module foo from .py file
running
running
distributed.scheduler - INFO - -----------------------------------------------
distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO -   Scheduler at: tcp://192.168.50.100:8786
distributed.scheduler - INFO -       bokeh at:                     :8787
distributed.utils - INFO - Reload module foo from .py file
running
Hello
bar
distributed.scheduler - INFO - Local Directory:    /tmp/scheduler-dcacf5e_
distributed.scheduler - INFO - -----------------------------------------------

Why the reloads? Why did running appear three times?

mrocklin · 2018-01-26T14:21:22Z

Otherwise everything feels smooth on my end

asford · 2018-01-26T19:10:19Z

This occurs when a source preload .py file is specified for preload, rather than a preload module name, due to the explicit reload call after import_module in distributed.utils.import_file.

The preload system performs multiple imports of the given name because the target preload must be accessed (A) to check for a click-ed dask_setup function and (B) to extract the preload interface for the actual dask_setup call. This doesn't result in triple-execution of a preload module (as opposed to preload file), as the module is retrieved from the import cache in (B).

This forced reload prevents the import system from using the cached initial .py import, so in (A) you see (1) an initial import of the file and (2) a forced reload of the file then in (B) (3) a final forced reload before the dask_setup call.

If this behavior seems too spooky it would be possible to fully parse the --preload component during dask-(worker|scheduler) click argument parsing then pass the complete interface via the preload argument value, as opposed to passing the string (file|module) name as currently implemented. However, this is really a workaround for the somewhat-strange multiple-load behavior in distributed.utils.import_file.

asford · 2018-01-26T19:16:21Z

Example tracebacks below, lightly trimmed for clarity:

Preload Source

fordas@salish:~/workspace/distributed_dev$ cat test_plugin.py
import traceback

import click

@click.command()
@click.option("--foo", default="bar")
def dask_setup(scheduler, foo):
    print("dask_setup test_plugin")
    print(foo)

print("run test_plugin")
traceback.print_stack()

Preload via module name, single import

fordas@salish:~/workspace/distributed_dev$ PYTHONPATH=`pwd` dask-scheduler --preload test_plugin --port 88786
run test_plugin
[...]
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/preloading.py", line 32, in validate_preload_argv
    preload_modules = _import_modules(ctx.params.get("preload"))
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/preloading.py", line 97, in _import_modules
    import_module(name)
  File "/home/fordas/.conda/envs/distributed_dev/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 985, in _gcd_import
  File "<frozen importlib._bootstrap>", line 968, in _find_and_load
  File "<frozen importlib._bootstrap>", line 957, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 697, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/fordas/workspace/distributed_dev/test_plugin.py", line 12, in <module>
    traceback.print_stack()
distributed.scheduler - INFO - -----------------------------------------------
distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO -   Scheduler at:    tcp://10.128.0.2:23250
distributed.scheduler - INFO -       bokeh at:                    :45101
dask_setup test_plugin
bar
distributed.scheduler - INFO - Local Directory:    /tmp/scheduler-qaix2em4
distributed.scheduler - INFO - -----------------------------------------------
^Cdistributed.scheduler - INFO - End scheduler at 'tcp://:88786'

Import via filename, repeated import

fordas@salish:~/workspace/distributed_dev$ PYTHONPATH=`pwd` dask-scheduler --preload test_plugin.py --port 88786
distributed.utils - INFO - Reload module test_plugin from .py file
run test_plugin
[...]
  File "/home/fordas/.conda/envs/distributed_dev/lib/python3.5/site-packages/click/core.py", line 78, in invoke_param_callback
    return callback(ctx, param, value)
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/preloading.py", line 32, in validate_preload_argv
    preload_modules = _import_modules(ctx.params.get("preload"))
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/preloading.py", line 92, in _import_modules
    module = import_file(name)[0]
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/utils.py", line 1022, in import_file
    loaded.append(reload(import_module(name)))
  File "/home/fordas/.conda/envs/distributed_dev/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 985, in _gcd_import
  File "<frozen importlib._bootstrap>", line 968, in _find_and_load
  File "<frozen importlib._bootstrap>", line 957, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 697, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/fordas/workspace/distributed_dev/test_plugin.py", line 12, in <module>
    traceback.print_stack()

run test_plugin
[...]
  File "/home/fordas/.conda/envs/distributed_dev/lib/python3.5/site-packages/click/core.py", line 78, in invoke_param_callback
    return callback(ctx, param, value)
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/preloading.py", line 32, in validate_preload_argv
    preload_modules = _import_modules(ctx.params.get("preload"))
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/preloading.py", line 92, in _import_modules
    module = import_file(name)[0]
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/utils.py", line 1022, in import_file
    loaded.append(reload(import_module(name)))
  File "/home/fordas/.conda/envs/distributed_dev/lib/python3.5/importlib/__init__.py", line 166, in reload
    _bootstrap._exec(spec, module)
  File "<frozen importlib._bootstrap>", line 626, in _exec
  File "<frozen importlib._bootstrap_external>", line 697, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/fordas/workspace/distributed_dev/test_plugin.py", line 12, in <module>
    traceback.print_stack()
distributed.scheduler - INFO - -----------------------------------------------
distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO -   Scheduler at:    tcp://10.128.0.2:23250
distributed.scheduler - INFO -       bokeh at:                    :34427
distributed.utils - INFO - Reload module test_plugin from .py file
run test_plugin
  File "/home/fordas/.conda/envs/distributed_dev/bin/dask-scheduler", line 11, in <module>
    load_entry_point('distributed', 'console_scripts', 'dask-scheduler')()
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/cli/dask_scheduler.py", line 145, in go
    main()
  File "/home/fordas/.conda/envs/distributed_dev/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/fordas/.conda/envs/distributed_dev/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/fordas/.conda/envs/distributed_dev/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/fordas/.conda/envs/distributed_dev/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/cli/dask_scheduler.py", line 125, in main
    preload_modules(preload, parameter=scheduler, file_dir=local_directory, argv=preload_argv)
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/preloading.py", line 123, in preload_modules
    imported_modules = _import_modules(names, file_dir=file_dir)
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/preloading.py", line 90, in _import_modules
    module = import_file(copy_dst)[0]
  File "/home/fordas/workspace/distributed_dev/src/distributed/distributed/utils.py", line 1022, in import_file
    loaded.append(reload(import_module(name)))
  File "/home/fordas/.conda/envs/distributed_dev/lib/python3.5/importlib/__init__.py", line 166, in reload
    _bootstrap._exec(spec, module)
  File "<frozen importlib._bootstrap>", line 626, in _exec
  File "<frozen importlib._bootstrap_external>", line 697, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/tmp/scheduler-p65s4rto/test_plugin.py", line 12, in <module>
    traceback.print_stack()
dask_setup test_plugin
bar
distributed.scheduler - INFO - Local Directory:    /tmp/scheduler-p65s4rto
distributed.scheduler - INFO - -----------------------------------------------
^Cdistributed.scheduler - INFO - End scheduler at 'tcp://:88786'

mrocklin · 2018-02-12T15:58:34Z

I again apologize for the long delay here. I've merged in master and pushed. Merging on passed tests.

mrocklin · 2018-02-12T17:44:57Z

This is in. Thank you for your work and patience @asford

asford mentioned this pull request Dec 14, 2017

Add min and max worker flags to adaptive cluster #1618

Closed

asford force-pushed the preload_passthrough branch from 3b99652 to 3fc9641 Compare December 14, 2017 17:56

asford changed the title ~~WIP Add pass-through arguments for scheduler/worker --preload modules.~~ [WIP] Add pass-through arguments for scheduler/worker --preload modules. Dec 14, 2017

asford changed the title ~~[WIP] Add pass-through arguments for scheduler/worker --preload modules.~~ [WIP] [skip ci] Add pass-through arguments for scheduler/worker --preload modules. Dec 14, 2017

asford force-pushed the preload_passthrough branch 2 times, most recently from 4b0d408 to f209527 Compare December 14, 2017 20:40

asford force-pushed the preload_passthrough branch from 6dbabe5 to 8b345f5 Compare December 29, 2017 03:39

asford changed the title ~~[WIP] [skip ci] Add pass-through arguments for scheduler/worker --preload modules.~~ Add pass-through arguments for scheduler/worker --preload modules. Dec 29, 2017

mrocklin reviewed Jan 16, 2018

View reviewed changes

Alex Ford added 3 commits January 26, 2018 03:38

Add pass-through arguments for scheduler/worker --preload modules.

39dcfba

Add initial test coverage for scheduler preload arguments.

d77026d

Adds initial test coverage for click-based preload arguments. Fixup errors in preload argument specification and preload main handling.

Alex Ford added 4 commits January 26, 2018 03:40

Document dask_command support.

bccc8f5

Clarify preload argument validation error messages.

25c4189

Delint.

afc79f8

Review, parse dask_setup as click command and clarify errors.

7fcdd79

From review, convert to parsing options on dask_setup if provided as a click command. Fixup error messages from unknown/extra options to match click conventions if preload argment module isn't provided.

asford force-pushed the preload_passthrough branch from cc35032 to 7fcdd79 Compare January 26, 2018 03:40

mrocklin added 2 commits February 12, 2018 10:56

Merge branch 'master' into preload_passthrough

c6282ea

break paragraph in docs

fd48ded

flake8

3b55107

mrocklin merged commit cc908e7 into dask:master Feb 12, 2018

Uh oh!

Conversation

asford commented Dec 14, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mrocklin commented Dec 14, 2017 via email

Uh oh!

asford commented Dec 14, 2017

Uh oh!

jakirkham commented Dec 14, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asford commented Dec 14, 2017

Uh oh!

asford commented Dec 14, 2017

Uh oh!

mrocklin commented Dec 14, 2017 via email

Uh oh!

jakirkham commented Dec 14, 2017

Uh oh!

mrocklin commented Dec 15, 2017

Uh oh!

mrocklin commented Dec 18, 2017

Uh oh!

danielfrg commented Dec 18, 2017

Uh oh!

mrocklin commented Dec 21, 2017

Uh oh!

asford commented Dec 22, 2017 via email

Uh oh!

mrocklin commented Dec 22, 2017 via email

Uh oh!

mrocklin commented Dec 24, 2017

Uh oh!

asford commented Dec 29, 2017

Uh oh!

asford commented Dec 29, 2017

Uh oh!

mrocklin commented Dec 29, 2017 via email

Uh oh!

asford commented Jan 16, 2018

Uh oh!

mrocklin left a comment

Choose a reason for hiding this comment

Uh oh!

mrocklin Jan 16, 2018

Choose a reason for hiding this comment

Uh oh!

asford Jan 26, 2018

Choose a reason for hiding this comment

Uh oh!

mrocklin Jan 16, 2018

Choose a reason for hiding this comment

Uh oh!

asford Jan 26, 2018

Choose a reason for hiding this comment

Uh oh!

mrocklin commented Jan 26, 2018

Uh oh!

mrocklin commented Jan 26, 2018

Uh oh!

asford commented Jan 26, 2018

Uh oh!

asford commented Jan 26, 2018

Preload Source

Preload via module name, single import

Import via filename, repeated import

Uh oh!

mrocklin commented Feb 12, 2018

Uh oh!

mrocklin commented Feb 12, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

asford commented Dec 14, 2017 •

edited

Loading

jakirkham commented Dec 14, 2017 •

edited

Loading