I'm not sure this PR has the best method of printing/logging information:

I'm also unsure how to test KeyboardInterrupts.

I'm having difficult time with implementation that catches the KeyboardInterrupt. When I test it manually in a notebook, the code throws an KeyboardInterrupt when I manually send a single keyboard interruption and does not stop the training cleanly. Here's the traceback of the error I get:

KeyboardInterrupt               Traceback (most recent call last)
<ipython-input-5-3748455ece57> in <module>()
----> 9 df = _test_keyboardinterrupt()
<ipython-input-4-135866acf811> in _test_keyboardinterrupt()
---> 13     search.fit(X, y)
~/Developer/stsievert/dask-ml/dask_ml/model_selection/_incremental.py in fit(self, X, y, **fit_params)
--> 641         return default_client().sync(self._fit, X, y, **fit_params)
~/anaconda3/lib/python3.6/site-packages/distributed/client.py in sync(self, func, *args, **kwargs)
--> 753             return sync(self.loop, func, *args, **kwargs)
~/anaconda3/lib/python3.6/site-packages/distributed/utils.py in sync(loop, func, *args, **kwargs)
--> 329             e.wait(10)
~/anaconda3/lib/python3.6/threading.py in wait(self, timeout)
--> 551                 signaled = self._cond.wait(timeout)
~/anaconda3/lib/python3.6/threading.py in wait(self, timeout)
--> 299                     gotit = waiter.acquire(True, timeout)

KeyboardInterrupt: 

I'm learning a bit about this for a distributed issue. Will take a look later today.

Can you talk a bit about what the expected behavior of a KeyboardInterrupt halfway through is? As a user, I might expect the following

1. All currently scheduled fits are cancelled (cluster should become idle)
2. The usual attributes like best_estimator_ are present, and reflect the state of the world when the training was interrupted.
3. I have some way of knowing that the training was stopped early? An attribute on the model?

Overall, this looks a bit more complicated that adding better logging. It may be worth splitting into its own PR.

1. All currently scheduled fits are cancelled (cluster should become idle)
2. The usual attributes like best_estimator_ are present, and reflect the state of the world when the training was interrupted.
3. I have some way of knowing that the training was stopped early? An attribute on the model?

I would certainly expect 1 and 2. I intend the KeyboardInterrupt to be a mechanism for the busy data scientist to get on with their workflow (using the best estimator, etc).

Looks like a merge conflict.

verbose is now a float as before, but I always cast to a float now (unless a bool is given). I've updated #528 (comment) to reflect this change.

The CI is failing because contextlib.nullcontext is new in Python 3.7 (source). Do we want to upgrade the minimum Python version or use a NullContext in dask_ml._utils.py?

Ah, I didn't realize it was that new. I think defining something like this in dask_ml._compat

@contextlib.contextmanager
def nullcontext():
    yield

should suffice

I've also deprecated the scores_per_fit parameter in IncrementalSearchCV; the name should be fits_per_score.

Thanks!