Skip to content

Temporarily restrict bokeh<3#7219

Merged
jrbourbeau merged 2 commits intodask:mainfrom
jrbourbeau:pin-bokeh-3
Oct 28, 2022
Merged

Temporarily restrict bokeh<3#7219
jrbourbeau merged 2 commits intodask:mainfrom
jrbourbeau:pin-bokeh-3

Conversation

@jrbourbeau
Copy link
Copy Markdown
Member

_min_bokeh_version = "2.1.1"

if parse_version(bokeh.__version__) < parse_version(_min_bokeh_version):
if BOKEH_VERSION < parse_version(_min_bokeh_version):
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that I'm not explicitly adding or BOKEH_VERSION.major > 2 here as that causes cluster creation to fail hard when using bokeh >= 3 (see the traceback below)

Traceback:
In [1]: from distributed import Client

In [2]: c = Client()
/Users/james/projects/dask/distributed/distributed/dashboard/core.py:20: UserWarning:
Dask needs bokeh >= 2.1.1, < 3 for the dashboard.
Continuing without the dashboard.
  warnings.warn(
2022-10-28 10:06:30,392 - distributed.deploy.spec - WARNING - Cluster closed without starting up
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
File ~/projects/dask/distributed/distributed/deploy/spec.py:308, in SpecCluster._start(self)
    307     cls = import_term(cls)
--> 308 self.scheduler = cls(**self.scheduler_spec.get("options", {}))
    309 self.scheduler = await self.scheduler

File ~/projects/dask/distributed/distributed/scheduler.py:3503, in Scheduler.__init__(self, loop, delete_interval, synchronize_worker_interval, services, service_kwargs, allowed_failures, extensions, validate, scheduler_file, security, worker_ttl, idle_timeout, interface, host, port, protocol, dashboard_address, dashboard, http_prefix, preload, preload_argv, plugins, contact_address, transition_counter_max, jupyter, **kwargs)
   3502         http_server_modules.append("distributed.http.scheduler.missing_bokeh")
-> 3503 routes = get_handlers(
   3504     server=self, modules=http_server_modules, prefix=http_prefix
   3505 )
   3506 self.start_http_server(routes, dashboard_address, default_port=8787)

File ~/projects/dask/distributed/distributed/http/utils.py:40, in get_handlers(server, modules, prefix)
     39 for module_name in modules:
---> 40     module = importlib.import_module(module_name)
     41     _routes.extend(module.routes)

File ~/mambaforge/envs/distributed/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/projects/dask/distributed/distributed/http/scheduler/missing_bokeh.py:3, in <module>
      1 from __future__ import annotations
----> 3 from distributed.dashboard.core import _min_bokeh_version
      4 from distributed.http.utils import RequestHandler, redirect

File ~/projects/dask/distributed/distributed/dashboard/core.py:24, in <module>
     20     warnings.warn(
     21         f"\nDask needs bokeh >= {_min_bokeh_version}, < 3 for the dashboard."
     22         "\nContinuing without the dashboard."
     23     )
---> 24     raise ImportError(f"Dask needs bokeh >= {_min_bokeh_version}, < 3")
     27 def BokehApplication(applications, server, prefix="/", template_variables=None):

ImportError: Dask needs bokeh >= 2.1.1, < 3

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Input In [2], in <cell line: 1>()
----> 1 c = Client()

File ~/projects/dask/distributed/distributed/client.py:982, in Client.__init__(self, address, loop, timeout, set_as_default, scheduler_file, security, asynchronous, name, heartbeat_interval, serializers, deserializers, extensions, direct_to_workers, connection_limit, **kwargs)
    979 preload_argv = dask.config.get("distributed.client.preload-argv")
    980 self.preloads = preloading.process_preloads(self, preload, preload_argv)
--> 982 self.start(timeout=timeout)
    983 Client._instances.add(self)
    985 from distributed.recreate_tasks import ReplayTaskClient

File ~/projects/dask/distributed/distributed/client.py:1172, in Client.start(self, **kwargs)
   1170     self._started = asyncio.ensure_future(self._start(**kwargs))
   1171 else:
-> 1172     sync(self.loop, self._start, **kwargs)

File ~/projects/dask/distributed/distributed/utils.py:406, in sync(loop, func, callback_timeout, *args, **kwargs)
    404 if error:
    405     typ, exc, tb = error
--> 406     raise exc.with_traceback(tb)
    407 else:
    408     return result

File ~/projects/dask/distributed/distributed/utils.py:379, in sync.<locals>.f()
    377         future = asyncio.wait_for(future, callback_timeout)
    378     future = asyncio.ensure_future(future)
--> 379     result = yield future
    380 except Exception:
    381     error = sys.exc_info()

File ~/mambaforge/envs/distributed/lib/python3.10/site-packages/tornado/gen.py:762, in Runner.run(self)
    759 exc_info = None
    761 try:
--> 762     value = future.result()
    763 except Exception:
    764     exc_info = sys.exc_info()

File ~/projects/dask/distributed/distributed/client.py:1238, in Client._start(self, timeout, **kwargs)
   1235 elif self._start_arg is None:
   1236     from distributed.deploy import LocalCluster
-> 1238     self.cluster = await LocalCluster(
   1239         loop=self.loop,
   1240         asynchronous=self._asynchronous,
   1241         **self._startup_kwargs,
   1242     )
   1243     address = self.cluster.scheduler_address
   1245 self._gather_semaphore = asyncio.Semaphore(5)

File ~/projects/dask/distributed/distributed/deploy/spec.py:398, in SpecCluster.__await__.<locals>._()
    396 async def _():
    397     if self.status == Status.created:
--> 398         await self._start()
    399     await self.scheduler
    400     await self._correct_state()

File ~/projects/dask/distributed/distributed/deploy/spec.py:319, in SpecCluster._start(self)
    317 self.status = Status.failed
    318 await self._close()
--> 319 raise RuntimeError(f"Cluster failed to start: {e}") from e

RuntimeError: Cluster failed to start: Dask needs bokeh >= 2.1.1, < 3

Instead the changes here are meant to let clusters spin up and be used for computation (bokeh isn't needed for this) and then when a user attempts to use the dashboard they'll encounter this message about the bokeh version restrictions

Screen Shot 2022-10-28 at 10 23 08 AM

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should also note that this comes into play because bokeh is an optional dependency for distributed (only needed if users want to use the dashboard). So something like

pip install distributed bokeh

could pull in bokeh=3.

@jrbourbeau jrbourbeau mentioned this pull request Oct 28, 2022
6 tasks
@jrbourbeau
Copy link
Copy Markdown
Member Author

Thanks for reviewing @fjetter -- I'll merge after CI finishes if there are no related failures

@jrbourbeau jrbourbeau mentioned this pull request Oct 28, 2022
@github-actions
Copy link
Copy Markdown
Contributor

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       15 files  ±0         15 suites  ±0   6h 40m 22s ⏱️ + 20m 0s
  3 160 tests ±0    3 075 ✔️ ±  0    84 💤 ±  0  1 ±0 
23 381 runs  +1  22 479 ✔️ +64  901 💤  - 60  1  - 3 

For more details on these failures, see this check.

Results for commit f02c356. ± Comparison against base commit cd3554a.

@jrbourbeau jrbourbeau merged commit f31fbde into dask:main Oct 28, 2022
@jrbourbeau jrbourbeau deleted the pin-bokeh-3 branch October 28, 2022 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants