Skip to content

Lots of warnings about Exception while handling op get_data #71

@TomAugspurger

Description

@TomAugspurger

The logs at https://github.com/rapidsai/dask-upstream-testing/actions/runs/16301240585/job/46036126104 include many messages like

2025-07-15 19:26:11,032 - distributed.core - ERROR - Exception while handling op get_data
Traceback (most recent call last):
  File "/__w/dask-upstream-testing/dask-upstream-testing/.venv/lib/python3.12/site-packages/distributed/core.py", line 834, in _handle_comm
    result = await result
             ^^^^^^^^^^^^
  File "/__w/dask-upstream-testing/dask-upstream-testing/.venv/lib/python3.12/site-packages/distributed/core.py", line 981, in wrapper
    return await func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/dask-upstream-testing/dask-upstream-testing/.venv/lib/python3.12/site-packages/distributed/worker.py", line 1794, in get_data
    compressed = await comm.write(msg, serializers=serializers)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/dask-upstream-testing/dask-upstream-testing/.venv/lib/python3.12/site-packages/distributed/utils.py", line 818, in wrapper
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/dask-upstream-testing/dask-upstream-testing/.venv/lib/python3.12/site-packages/distributed_ucxx/ucxx.py", line 505, in write
    synchronize_stream(0)
  File "/__w/dask-upstream-testing/dask-upstream-testing/.venv/lib/python3.12/site-packages/distributed_ucxx/ucxx.py", line 95, in synchronize_stream
    stream.synchronize()
  File "/__w/dask-upstream-testing/dask-upstream-testing/.venv/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/driver.py", line 2363, in synchronize
    driver.cuStreamSynchronize(self.handle)
  File "/__w/dask-upstream-testing/dask-upstream-testing/.venv/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/driver.py", line 395, in safe_cuda_api_call
    return self._check_cuda_python_error(fname, libfn(*args))
                                                ^^^^^^^^^^^^
  File "cuda/bindings/driver.pyx", line 31539, in cuda.bindings.driver.cuStreamSynchronize
  File "cuda/bindings/driver.pyx", line 5547, in cuda.bindings.driver.CUstream.__cinit__
TypeError: an integer is required
an integer is required

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions