LocalCluster crashed with: distributed.core - ERROR - 'tuple' object does not support item assignment #4625
Description
What happened:
Python programs crashes with the following error message:
distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
msgs = await comm.read()
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
msg = nested_deserialize(msg)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
return replace_inner(x)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
x[k] = replace_inner(v)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
x[k] = deserialize(v.header, v.frames)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
return loads(header, frames)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
await self.handle_stream(
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
msgs = await comm.read()
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
msg = nested_deserialize(msg)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
return replace_inner(x)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
x[k] = replace_inner(v)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
x[k] = deserialize(v.header, v.frames)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
return loads(header, frames)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f7e6575cc40>>, <Task finished name='Task-22' coro=<Worker.handle_scheduler() done, defined at /home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/worker.py:988> exception=TypeError("'tuple' object does not support item assignment")>)
Traceback (most recent call last):
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
ret = callback()
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
future.result()
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
await self.handle_stream(
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
msgs = await comm.read()
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
msg = nested_deserialize(msg)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
return replace_inner(x)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
x[k] = replace_inner(v)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
x[k] = deserialize(v.header, v.frames)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
return loads(header, frames)
File "/home/ffischer/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
What you expected to happen:
No error: persist or compute call returns as usual
Minimal Complete Verifiable Example:
import pandas as pd
import dask.dataframe as dd
from dask.distributed import Client, LocalCluster
# if you dont create a local cluster and the client everything works
cluster = LocalCluster(processes=False)
client = Client(cluster, set_as_default=True)
def do_nothing(df, other_stuff):
return df
ddf = (
dd.from_pandas(pd.DataFrame(range(20)), npartitions=5)
.map_partitions(do_nothing, other_stuff=1, meta={0: 'i8'})
.persist()
)Anything else we need to know?:
Without creating the LocalCluster and Client everything works as expected, so I would guess this issue has something to do with the LocalCluster
With dask version 2021.2 everything works correctly
Environment:
- Dask version: 2021.3
- Python version: 3.8.8
- Operating System: Ubuntu 20.04.2 LTS
- Install method (conda, pip, source): conda install dask=2021.3