You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Mar 6, 2026. It is now read-only.
We use luigi in our project and after recent update of our python dependencies we started to receive following errors.
/usr/local/lib/python3.10/site-packages/luigi/worker.py 701 Luigi unexpected framework error while scheduling MyTask()
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/luigi/worker.py", line 796, in add
current = queue.get()
File "<string>", line 2, in get
File "/usr/local/lib/python3.10/multiprocessing/managers.py", line 833, in _callmethod
raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Unserializable message: Traceback (most recent call last):
File "/usr/local/lib/python3.10/multiprocessing/managers.py", line 308, in serve_client
send(msg)
File "/usr/local/lib/python3.10/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/local/lib/python3.10/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object
Workaround
I found out that dependency that caused it was google-auth. Bringing it down from 2.27.0 to 2.23.2 while keeping other dependencies updated solved the issue. It looks like this PR may be related to this behaviour.
Background
We use luigi with parallel_scheduling=true option which means multiprocessing.Queue is used underneath and it requires every object in it to be pickle-able.
Error description
We use luigi in our project and after recent update of our python dependencies we started to receive following errors.
Workaround
I found out that dependency that caused it was
google-auth. Bringing it down from2.27.0to2.23.2while keeping other dependencies updated solved the issue. It looks like this PR may be related to this behaviour.Background
We use luigi with
parallel_scheduling=trueoption which meansmultiprocessing.Queueis used underneath and it requires every object in it to be pickle-able.Environment details
google-authversion: 2.27.0luigiversion: 3.5.0