-
Notifications
You must be signed in to change notification settings - Fork 966
Popen stall in ThreadPoolEecutor under docker #1865
Copy link
Copy link
Open
Description
- gevent version:
21.12.0, wheels from pypi - Python version: python 3.10.2
- Operating System: docker image 3.10.2-slim-bullseye
Description:
Hi, the problem appears suddenly today. Build stage has stall on manage.py compilemessages. strace shows only a infinite futex waiting.
Usually it's always worked ok. I don't know why exactly a behaviour is changed. But I found why it is stall.
Simple reproducible example ((not) worked only under docker)
from gevent import monkey
import gevent
if __name__ == '__main__':
monkey.patch_thread();
monkey.patch_subprocess();
import subprocess
from concurrent.futures import ThreadPoolExecutor
def func():
# gevent.sleep(0) # uncomment to fail
return subprocess.run(["ls", "-l", "/dev/null"], capture_output=True)
with ThreadPoolExecutor(max_workers=5) as executor:
fut = executor.submit(func)
print(fut.result())This code will stall, but ctr+c give traceback:
^CException ignored in: <built-in method acquire of _thread.lock object at 0x7f1af2ab8800>
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/gevent/os.py", line 431, in fork_and_watch
pid = fork()
KeyboardInterrupt:
Traceback (most recent call last):
File "/var/www/cabinet/test.py", line 16, in <module>
fut = executor.submit(func)
File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 162, in submit
with self._shutdown_lock, _global_shutdown_lock:
RuntimeError: release unlocked lockIf uncomment sleep in func all works fine. So problem occurs due to the code is not concurrent really.
ThreadPoolExecutor acquires a several locks in submit. CPython calls a some handlers before fork here. Particularly concurrent.futures.thread sets one. As the code are not runned concurrently and _global_shutdown_lock is the simple Lock, not RLock, so it will be deadlocked.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels