Skip to content

Popen stall in ThreadPoolEecutor under docker #1865

@horpto

Description

@horpto
  • gevent version:
    21.12.0, wheels from pypi
  • Python version: python 3.10.2
  • Operating System: docker image 3.10.2-slim-bullseye

Description:

Hi, the problem appears suddenly today. Build stage has stall on manage.py compilemessages. strace shows only a infinite futex waiting.
Usually it's always worked ok. I don't know why exactly a behaviour is changed. But I found why it is stall.

Simple reproducible example ((not) worked only under docker)

from gevent import monkey
import gevent

if __name__ == '__main__':
    monkey.patch_thread();
    monkey.patch_subprocess();

    import subprocess
    from concurrent.futures import ThreadPoolExecutor

    def func():
        # gevent.sleep(0) # uncomment to fail
        return subprocess.run(["ls", "-l", "/dev/null"], capture_output=True)

    with ThreadPoolExecutor(max_workers=5) as executor:
        fut = executor.submit(func)
        print(fut.result())

This code will stall, but ctr+c give traceback:

^CException ignored in: <built-in method acquire of _thread.lock object at 0x7f1af2ab8800>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/gevent/os.py", line 431, in fork_and_watch
    pid = fork()
KeyboardInterrupt:
Traceback (most recent call last):
  File "/var/www/cabinet/test.py", line 16, in <module>
    fut = executor.submit(func)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 162, in submit
    with self._shutdown_lock, _global_shutdown_lock:
RuntimeError: release unlocked lock

If uncomment sleep in func all works fine. So problem occurs due to the code is not concurrent really.
ThreadPoolExecutor acquires a several locks in submit. CPython calls a some handlers before fork here. Particularly concurrent.futures.thread sets one. As the code are not runned concurrently and _global_shutdown_lock is the simple Lock, not RLock, so it will be deadlocked.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions