🐛 Bug

Our use of pybind11::gil_scoped_release can cause crashes when used with daemon threads (from the threading package) and with C++ created threads that acquire/release the GIL. Note that we previously used AutoNoGIL, but that has the same problem.

This manifests as a crash on exit with a message like:

terminate called without an active exception
Aborted

The problem occurs when a thread tries to acquire the GIL during Python interpreter finalization. The interpreter calls pthread_exit(0) to terminate the thread. Note that the call is from the destructor of gil_scoped_release. The glibc pthread_exit implementation uses unwind; calling unwind from a destructor has similar problems to throwing C++ exceptions from destructors. This triggers the call to std::terminate and the abort.

Here is some code that reproduces the issue. I think #22924 is essentially the same issue. I've also seen this happen in research code that mixes C++ threads with Python.

import torch
import threading

def thread_loop():
    a = torch.randn(100)
    while True:
        a + a

t = threading.Thread(target=thread_loop, daemon=True)
t.start()

cc @ezyang @gchanan @zou3519