I have a threading Web spider using httpx that is triggering a race condition in anyio/_core/_eventloop.py's get_asynclib() function. This race can be triggered if you have a lot of threads trying to initialize their own asyncio event loop and initializing httpx all at once. It happens intermittently -- sometimes we don't trigger the race, sometimes we do. When we do, the threads catch get_asynclib() mid-initialization and certain attributes that should be present are not available. I am using anyio 3.5.0, httpx 0.22.0 and python3.7.
httpx-exception.txt
An ugly but functioning patch which I can confirm resolves the issue is below:
get_asynclib_thread_fix.patch.txt
In my test patch, I made the lock cover essentially the entire get_asynclib() call -- it could potentially be made tighter, with questionable benefit -- not sure.
I have a threading Web spider using httpx that is triggering a race condition in anyio/_core/_eventloop.py's get_asynclib() function. This race can be triggered if you have a lot of threads trying to initialize their own asyncio event loop and initializing httpx all at once. It happens intermittently -- sometimes we don't trigger the race, sometimes we do. When we do, the threads catch get_asynclib() mid-initialization and certain attributes that should be present are not available. I am using anyio 3.5.0, httpx 0.22.0 and python3.7.
httpx-exception.txt
An ugly but functioning patch which I can confirm resolves the issue is below:
get_asynclib_thread_fix.patch.txt
In my test patch, I made the lock cover essentially the entire get_asynclib() call -- it could potentially be made tighter, with questionable benefit -- not sure.