Skip to content

import llvmlite race condition with multiple interpreters on Windows #808

@crusaderky

Description

@crusaderky

llvmlite 0.37.0 py37habb0c8c_1 conda-forge/win-64

I found a race condition on Windows when multiple Python interpreters try to import llvmlite in parallel.

The dask test suite just crashed on github actions windows:
https://github.com/dask/dask/runs/4761444273?check_suite_focus=true:

ctx = mp.get_context("spawn")
with ProcessPoolExecutor(8, ctx) as pool:
     ... = set(pool.map(...))

Output:

E           concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
[...]

    import numba

  File "C:\Miniconda3\envs\test-environment\lib\site-packages\numba\__init__.py", line 19, in <module>

    from numba.core import config

  File "C:\Miniconda3\envs\test-environment\lib\site-packages\numba\core\config.py", line 16, in <module>

    import llvmlite.binding as ll

  File "C:\Miniconda3\envs\test-environment\lib\site-packages\llvmlite\binding\__init__.py", line 4, in <module>

    from .dylib import *

  File "C:\Miniconda3\envs\test-environment\lib\site-packages\llvmlite\binding\dylib.py", line 3, in <module>

    from llvmlite.binding import ffi

  File "C:\Miniconda3\envs\test-environment\lib\site-packages\llvmlite\binding\ffi.py", line 191, in <module>

    raise OSError("Could not load shared object file: {}".format(_lib_name))

OSError: Could not load shared object file: llvmlite.dll

Metadata

Metadata

Assignees

No one assigned

    Labels

    needtriageNeeds to be triaged further

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions