Describe the issue:
When running the test suite under pytest-run-parallel and ThreadSanitizer, many tests cause data races. The affected tests (~80 test) are listed in this spreadsheet. Below are some tests I thought were notable.
Tests cause a crash, with no output (some of these overlap with ASan errors I found previously, and likely have to do with the tests crashing due to memory issues):
Segmentation fault after data race
RandomState data race (may be a linux-only bug)
dec_addstatus data races
Caused test failure
Steps to reproduce
- Use Python
3.14t with ThreadSanitizer enabled.
- Steps on how to do this can be found here.
- Currently using these TSAN options
export TSAN_OPTIONS=suppressions=/numpy_suppressions.txt:allocator_may_return_null=1:halt_on_error=0
- Using the suppression file from here
- Build NumPy using clang, ThreadSanitizer and without openBLAS
spin build --clean -- -Db_sanitize=thread -Dbuildtype=debugoptimized -Dallow-noblas=true
- Run test suite under
pytest-run-parallel
Error message:
I have the traceback files for all TSan warnings I ran into. Don't want to post them all here and make a mess, but I can provide any of them if needed.
Python and NumPy Versions:
NumPy: 2.4.0.dev0+git20251003.da5c632
Python: 3.14.0rc3+ free-threading build (heads/3.14:460ad1aa1ba, Sep 22 2025, 23:51:20) [Clang 18.1.3 (1ubuntu1)]
Runtime Environment:
[{'numpy_version': '2.4.0.dev0+git20251003.da5c632',
'python': '3.14.0rc3+ free-threading build (heads/3.14:460ad1aa1ba, Sep 22 '
'2025, 23:51:20) [Clang 18.1.3 (1ubuntu1)]',
'uname': uname_result(system='Linux', node='SPACE', release='6.6.87.2-microsoft-standard-WSL2', version='#1 SMP PREEMPT_DYNAMIC Thu Jun 5 18:30:46 UTC 2025', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'architecture': 'Haswell',
'filepath': '/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so',
'internal_api': 'openblas',
'num_threads': 24,
'prefix': 'libopenblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.26'}]
Context for the issue:
No response
Describe the issue:
When running the test suite under pytest-run-parallel and ThreadSanitizer, many tests cause data races. The affected tests (~80 test) are listed in this spreadsheet. Below are some tests I thought were notable.
Tests cause a crash, with no output (some of these overlap with ASan errors I found previously, and likely have to do with the tests crashing due to memory issues):
Segmentation fault after data race
RandomState data race (may be a linux-only bug)
dec_addstatus data races
Caused test failure
Steps to reproduce
3.14twith ThreadSanitizer enabled.export TSAN_OPTIONS=suppressions=/numpy_suppressions.txt:allocator_may_return_null=1:halt_on_error=0spin build --clean -- -Db_sanitize=thread -Dbuildtype=debugoptimized -Dallow-noblas=truepytest-run-parallelspin test -p auto -svError message:
I have the traceback files for all TSan warnings I ran into. Don't want to post them all here and make a mess, but I can provide any of them if needed.
Python and NumPy Versions:
NumPy: 2.4.0.dev0+git20251003.da5c632
Python: 3.14.0rc3+ free-threading build (heads/3.14:460ad1aa1ba, Sep 22 2025, 23:51:20) [Clang 18.1.3 (1ubuntu1)]
Runtime Environment:
[{'numpy_version': '2.4.0.dev0+git20251003.da5c632',
'python': '3.14.0rc3+ free-threading build (heads/3.14:460ad1aa1ba, Sep 22 '
'2025, 23:51:20) [Clang 18.1.3 (1ubuntu1)]',
'uname': uname_result(system='Linux', node='SPACE', release='6.6.87.2-microsoft-standard-WSL2', version='#1 SMP PREEMPT_DYNAMIC Thu Jun 5 18:30:46 UTC 2025', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'architecture': 'Haswell',
'filepath': '/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so',
'internal_api': 'openblas',
'num_threads': 24,
'prefix': 'libopenblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.26'}]
Context for the issue:
No response