Test scikit-image on free-threaded Python using pytest-run-parallel by andfoy · Pull Request #7678 · scikit-image/scikit-image

andfoy · 2025-01-28T21:35:43Z

Description

This PR adds an additional CI job on top of the already-existing 3.13 free threaded one, this new CI pipeline runs the scikit-image testsuite using the pytest-run-parallel plugin, which allows to run individual tests concurrently during a pytest session

Some tests were using the global random number generator, via np.random.seed, which was causing numeric result mismatches when the tests were being run in parallel. Instead, all tests now use np.random.RandomState, which ensures that each thread gets its own independent random generator.

Also, several tests check for warnings, which as 3.13 is not thread-safe, therefore, such tests are marked as thread unsafe and are run in a single thread

Checklist

A descriptive but concise pull request title
Docstrings for all functions
Unit tests
A gallery example in ./doc/examples for new features
Contribution guide is followed

Release note

For maintainers and optionally contributors, please refer to the instructions on how to document this PR for the release notes.

...

stefanv · 2025-01-28T22:13:12Z

Thanks @andfoy! Can you explain a bit of which tests are / aren't thread safe?

andfoy · 2025-01-29T16:23:39Z

@stefanv, the main bulk of "thread-unsafe" tests are the ones that depend on some kind of warning capture logic, given that the warnings module is not thread-safe, those tests are bound to fail under concurrent conditions, see python/cpython#91505.

Other tests use the general NumPy random number generator, as opposed to using of the local generator interfaces, e.g., RandomState or default_rng, those tests were "fixed" by replacing the global instance by a local one.

There was a single instance of a module that used global shared state variables and not local ones:

scikit-image/skimage/segmentation/morphsnakes.py

Line 82 in 8b9fb57

_curvop = threading.local()

Finally, there is a set of C-modules that use the stdlib random generator function calls, which are not thread-safe, those need to be refactored in order to use a local state generator:

scikit-image/skimage/restoration/unwrap_2d_ljmu.c

Line 168 in c458265

srand(seed);

stefanv · 2025-01-30T00:59:05Z

Do you know how to address those last two items? I'm happy to get all this in, but would prefer if we don't merge items requiring later TODOs that we don't know how to handle.

ngoldbaum · 2025-01-30T17:31:51Z

.github/workflows/tests.yml

      # TODO: replace with setup-python when there is support
-      - uses: deadsnakes/action@6c8b9b82fe0b4344f4b98f2775fcc395df45e494 # v3.1.0
+      # TODO: replace with setup-python when there is support
+      - uses: Quansight-Labs/setup-python@b9ab292c751a42bcd2bb465b7fa202ea2c3f5796 # v5.3.1


you might also try with setup-uv

ngoldbaum · 2025-01-30T17:53:53Z

Out of curiosity, how much slower does this make the free-threaded CI run?

stefanv · 2025-01-31T08:41:47Z

I significantly reworked the CI over the past few days, so unfortunately you'll have a few merge conflicts. On the bright side, it should now be easier to follow what's happening, and how to adapt the various scripts.

andfoy · 2025-02-03T15:41:52Z

According to this test-run attempt https://github.com/scikit-image/scikit-image/actions/runs/13117156401, the action astral-sh/setup-uv needs to be registered somewhere in the project settings

lagru · 2025-02-03T17:45:53Z

astral-sh/setup-uv needs to be registered somewhere in the project settings

I added it. Should work now. :)

lagru · 2025-02-03T17:48:01Z

Should remember to remove the deadsnakes one, once this is merged. 😬

.github/workflows/test-linux.yaml

Address failing tests Address failing test Mark test as thread_unsafe Mark test as thred_unsafe Debug via SSH Mark test_rotate as thread_unsafe

Schefflera-Arboricola

test_var in test_texture.py::TestLBP needs a @pytest.mark.thread_unsafe.

here - https://github.com/scikit-image/scikit-image/pull/7678/files#diff-f2e0cb10a01fe0e042f74fcff7da9a52688cf4f935c085f4b44828d751125a5bR311

.github/workflows/test-linux.yaml

lagru

Thank you @andfoy for tackling this. I'm happy to get rid of global state and random number generation. I haven't gotten through everything yet.

A general question, how obvious is it that a test isn't thread-safe in case we forget to mark it such?

lagru · 2025-03-08T17:27:15Z

skimage/_shared/testing.py

    testing.assert_allclose(r5, img5)


+FETCH_LOCK = threading.Lock()


Nitpick: should probably be named _FETCH_LOCK.

lagru · 2025-03-08T17:51:38Z

skimage/color/tests/test_colorconv.py

-    with expected_warnings(["alpha cannot be safely cast to image dtype"]):
-        rgba = gray2rgba(img_u8, alpha)
-        assert_equal(rgba[..., :3], gray2rgb(img_u8))
+    if num_parallel_threads == 1:


Any specific reason for using this approach here instead of marking the whole test as thread_unsafe? The latter seems like the simpler option which also doesn't require defining a fallback fixture for num_parallel_threads.

Given that the test is checking for other conditions rather other than the warning, I thought it was a good idea not losing that coverage? I can skip it, however maybe the warning-specific test could be moved to its own separate test?

lagru · 2025-03-08T17:55:34Z

skimage/data/tests/test_data.py



+@pytest.mark.thread_unsafe
 def test_download_all_with_pooch():


To clarify, I'm not sure why this one isn't thread-safe? I assume our pooch code isn't thread-safe? In that case, we should probably make it so!

The issue is related to the fact of multiple threads trying to write to the same file, however, in practice it would be sound to delegate downloads to a single thread, thus testing such functionality in parallel would not make much sense. A separate question would be if pooch spawns multiple threads to accelerate downloads, but that's a different question.

lagru · 2025-03-08T18:00:10Z

skimage/feature/tests/test_haar.py

    img = np.ones((5, 5), dtype=np.int8)
    img_ii = integral_image(img)
    if isinstance(feature_type, list):
+        feature_type = list(feature_type)


Nitpick: Make feature_type a tuple from the start in parametrize instead?

The extra list is used to take a copy of feature_type, which is being mutated with shuffle afterwards

skimage/graph/tests/test_rag.py

lagru · 2025-03-08T18:03:58Z

skimage/graph/tests/test_spath.py



+@pytest.mark.thread_unsafe
 def test_basic():


Clarification: Same here, why isn't this test thread-safe? I don't see a warning or global state here.

There is an ongoing Cython/CPython issue that is causing sporadic segfaults: cython/cython#6718, this is completely unrelated to the actual scikit-image code.

lagru · 2025-03-08T18:08:34Z

skimage/io/tests/test_imageio.py

+        if hasattr(threading, 'get_native_id'):
+            tmp_dir = tmp_path / str(threading.get_native_id())
+        else:
+            tmp_dir = tmp_path / str(threading.get_ident())
+        tmp_dir.mkdir()
+        file_path = tmp_dir / "roundtrip.png"


I assume tmp_path isn't thread-safe. Could we conditionally override it in conftest.py with a thread-safe version? That way we would avoid future bugs because I would certainly forget that I needed to do this in one year.

Ensuring a different tmp_path per thread is not possible due to a limitation in pytest-run-parallel: Quansight-Labs/pytest-run-parallel#14

skimage/morphology/tests/test_misc.py

lagru · 2025-03-08T18:16:30Z

skimage/restoration/tests/test_unwrap.py

+    reason='This test is failing under free-threaded Python',
+)
 @testing.parametrize("ndim, axis", dim_axis)
 def test_wrap_around(ndim, axis):


Why is this necessary now? This test seems to run fine on main on 313t?

It fails with a numerical error, even when marked as single-threaded:

PYTHON_GIL=0 pytest -x -v --parallel-threads=10 skimage/restoration/tests/test_unwrap.py ============================================================= test session starts ============================================================== platform linux -- Python 3.13.2, pytest-8.3.3, pluggy-1.5.0 -- /home/andfoy/anaconda3/envs/nogil/bin/python3.13 cachedir: .pytest_cache hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/home/andfoy/Documentos/Quansight/nogil/scikit-image/.hypothesis/examples')) rootdir: /home/andfoy/Documentos/Quansight/nogil/scikit-image configfile: pyproject.toml plugins: localserver-0.9.0.post0, hypothesis-6.118.8, run-parallel-0.2.1.dev0, order-1.3.0 collected 16 items skimage/restoration/tests/test_unwrap.py::test_unwrap_1d PASSED [ 6%] skimage/restoration/tests/test_unwrap.py::test_unwrap_2d[False] PASSED [ 12%] skimage/restoration/tests/test_unwrap.py::test_unwrap_2d[True] PASSED [ 18%] skimage/restoration/tests/test_unwrap.py::test_unwrap_3d[False] PASSED [ 25%] skimage/restoration/tests/test_unwrap.py::test_unwrap_3d[True] PASSED [ 31%] skimage/restoration/tests/test_unwrap.py::test_wrap_around[2-0] FAILED [ 37%] =================================================================== FAILURES =================================================================== ____________________________________________________________ test_wrap_around[2-0] _____________________________________________________________ ndim = 2, axis = 0 @pytest.mark.thread_unsafe @skipif( sys.version_info[:2] == (3, 4), reason="Doesn't work with python 3.4. See issue #3079", ) # @skipif( # bool(sysconfig.get_config_var("Py_GIL_DISABLED")), # reason='This test is failing under free-threaded Python', # ) @testing.parametrize("ndim, axis", dim_axis) def test_wrap_around(ndim, axis): > check_wrap_around(ndim, axis) skimage/restoration/tests/test_unwrap.py:145: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ndim = 2, axis = 0 def check_wrap_around(ndim, axis): # create a ramp, but with the last pixel along axis equalling the first elements = 100 ramp = np.linspace(0, 12 * np.pi, elements) ramp[-1] = ramp[0] image = ramp.reshape(tuple([elements if n == axis else 1 for n in range(ndim)])) image_wrapped = np.angle(np.exp(1j * image)) index_first = tuple([0] * ndim) index_last = tuple([-1 if n == axis else 0 for n in range(ndim)]) # unwrap the image without wrap around # We do not want warnings about length 1 dimensions with expected_warnings([r'Image has a length 1 dimension|\A\Z']): image_unwrap_no_wrap_around = unwrap_phase(image_wrapped, rng=0) print( 'endpoints without wrap_around:', image_unwrap_no_wrap_around[index_first], image_unwrap_no_wrap_around[index_last], ) # without wrap around, the endpoints of the image should differ assert_( abs( image_unwrap_no_wrap_around[index_first] - image_unwrap_no_wrap_around[index_last] ) > np.pi ) # unwrap the image with wrap around wrap_around = [n == axis for n in range(ndim)] # We do not want warnings about length 1 dimensions with expected_warnings([r'Image has a length 1 dimension.|\A\Z']): image_unwrap_wrap_around = unwrap_phase(image_wrapped, wrap_around, rng=0) print( 'endpoints with wrap_around:', image_unwrap_wrap_around[index_first], image_unwrap_wrap_around[index_last], ) # with wrap around, the endpoints of the image should be equal > assert_almost_equal( image_unwrap_wrap_around[index_first], image_unwrap_wrap_around[index_last] ) E AssertionError: E Arrays are not almost equal to 7 decimals E ACTUAL: 0.0 E DESIRED: 37.69911184307752 skimage/restoration/tests/test_unwrap.py:126: AssertionError ------------------------------------------------------------- Captured stdout call ------------------------------------------------------------- endpoints without wrap_around: -6.283185307179586 31.41592653589793 endpoints with wrap_around: 0.0 37.69911184307752 =========================================================== short test summary info ============================================================ FAILED skimage/restoration/tests/test_unwrap.py::test_wrap_around[2-0] - AssertionError: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ========================================================= 1 failed, 5 passed in 0.69s ==========================================================

andfoy · 2025-03-10T16:20:28Z

@lagru thanks for the review, let me check again the skip marks I left, maybe they are now fixed with the latest Cython nightlies

ngoldbaum · 2026-02-17T15:57:13Z

Hi all, I'm going to try to resurrect this.

@andfoy hope you don't mind.

Hopefully more to come from me soon.

Towards fixing #7464 Currently, running the scikit-image tests fails on the free-threaded build because the GIL is enabled at runtime: ``` _______________________________________________ ERROR collecting build-install/usr/lib/python3.14t/site-packages/skimage/feature/censure.py ________________________________________________ build-install/usr/lib/python3.14t/site-packages/skimage/feature/censure.py:6: in <module> from ..morphology import octagon, star build-install/usr/lib/python3.14t/site-packages/skimage/morphology/__init__.py:28: in <module> from ._skeletonize import medial_axis, skeletonize, thin build-install/usr/lib/python3.14t/site-packages/skimage/morphology/_skeletonize.py:10: in <module> from ._skeletonize_lee_cy import _compute_thin_image E RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'skimage.morphology._skeletonize_lee_cy', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0. ``` This is happening because the `cython_args` aren't being passed to the C++ codegen function in the project's meson configuration. I changed the name of the variable to `cython_codegen_args` to make it a little clearer it has nothing to do with `cython_c_args` and `cython_cpp_args`, which are passed to e.g. clang and not Cython. I also made `cython_gen_cpp` use the same arguments as `cython_gen`. `c_undefined_ok` is unused, so I deleted it. Finally, I added a regression test for this in the project's `conftest.py`, following a similar check I added to numpy's `conftest.py` a while back. I also added a 3.14t testing job and deleted the 3.13t job and special testing logic. All of that is unnecessary now. I will follow up after this to make the 3.14t testing job use pytest-run-parallel, superseding #7678.

ngoldbaum · 2026-02-25T23:38:48Z

I opened #8065, this should probably be closed in favor of that PR.

Towards fixing scikit-image#7464 Currently, running the scikit-image tests fails on the free-threaded build because the GIL is enabled at runtime: ``` _______________________________________________ ERROR collecting build-install/usr/lib/python3.14t/site-packages/skimage/feature/censure.py ________________________________________________ build-install/usr/lib/python3.14t/site-packages/skimage/feature/censure.py:6: in <module> from ..morphology import octagon, star build-install/usr/lib/python3.14t/site-packages/skimage/morphology/__init__.py:28: in <module> from ._skeletonize import medial_axis, skeletonize, thin build-install/usr/lib/python3.14t/site-packages/skimage/morphology/_skeletonize.py:10: in <module> from ._skeletonize_lee_cy import _compute_thin_image E RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'skimage.morphology._skeletonize_lee_cy', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0. ``` This is happening because the `cython_args` aren't being passed to the C++ codegen function in the project's meson configuration. I changed the name of the variable to `cython_codegen_args` to make it a little clearer it has nothing to do with `cython_c_args` and `cython_cpp_args`, which are passed to e.g. clang and not Cython. I also made `cython_gen_cpp` use the same arguments as `cython_gen`. `c_undefined_ok` is unused, so I deleted it. Finally, I added a regression test for this in the project's `conftest.py`, following a similar check I added to numpy's `conftest.py` a while back. I also added a 3.14t testing job and deleted the 3.13t job and special testing logic. All of that is unnecessary now. I will follow up after this to make the 3.14t testing job use pytest-run-parallel, superseding scikit-image#7678.

stefanv · 2026-03-06T05:16:14Z

Work is continued in #8065

stefanv added the 🤖 type: Infrastructure CI, packaging, tools and automation label Jan 30, 2025

ngoldbaum reviewed Jan 30, 2025

View reviewed changes

andfoy force-pushed the use_pytest_run_parallel branch from 361487d to 2c96b36 Compare February 3, 2025 15:39

lagru reviewed Feb 3, 2025

View reviewed changes

.github/workflows/test-linux.yaml Outdated Show resolved Hide resolved

andfoy force-pushed the use_pytest_run_parallel branch from b434448 to 24114e6 Compare February 4, 2025 19:15

andfoy force-pushed the use_pytest_run_parallel branch 2 times, most recently from c583fd6 to 67c9828 Compare February 11, 2025 23:30

andfoy added 15 commits March 5, 2025 12:20

Test scikit-image on free-threaded Python using pytest-run-parallel

80617b1

Remove unused script

40b85bc

Debug via SSH

4f5df75

Install uv on a separate directory

b30691c

Do not install when pulling uv by default

6ab5c57

Create venv in a separate folder

651213c

Move parallel-threads flag to env variable

f572949

Ensure that uv environment is picked up

6b9aaf5

Activate environment everyhwere

df09d10

Reverse comparison order

3f961ba

Define stub fixtures

c6343dd

Address failing tests Address failing test Mark test as thread_unsafe Mark test as thred_unsafe Debug via SSH Mark test_rotate as thread_unsafe

Fix CI execution

999e808

Remove SSH debugging

7a14dbe

Try to skip test_basic

7999d48

Install nightly numpy

bcf0233

andfoy added 2 commits March 5, 2025 12:20

Do not use threading.get_native_id on Pyodide

457f44d

Make sure nightly dependenccies are installed

935821a

andfoy force-pushed the use_pytest_run_parallel branch from cbdfb18 to 935821a Compare March 5, 2025 17:20

Remove duplicate import

36bb6f7

Schefflera-Arboricola reviewed Mar 6, 2025

View reviewed changes

.github/workflows/test-linux.yaml Outdated Show resolved Hide resolved

lagru self-requested a review March 6, 2025 09:23

andfoy added 2 commits March 6, 2025 11:51

Mark test_var as thread unsafe

f9d1988

Use actions/setup-python

ed56bbb

lagru mentioned this pull request Mar 8, 2025

Remove obsolete doctest_skip_parser #7744

Closed

lagru reviewed Mar 8, 2025

View reviewed changes

Address review comments

c17aecb

This comment was marked as outdated.

Sign in to view

github-actions bot added the 😴 Dormant No recent activity label Feb 13, 2026

lagru removed the 😴 Dormant No recent activity label Feb 13, 2026

ngoldbaum mentioned this pull request Feb 18, 2026

Fix GIL being re-enabled by C++ extensions #8059

Merged

ngoldbaum mentioned this pull request Feb 25, 2026

Updates test suite to be used with pytest-run-parallel #8065

Open

stefanv closed this Mar 6, 2026

		testing.assert_allclose(r5, img5)


		FETCH_LOCK = threading.Lock()



		@pytest.mark.thread_unsafe
		def test_download_all_with_pooch():

Uh oh!

Conversation

andfoy commented Jan 28, 2025

Description

Checklist

Release note

Uh oh!

stefanv commented Jan 28, 2025

Uh oh!

andfoy commented Jan 29, 2025

Uh oh!

stefanv commented Jan 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ngoldbaum commented Jan 30, 2025

Uh oh!

stefanv commented Jan 31, 2025

Uh oh!

andfoy commented Feb 3, 2025

Uh oh!

lagru commented Feb 3, 2025

Uh oh!

lagru commented Feb 3, 2025

Uh oh!

Uh oh!

Schefflera-Arboricola left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lagru left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andfoy Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andfoy Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andfoy commented Mar 10, 2025

Uh oh!

This comment was marked as outdated.

ngoldbaum commented Feb 17, 2026

Uh oh!

ngoldbaum commented Feb 25, 2026

Uh oh!

stefanv commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

andfoy Mar 10, 2025 •

edited

Loading

andfoy Mar 10, 2025 •

edited

Loading

stefanv commented Mar 6, 2026 •

edited

Loading