Skip to content

CI: Add Linux workflow to test on free-threaded Python builds#20822

Merged
rgommers merged 6 commits intoscipy:mainfrom
andfoy:add_free_threaded_ci
May 30, 2024
Merged

CI: Add Linux workflow to test on free-threaded Python builds#20822
rgommers merged 6 commits intoscipy:mainfrom
andfoy:add_free_threaded_ci

Conversation

@andfoy
Copy link
Copy Markdown
Contributor

@andfoy andfoy commented May 28, 2024

Reference issue

See #20669

What does this implement/fix?

This PR enables testing SciPy against newer Python free-threaded builds

Additional information

Equivalent NumPy PRs: numpy/numpy#26512, numpy/numpy#26463

@github-actions github-actions bot added CI Items related to the CI tools such as CircleCI, GitHub Actions or Azure enhancement A new feature or improvement labels May 28, 2024
@andfoy andfoy force-pushed the add_free_threaded_ci branch from 7d95043 to fcf5ee1 Compare May 28, 2024 16:17
@lucascolley lucascolley added the free-threading Items related to supporting free-threaded (a.k.a. "no-GIL") builds of CPython label May 28, 2024
@andfoy andfoy force-pushed the add_free_threaded_ci branch from b8f20ab to a1f0674 Compare May 28, 2024 18:39
@lucascolley lucascolley changed the title ENH: Add Linux workflow to test on free-threaded Python builds CI: Add Linux workflow to test on free-threaded Python builds May 28, 2024
@lucascolley lucascolley removed the enhancement A new feature or improvement label May 28, 2024
@andfoy andfoy force-pushed the add_free_threaded_ci branch from 4b5daa2 to 796fef1 Compare May 28, 2024 20:29
@rgommers
Copy link
Copy Markdown
Member

This looks close to right already, thanks @andfoy. Note that if you need to iterate on a CI job, it's usually best to do that on your own fork, to avoid running a ton of other jobs over and over. If you include a debug commit like rgommers/numpy@9e45f73, you'll trigger just this new job on your fork.

@andfoy andfoy force-pushed the add_free_threaded_ci branch from 796fef1 to 6a942cd Compare May 28, 2024 20:38
@rgommers
Copy link
Copy Markdown
Member

you'll trigger just this new job on your fork.

(on a new branch from which you didn't already make a PR)

@andfoy
Copy link
Copy Markdown
Contributor Author

andfoy commented May 28, 2024

(on a new branch from which you didn't already make a PR)

That makes sense, I'll do that then!

@andfoy andfoy force-pushed the add_free_threaded_ci branch from 11352a6 to 6a942cd Compare May 28, 2024 20:54
env:
PYTHON_GIL: 0
# TODO: For some reason the Meson installation path points to
# python3/site-packages as opposed to python3.13/site-packages,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds like a CPython issue to me (in sysconfig), since path handling shouldn't depend on --disable-gil yes or no. And all we're doing in dev.py I think is sysconfig.get_path('platlib').

Okay to ignore here for now.

@andfoy
Copy link
Copy Markdown
Contributor Author

andfoy commented May 29, 2024

Regarding the test failure at scipy.misc.tests.test_doccer::test_decorator, multiline function docstrings seem to differ between Python 3.12 and 3.13:

def test_decorator():
    with suppress_warnings() as sup:
        ...
        @decorator
        def func():
            """ Docstring
            %(strtest3)s
            """
# Python 3.12
(Pdb) f.__doc__
' Docstring\n            %(strtest3)s\n            '

# Python 3.13
(Pdb) f.__doc__
'Docstring\n%(strtest3)s\n'

I don't know if this is an expected change or a regression in CPython, I took a look at the Changelog, but there is no mention to multiline strings besides python/cpython#112943. However, that issue doesn't seems to be the culprit here, since those changes were also backported into 3.12.2 (the output trace comes from a 3.12.3 instance)

@rgommers
Copy link
Copy Markdown
Member

From https://docs.python.org/3.13/whatsnew/3.13.html#other-language-changes: "Compiler now strip indents from docstrings." I bet that that's the culprit

Please feel to skip that test here and open a new issue about it, marked for 0.14.0

@ev-br
Copy link
Copy Markdown
Member

ev-br commented May 29, 2024

#19572 x-ref

@andfoy andfoy marked this pull request as ready for review May 29, 2024 23:49
@andfoy andfoy requested review from andyfaff and larsoner as code owners May 29, 2024 23:49
@rgommers rgommers added this to the 1.15.0 milestone May 30, 2024
Copy link
Copy Markdown
Member

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything seems to be in order here now. It's installing or building the cp313t flavors for all build and runtime dependencies that use the Python C API. For SciPy:

Created wheel for scipy: filename=scipy-1.15.0.dev0-cp313-cp313t-linux_x86_64.whl size=27954025

Let's give this a go, thanks @andfoy!

@andfoy andfoy deleted the add_free_threaded_ci branch May 30, 2024 16:47
@lesteve
Copy link
Copy Markdown
Contributor

lesteve commented May 31, 2024

Great to see the progress on this! Does that mean that there will be a development free-threaded wheel in https://anaconda.org/scientific-python-nightly-wheels/scipy/files at one point?

Asking mostly and out of curiosity and also with the goal one day of setting up a similar build for scikit-learn.

@rgommers
Copy link
Copy Markdown
Member

Yes, we should start building nightly wheels, at least for Linux x86-64.

@andfoy could you have a go at doing that as the next step here? I think it should look very similar to numpy/numpy#26512; if you start from copying SciPy's wheels.yml to free-threaded-wheels.yml and adapt from there, hopefully it'll work without new surprises.



@pytest.mark.skipif(DOCSTRINGS_STRIPPED, reason="docstrings stripped")
@pytest.mark.skipif(sys.version_info >= (3, 13), reason='it fails on Py3.13')
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we reference gh-19572 in the skip message?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Items related to the CI tools such as CircleCI, GitHub Actions or Azure free-threading Items related to supporting free-threaded (a.k.a. "no-GIL") builds of CPython

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants