I see that SciPy is being reverted from v1.13.0 to v1.12.0 in #4712 – I'm not sure what that means, so I'll be happy to wait until further progress or discussion on that PR (my understanding is that whichever packages still pass the tests with newer versions can stay and will not be reverted?)

I'm not sure what that means, so I'll be happy to wait until further progress or discussion on that PR (my understanding is that whichever packages still pass the tests with newer versions can stay and will not be reverted?)

We do one big PR for all the easy packages and then have to handle the ones that cause trouble one at a time as we have time. Please go ahead with working on this PR. If there is a conflict with #4712 for some reason I can help resolve it.

@agriyakhetarpal just curious, what is the status of this? Since you mentioned "as as I look to fix some of the older patches locally" I am guessing that you are still working on updating the patches?

For some weird reasons, it looks like the full CI was not run (random guess maybe because this is a Draft PR, although seems unlikely), can you try pushing an empty commit to trigger the CI again? I was confused and thought this PR was green untill I unfolded the "All checks" and found out that there are only readthedocs and pre-commit ...

@agriyakhetarpal just curious, what is the status of this? Since you mentioned "as as I look to fix some of the older patches locally" I am guessing that you are still working on updating the patches?

Hi, @lesteve, thank you for the ping. I have just returned to this PR late last week, and I was going to post an update today about how this is going, so, yes, doing this and updating the patches here is well on my radar for this week. I'll be sure to request help wherever I don't understand something fully!

For some weird reasons, it looks like the full CI was not run (random guess maybe because this is a Draft PR, although seems unlikely), can you try pushing an empty commit to trigger the CI again? I was confused and thought this PR was green untill I unfolded the "All checks" and found out that there are only readthedocs and pre-commit ...

Ah, I guess the full CI was not run because I added the [skip ci] label in the commit message in order to save resources for others. :) I shall trigger CI in subsequent commits or when I rebase.

Ah makes sense I missed the [skip ci] thing and thought "OK the CI seems green, so this should be good to merge"

It's an unfortunate thing about Skip Ci. Maybe we should add a "ci ran" check. Of course we can merge changes that skip CI, but the Ci is flaky enough that we merge failing CI a lot.

I rebased on top of the latest changes from the main branch and fixed up the patches (all that was to fix, most likely) – I will hope that CI passes (🤞), and I shall update the to-do list in the PR description accordingly.

I assume that the failures under ci/circleci: test-core-safari are unrelated, I will wait until the ci/circleci: build-libraries job passes (or runs till its completion).

Oops, it looks like scipy is under the "The following packages are disabled" list: https://app.circleci.com/pipelines/github/pyodide/pyodide/7181/workflows/88fa2666-a007-427a-826d-cd207b3a4485/jobs/100486?invite=true#step-104-9703_80, so it's not being built. I'll look if there is a special commit message to build SciPy or the SciPy stack in general, based on https://pyodide.org/en/stable/development/building-from-sources.html#partial-builds.

Edit: never mind, I saw that some new builds popped up on CircleCI.

Build logs: https://app.circleci.com/pipelines/github/pyodide/pyodide/7182/workflows/36dda168-fe41-4dcf-859d-278178308521/jobs/100512/parallel-runs/0/steps/0-104

Note to self: most of the errors look to be coming from Fortran code because the f2c sub-command failed; I have to look at syntax errors in a few files, remove warnings about a few compiler flags (probably replace -Wno-maybe-unintialized with -Wno-unintialized and look at how Emscripten sets runtime search paths because -rpath was reported to be unknown, etc.), and correctly identify the file leading to the cause of the error (I am currently browsing through the logs on a mobile device and that does not help) – it's most likely inside FITPACK.

It's an unfortunate thing about Skip Ci. Maybe we should add a "ci ran" check. Of course we can merge changes that skip CI, but the Ci is flaky enough that we merge failing CI a lot.

Btw, @hoodmane, if you could elaborate on this slightly, I would be happy to add this "ci ran" check in some shape or form (if the change would not be non-trivial, that is, of course).

The errors were in scipy/interpolate/fitpack/fpchec.f (sub-module 1076 out of 1464) and were coming from lines like these:

if(t(i).le.t(i-1))then; ier=30; go to 80; endif
...
if(x(1).lt.t(k1) .or. x(m).gt.t(nk2))then; ier=40; go to 80;
endif
...
if(i.ge.m)then; ier=50; go to 80; endif

and so on. I checked the file versus SciPy v1.12 and scipy/scipy@a6284f6.patch is where this comes from, so I switched the if-else constructs to multiple lines in 6917e33, and that makes them work for some reason.

and now SciPy v1.13.0 builds successfully (🎉): workflow logs. I'll wait for the build to finish, following which I can add a CHANGELOG entry when this will be merged (please let me know if I should skip CI for that one) and complete the rest of the checklists.

Never mind, there are 4 failures out of 5 tests on SciPy: ImportError: dynamic module does not define module export function (PyInit_cython_lapack)... I am diving further into this.

Edit: and similarly for PyInit__sparsetools

I don't entirely know how to debug this and I am afraid that my attempts to look further did not lead to much insight. As far as I understand, not being experienced with Fortran codes, there could be a missing PyInit_* statement in the scipy.linalg side of things, or something was missed during the compilation of cython_lapack inside scipy/linalg/meson.build – I notice that 1.12.0 used link_with: fwrappers and 1.13.0 uses link_with: [g77_abi_wrappers], and I tried to follow the discussions around that and I think the modified patch 5/8 should fix that in scipy/_build_utils/src/wrap_g77_abi.c. I inspected the auto-generated cython_lapack.c file diffs between v1.12.0 and v1.13.0 and I didn't catch anything relevant, except for the F_FUNC macros having been replaced by BLAS_FUNC everywhere, likely a result of the ABI wrappers.

Happy to receive suggestions on how I can debug this further and work towards a solution, @ryanking13 and @hoodmane – your comments will be highly appreciated!

I'll take a look when I have a minute. At pycon right now.

@rgommers, since we don't have an out-of-tree build for SciPy yet, would you suggest going upstream or skipping it here? I'm of the impression that we could fix things upstream with more changes than fixing just one failing test: open to suggestions.

Every patch that makes sense to upstream (which should include most/all test skips) is welcome to be upstreamed for SciPy, so after adding the patch here to make this PR green it makes sense to me up upstream it so that when SciPy 1.15.0 comes around the patch here can be dropped.

It doesn't really matter that SciPy doesn't yet have an out-of-tree build, we carry lots of fixes for things like niche CPU architectures that Debian supports or for AIX, none of which we test in CI.

All tests passing after relevant ones were skipped or removed: https://github.com/pyodide/pyodide/actions/runs/10320686508/job/28573393091

The CircleCI test failures aren't related, I think, because they show that they timed out. I don't have permissions to re-run the CircleCI runners. Could someone trigger a re-run for them?

Edit: I merged main since the branch was out of date and triggered a run.

The CircleCI test failures aren't related, I think, because they show that they timed out. I don't have permissions to re-run the CircleCI runners. Could someone trigger a re-run for them?

Hmm, that's strange, CircleCI roles and permission settings are inherited from GitHub, so you should be able to run it. I sent you an circle CI invite link just in case.

I see that the test_SVDS_PROPACK test that we skip for GHA test-scipy job is running on CircleCI, which causes the function signature mismatch. It causes other tests to time out.

(maybe we merged too early)