Skip to content

ENH: update SuperLU to version 6.0.1#19284

Merged
tylerjereddy merged 7 commits intoscipy:mainfrom
rgommers:update-superlu
Oct 11, 2023
Merged

ENH: update SuperLU to version 6.0.1#19284
tylerjereddy merged 7 commits intoscipy:mainfrom
rgommers:update-superlu

Conversation

@rgommers
Copy link
Copy Markdown
Member

@rgommers rgommers commented Sep 23, 2023 •
edited
Loading

This was quite a pain to untangle. This updates to the latest commit in the master branch of https://github.com/xiaoyeli/superlu, which has a few fixes on top of the 6.0.1 tag. Some of our patches have been upstreamed, others are still to be upstreamed - this is easier now that SuperLU is actively maintained on GitHub.

Closes gh-19092

Potential follow-ups:

  • upstreaming the remaining patches
  • enable 64-bit build of SuperLU (needs to go with an ILP64 BLAS, so is a little tricky)
  • convert to using a git submodule perhaps (after dropping the setup.py build)

EDIT: for reference, here are the previous SuperLU update PRs: gh-7554, gh-12243

@rgommers
MAINT: sparse/dsolve: drop patch for stdint.h with old MSVC versions
88a5af8 
We enforce a minimum version of MSVC, and that version does have the
stdint.h header.

This reverts commit c93a509
@rgommers
ENH: sparse: upgrade SuperLU to 6.0.1
f4bbb25 
Note that this includes a few patches on top of 6.0.1.
The exact upstream commit from https://github.com/xiaoyeli/superlu/
that this upgrades to is 2c8ef7da153e87745f04f968eb2a995d735e64ec.
@rgommers
MAINT: update SuperLU patch files and reapply patches
85a1c28 
The complex changes have been upstreamed and hence we drop the patch
file.

The patches didn't cleanly apply, hence we dropped the non-essential
ones. Those correspond to the "fix uninitialized warnings" from
PR 14442.
@rgommers
BLD: SuperLU: silence -Wunused-result and -Wmaybe-uninitialized warnings
e1431ee 
This replaces the fixes from scipygh-14442 and scipygh-14602, which are too
annoying to maintain as patches. Instead, those fixes should be
upstreamed to SuperLU.
@rgommers
TST: sparse: fix test tolerances for splu/spilu tests
12cf125
@rgommers rgommers added enhancement A new feature or improvement scipy.sparse.linalg labels Sep 23, 2023
@rgommers rgommers added this to the 1.12.0 milestone Sep 23, 2023
@rgommers
Copy link
Copy Markdown
Member Author

rgommers commented Sep 23, 2023

Recording the refguide check failure for future reference:

Details 
===================
scipy.sparse.linalg
===================

scipy.sparse.linalg.SuperLU
---------------------------

File "/tmp/tmpezm8vra_/builtins", line ?, in SuperLU
Failed example:
    lu.perm_r
Expected:
    array([0, 2, 1, 3], dtype=int32)
Got:
    array([2, 1, 3, 0], dtype=int32)


File "/tmp/tmpezm8vra_/builtins", line ?, in SuperLU
Failed example:
    lu.perm_c
Expected:
    array([2, 0, 1, 3], dtype=int32)
Got:
    array([0, 1, 3, 2], dtype=int32)


File "/tmp/tmpezm8vra_/builtins", line ?, in SuperLU
Failed example:
    lu.L.A
Expected:
    array([[ 1. ,  0. ,  0. ,  0. ],
           [ 0. ,  1. ,  0. ,  0. ],
           [ 0. ,  0. ,  1. ,  0. ],
           [ 1. ,  0.5,  0.5,  1. ]])
Got:
    array([[ 1. ,  0. ,  0. ,  0. ],
           [ 0.5,  1. ,  0. ,  0. ],
           [ 0.5, -1. ,  1. ,  0. ],
           [ 0.5,  1. ,  0. ,  1. ]])


File "/tmp/tmpezm8vra_/builtins", line ?, in SuperLU
Failed example:
    lu.U.A
Expected:
    array([[ 2.,  0.,  1.,  4.],
           [ 0.,  2.,  1.,  1.],
           [ 0.,  0.,  1.,  1.],
           [ 0.,  0.,  0., -5.]])
Got:
    array([[ 2. ,  2. ,  0. ,  1. ],
           [ 0. , -1. ,  1. , -0.5],
           [ 0. ,  0. ,  5. , -1. ],
           [ 0. ,  0. ,  0. ,  2. ]])

The results are correct; the decomposition still reconstructs, but the numerical values have all changed. Adding a # may vary.

@rgommers
Copy link
Copy Markdown
Member Author

rgommers commented Oct 11, 2023

This is a huge PR, but most of it is changes pulled in from upstream that don't need review. I'd suggest reviewing only the last four commits (which are small and self-contained), and only briefly browsing the rest for really obvious issues like in the README.

Can I tempt someone to hit the Merge button here (don't squash-merge please)? This was a pain to unravel, so I'd much prefer for it not to accumulate merge conflicts.

@tylerjereddy
Copy link
Copy Markdown
Contributor

tylerjereddy commented Oct 11, 2023

Ok, I spent some time looking over the diff and it seems like this moves us in a desirable direction re: 64-bit stuff/upstreaming patches. If this does cause problems you'll likely help me with it at release time anyway.

I will ping on the testing change as a precaution after merging, just because there's a type coercion in there that wasn't there before.

@tylerjereddy tylerjereddy merged commit a608ea4 into scipy:main Oct 11, 2023
tylerjereddy
tylerjereddy reviewed Oct 11, 2023
View reviewed changes
Comment thread scipy/sparse/tests/test_array_api.py
LU.solve(np.array([1, 2, 3, 4])),
np.asarray([1, 0, 0, 0], dtype=np.float64),
rtol=1e-14, atol=3e-16
)
Copy link
Copy Markdown
Contributor

@tylerjereddy tylerjereddy Oct 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stefanv @perimosocordiae I merged this PR to move things along for i.e., eventual 64-bit SuperLU support, patch upstreaming, etc. The testing shims don't look too controversial to me, but if there is something that worries you can you ping me? It was just the new coercion that made me slightly nervous as non-expert reviewer on sparse linalg.

Copy link
Copy Markdown
Member Author

@rgommers rgommers Oct 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC this was a matter of the result from LU.solve being on the order of 1e-17 rather than exactly zero, and the default atol=0 not working for that. It's been a few weeks, but IIRC this is business as usual with minor differences in floating-point results.

@rgommers rgommers deleted the update-superlu branch October 11, 2023 18:53
This was referenced Oct 16, 2023
Closed
Merged
@dschmitz89 dschmitz89 mentioned this pull request Nov 4, 2023
Merged
@tylerjereddy tylerjereddy mentioned this pull request Dec 27, 2023
Merged
@nickodell nickodell mentioned this pull request Jun 19, 2024
Closed
@dschmitz89 dschmitz89 mentioned this pull request Jan 28, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tylerjereddy tylerjereddy tylerjereddy left review comments

Assignees

No one assigned

Labels

enhancement A new feature or improvement scipy.sparse.linalg

Projects

None yet

Milestone

1.12.0

Development

Successfully merging this pull request may close these issues.

ENH: update vendored SuperLU version

2 participants

@rgommers @tylerjereddy