Skip to content

python3Packages.numpy: 1.20.3 -> 1.21.1 #132754

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
veprbl merged 2 commits into from
Aug 7, 2021
Merged

python3Packages.numpy: 1.20.3 -> 1.21.1 #132754

veprbl merged 2 commits into from
Aug 7, 2021

Conversation

@mweinelt
Copy link
Member

@mweinelt mweinelt commented Aug 5, 2021

Motivation for this change

https://github.com/numpy/numpy/releases/tag/v1.21.1
https://github.com/numpy/numpy/releases/tag/v1.21.0

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • 21.11 Release Notes (or backporting 21.05 Relase notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Aug 5, 2021
@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 2501-5000 This PR causes many rebuilds on Darwin and should target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 2501-5000 This PR causes many rebuilds on Linux and should target the staging branches. labels Aug 5, 2021
@veprbl
Copy link
Member

veprbl commented Aug 5, 2021

@GrahamcOfBorg build python38Packages.numpy python39Packages.numpy

@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Aug 5, 2021

@GrahamcOfBorg build python38Packages.numpy python39Packages.numpy

I build them and they build successful.

@risicle
Copy link
Contributor

risicle commented Aug 5, 2021 •
edited
Loading

Failure on macos 10.15:

running install tests
/private/tmp/nix-build-python3.8-numpy-1.21.1.drv-0/numpy-1.21.1/dist /private/tmp/nix-build-python3.8-numpy-1.21.1.drv-0/numpy-1.21.1
Traceback (most recent call last):
  File "/nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: dlopen(/nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so, 2): Symbol not found: __cvtmask32_u32
  Referenced from: /nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so
  Expected in: flat namespace
 in /nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/__init__.py", line 150, in <module>
    from . import core
  File "/nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/core/__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/nix/store/vxgnw3468p06jb6n98wfsbiwz1q2xxhc-python3-3.8.11/bin/python3.8"
  * The NumPy version is: "1.21.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: dlopen(/nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so, 2): Symbol not found: __cvtmask32_u32
  Referenced from: /nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so
  Expected in: flat namespace
 in /nix/store/d1bchdjp93h7kxq8hlawllzc0kv7r3lg-python3.8-numpy-1.21.1/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so

builder for '/nix/store/9wybj7qjx1m9s6xgdkgd9ya0rvp0q2bq-python3.8-numpy-1.21.1.drv' failed with exit code 1

Short version: Symbol not found: __cvtmask32_u32

This was with a cherry-pick to master (could there be another commit in staging that fixes this though?)

@risicle
Copy link
Contributor

risicle commented Aug 5, 2021 •
edited
Loading

_cvtmask32_u32 seems to be a gcc AVX512 intrinsic (though clang seems to have it too if you dig enough)

@risicle
Copy link
Contributor

risicle commented Aug 5, 2021 •
edited
Loading

Some brief investigation suggests it was added in clang 8. I wonder if https://github.com/numpy/numpy/blob/623bc1fae1d47df24e7f1e29321d0c0ba2771ce0/numpy/distutils/checks/extra_avx512bw_mask.c is not doing its job properly.

@risicle
Copy link
Contributor

risicle commented Aug 5, 2021

Sure enough if we replace the line https://github.com/numpy/numpy/blob/623bc1fae1d47df24e7f1e29321d0c0ba2771ce0/numpy/core/src/common/simd/avx512/conversion.h#L109 with an #error ... declaration it fails there, so it looks like NPY_HAVE_AVX512BW_MASK is being set true for clang7 even though it definitely shouldn't be.

@veprbl
Copy link
Member

veprbl commented Aug 6, 2021

Result of nixpkgs-review pr 132754 run on x86_64-darwin 1

2 packages failed to build:
  • python38Packages.numpy
  • python39Packages.numpy

@veprbl veprbl mentioned this pull request Aug 6, 2021
Closed
@risicle
Copy link
Contributor

risicle commented Aug 6, 2021 •
edited
Loading

@veprbl nice! Is this something we want to conditionally set for clang only though? (p.s. fixes it for me, macos 10.15)

@veprbl
Copy link
Member

veprbl commented Aug 6, 2021

@veprbl nice! Is this something we want to conditionally set for clang only though? (p.s. fixes it for me, macos 10.15)

I had that thought, but it doesn't seem to be worth it to run into issues like this.

@veprbl veprbl merged commit 0cde594 into NixOS:staging Aug 7, 2021
@mweinelt mweinelt deleted the python/numpy branch August 7, 2021 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SuperSandro2000 SuperSandro2000 SuperSandro2000 approved these changes

@FRidh FRidh Awaiting requested review from FRidh

@jonringer jonringer Awaiting requested review from jonringer

Assignees

No one assigned

Labels

6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 2501-5000 This PR causes many rebuilds on Darwin and should target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 2501-5000 This PR causes many rebuilds on Linux and should target the staging branches.

Projects

No open projects
Staging
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mweinelt @veprbl @SuperSandro2000 @risicle