I think we try to dispatch to the proper loops at runtime. @seiko2plus Thoughts?

I didn't try too hard to fix the undefined symbol issue on the AMD so I don't know if this was some oversight on my end. The bigger motivator here is the cross-compilation issue. Since SVML supports a very limited slice of numpy targets and is disabled (for native builds) on all others, it seems that the cleanest way around this is and any potential incompatibilities is to just add a switch that short-circuits the compatibility check and disables the library.

It seems to hook into the dispatch mechanism in a correct way in gh-19478. @ahesford what compiler and Linux distro are you using?

Adding a short-circuit for cross-compiling seems fine to me.

I am building this with xbps-src in Void Linux in preparation for updating our distribution package (void-linux/void-packages#34817). After rebuilding the package, I can't seem to reproduce the unresolved symbol issue, so that was likely an oversight (like a dirty build directory) on my end. The shlib _multiarray_umath.so in my latest build definitely containts __svml_log108 (and other __svml_* symbols) and is importable without issue on an AMD Zen 2 system. The numpy.test() suite passes just fine (except for two expected failures that result from Void's renaming of Python shlibs).

Thus, it seems the only real utility here is easy support for cross-compilation.

@ahesford, you can disable avx512 features through CPU build options, it will not exclude SVML objects from the final linking but it gonna fix this undefined symbol error.

The undefined symbol error is no longer an issue. It seems to have been a one-off error as a result of repeated build trials.

@ahesford, May I know what is the current issue this pr trying to solve?

At this point, convenient disabling of SVML when cross-compiling numpy. (The build will fail when an x86_64 host is used to compile for any other architecture because the runtime checks on the host will enable the build but the cross assembler will be unable to understand the instructions.)

LGTM +1.

Thanks @ahesford

Thanks!

Hi-five on merging your first pull request to NumPy, @ahesford! We hope you stick around! Your choices aren’t limited to programming – you can review pull requests, help us stay on top of new and old issues, develop educational material, work on our website, add or improve graphic design, create marketing materials, translate website content, write grant proposals, and help with other fundraising initiatives. For more info, check out: https://numpy.org/contribute/.
Also, consider joining our mailing list. This is a great way to connect with other cool people in our community and be part of important conversations that affect the development of NumPy: https://mail.python.org/mailman/listinfo/numpy-discussion

I want to revert these changes, there are two ways to fix this issue rather than adding an optional environment variable:

• add support to dispatchable sources to handle ASM sources similar to C/C++ sources so it can be fully controlled via --cpu-baseline, --cpu-dispatch note: current situation SVML functions are ignored only and still passed to the linker
• test avx512 compiler/linker capability inside core/setup.py before attaching SVML sources.

The second is the only viable option, since dispatch options do not overcome the fact that non-x86_64 cross compilers will fail when trying to assemble the SVML sources.

I do agree that a build-time check that the sources successfully compile should gate the SVML requirement rather than the current platform check. However, why prohibit exclusion even for native builds given that SVML currently supports only a tiny subset of the target platforms?

I recommend the current SVML disable flag remain until SVML supports a wide range of supported hardware and becomes more critical to NumPy operation. The flag is off by default and doesn't hurt anybody.

@ahesford,

The second is the only viable option, since dispatch options do not overcome the fact that non-x86_64 cross compilers will fail when trying to assemble the SVML sources.

it isn't true since we have an internal mechanism for testing each specified CPU feature against compiler and assembler before enabling it, that's why you didn't face the same issue with AVX512/C intrinsics. you can check the build log or try to take a look at the SIMD doc for more details.

However, why prohibit exclusion even for native builds given that SVML currently supports only a tiny subset of the target platforms?

SVML functions aren't executed unless the target CPU support it, otherwise we fall back to C standard math lib. It's hard for maintainers to miss the chance for Linux users to get SVML/AVX512 for free so I think it was the right decision to accept the Intel gift. currently, we are trying to convert Intel ASM code into universal intrinsics see #20363.

I understand that SVML functions aren't executed unless the target CPU supports it and it's nice to have accelerated routines on those (few) systems that support them. My point is that, because the build process must always support not compiling SVML because it is only compilable on native x86_64 Linux builds, I fail to see the downside of keeping a switch that short-circuits the check (which is currently "Am I a Linux x86_64 host?" and is inaccurate for cross builds, but will hopefully one day become "Will the library successfully compile?") and always returns False in answer to these questions. At a minimum, having the switch is good for debugging and testing by allowing quick removal of the AVX512 routines. Even better, it solves a real problem---cross compilation when packaging NumPy for Void Linux---even before a more complicated build process gets smarter about checking for SVML support on the target.

@charris you marked this for 1.22.1 but it wasn't actually backported - I think that's an oversight?