WIP/MAINT: distutils: Filter Accelerate libs, and don't use 'accelerate' for BLAS or LAPACK.#15966
WIP/MAINT: distutils: Filter Accelerate libs, and don't use 'accelerate' for BLAS or LAPACK.#15966WarrenWeckesser wants to merge 3 commits intonumpy:masterfrom
Conversation
…iles... ...and use it to filter BLAS and LAPACK symlinks to the Accelerate framework on MacOS.
…S or LAPACK libraries.
| # Now, check for optional libraries | ||
| opt_found_dirs, opt_found_libs = self._find_libs(lib_dirs, opt_libs, exts) | ||
| opt_found_dirs, opt_found_libs = self._find_libs( | ||
| lib_dirs, opt_libs, exts, filefilter=filefilter) |
There was a problem hiding this comment.
The linker picks the first library file with the correct name in lib_dirs it finds.
Does the filefilter logic here match that, i.e., stop at the first file (and then use filefilter to decide whether to accept/reject the whole shebang)?
There was a problem hiding this comment.
If the filter returns False, check_libs will act like the file doesn't exist. If there is some combination of inputs that would make the linker find that file during the build, then that could be a problem. I think I see what you're worried about, but to be clear, could you describe a concrete example where something like this might happen?
There was a problem hiding this comment.
For example:
/usr/lib/libblas.so -> Accelerate
/home/mypath/libblas.so -> Some other BLAS
With library_dirs /usr/lib:/home/mypath I guess the question is, which library directories get passed to the linker?
There was a problem hiding this comment.
I'm experimenting with something like that now. What is the standard way to tell numpy to use blas, cblas and lapack from a custom directory? Is site.cfg only for known libraries (i.e. [mkl], [atlas], etc), or can I add a section such as [mylib]? (My first attempt failed, but I had some other issues, so I'm still experimenting.)
There was a problem hiding this comment.
Nevermind. site.cfg like this seems to do it:
[blas]
library_dirs = /Users/warren/foo/lib
include_dirs = /Users/warren/foo/include
libraries = refblas,cblas
[lapack]
library_dirs = /Users/warren/foo/lib
include_dirs = /Users/warren/foo/include
libraries = lapack
There was a problem hiding this comment.
Since the code in numpy.distutils is attempting to do what the linker will also eventually do (i.e. find the binary library file), is it feasible to replace the use of -l and -L for a given library with the full path to the binary itself? So instead of -lblas -L/home/mypath, the linker would be given the filename itself /home/mypath/libblas.so (or whatever file was found, whether that be .so, .a, .dylib, etc.).
| # List of all known BLAS libraries, in the default order | ||
| lapack_order = ['mkl', 'openblas', 'flame', 'atlas', 'accelerate', 'lapack'] | ||
| # List of all known LAPACK libraries, in the default order | ||
| lapack_order = ['mkl', 'openblas', 'flame', 'atlas', 'lapack'] |
There was a problem hiding this comment.
If you take accelerate out here, you need to
- remove it from one of the travis configurations
- disable the test on azure that checks for an error when using accelerate
|
Closing. gh-15759 handles this now. |
No description provided.