LAPACK_xxx is the universal way to go for lapack >= 3.40 with c/c++ i… by dwardor · Pull Request #21291 · opencv/opencv

dwardor · 2021-12-19T23:44:34Z

…nterface.

This is an alternate solution to #21116 (explained by #21114 (comment))
It works on my gentoo box... doing a pull request to test it here on th ebuildbot. If it works it seems better/cleaner than #21114 to me as it uses lapack's "unpreprocessed" function names and relies on lapack's preprocessor macros to do all the magic...

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
The PR is proposed to proper branch
There is reference to original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

…nterface.

alalek · 2021-12-21T16:02:13Z

OpenCV LAPACK`'s support is wider that just lapack integration.

different lapack implementations could be used including Apple's Accelerate framework (which fails on "Mac" builder)
some BLAS routines are used under this check too (this is confusing)

Tested on ArchLinux with fresh lapack-3.10.0-1 + lapacke-3.10.0-1:

CMake:

-- LAPACK(LAPACK/Generic): LAPACK_LIBRARIES: /lib64/liblapack.so;/lib64/libblas.so
-- LAPACK(LAPACK/Generic): Support is enabled.
...
--     Lapack:                      YES (/lib64/liblapack.so /lib64/libblas.so)

Failed on linking step for opencv_test_core:

/usr/bin/ld: lib/libopencv_core.so.4.5.4: undefined reference to `cblas_dgemm'
/usr/bin/ld: lib/libopencv_core.so.4.5.4: undefined reference to `cblas_cgemm'
/usr/bin/ld: lib/libopencv_core.so.4.5.4: undefined reference to `cblas_zgemm'
/usr/bin/ld: lib/libopencv_core.so.4.5.4: undefined reference to `cblas_sgemm'

Need to investigate (probably it is a different problem).

dwardor · 2021-12-21T19:30:44Z

OpenCV LAPACK`'s support is wider that just lapack integration.

different lapack implementations could be used including Apple's Accelerate framework (which fails on "Mac" builder)

some BLAS routines are used under this check too (this is confusing)

Tested on ArchLinux with fresh lapack-3.10.0-1 + lapacke-3.10.0-1:

CMake:
-- LAPACK(LAPACK/Generic): LAPACK_LIBRARIES: /lib64/liblapack.so;/lib64/libblas.so
-- LAPACK(LAPACK/Generic): Support is enabled.
...
--     Lapack:                      YES (/lib64/liblapack.so /lib64/libblas.so)
Failed on linking step for opencv_test_core:
/usr/bin/ld: lib/libopencv_core.so.4.5.4: undefined reference to `cblas_dgemm'
/usr/bin/ld: lib/libopencv_core.so.4.5.4: undefined reference to `cblas_cgemm'
/usr/bin/ld: lib/libopencv_core.so.4.5.4: undefined reference to `cblas_zgemm'
/usr/bin/ld: lib/libopencv_core.so.4.5.4: undefined reference to `cblas_sgemm'
Need to investigate (probably it is a different problem).

I haven't touched anything "cblas" so yes I'd say those undefined references are another issue...

How does Apple's "Accelerate" (or other lapack implementations) define the lapack functions in the appropriate ".h" files ? LAPACK_xxxx or xxxx_ or ????

alalek · 2021-12-21T19:53:45Z

CMakeError log has this:

LAPACK(LAPACK/Apple) check FAILED:
    LAPACK_INCLUDE_DIR: ''
    LAPACK_LIBRARIES: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Accelerate.framework;-lm;-ldl'
    LAPACK_LINK_LIBRARIES: '/usr/local/lib'
    Output:
Change Dir: /build/precommit_macosx/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/local/bin/ninja cmTC_ec99f && [1/2] Building CXX object CMakeFiles/cmTC_ec99f.dir/lapack_check.cpp.o
FAILED: CMakeFiles/cmTC_ec99f.dir/lapack_check.cpp.o 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++  -I. -I/build/precommit_macosx/build -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Wno-long-long -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections  -fvisibility=hidden -fvisibility-inlines-hidden  -O3 -DNDEBUG  -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -fPIE -std=c++11 -MD -MT CMakeFiles/cmTC_ec99f.dir/lapack_check.cpp.o -MF CMakeFiles/cmTC_ec99f.dir/lapack_check.cpp.o.d -o CMakeFiles/cmTC_ec99f.dir/lapack_check.cpp.o -c /build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:11:2: error: Developper needs to figure this out as both LAPACK_GLOBAL and LAPACK_NAME are undefined
#error Developper needs to figure this out as both LAPACK_GLOBAL and LAPACK_NAME are undefined
 ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:15:47: error: use of undeclared identifier 'sgesv'
static char* check_fn1 = (char*)LAPACK_GLOBAL(sgesv,SGESV);
                                              ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:15:53: error: use of undeclared identifier 'SGESV'
static char* check_fn1 = (char*)LAPACK_GLOBAL(sgesv,SGESV);
                                                    ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:16:47: error: use of undeclared identifier 'sposv'
static char* check_fn2 = (char*)LAPACK_GLOBAL(sposv,SPOSV);
                                              ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:16:53: error: use of undeclared identifier 'SPOSV'
static char* check_fn2 = (char*)LAPACK_GLOBAL(sposv,SPOSV);
                                                    ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:17:47: error: use of undeclared identifier 'spotrf'
static char* check_fn3 = (char*)LAPACK_GLOBAL(spotrf,SPOTRF);
                                              ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:17:54: error: use of undeclared identifier 'SPOTRF'
static char* check_fn3 = (char*)LAPACK_GLOBAL(spotrf,SPOTRF);
                                                     ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:18:47: error: use of undeclared identifier 'sgesdd'
static char* check_fn4 = (char*)LAPACK_GLOBAL(sgesdd,SGESDD);
                                              ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:18:54: error: use of undeclared identifier 'SGESDD'
static char* check_fn4 = (char*)LAPACK_GLOBAL(sgesdd,SGESDD);
                                                     ^
9 errors generated.
ninja: build stopped: subcommand failed.

… using netlib's reference LAPACK implementation version >= 3.4.0

dwardor · 2021-12-21T20:58:31Z

CMakeError log has this:

LAPACK(LAPACK/Apple) check FAILED:
    LAPACK_INCLUDE_DIR: ''
    LAPACK_LIBRARIES: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Accelerate.framework;-lm;-ldl'
    LAPACK_LINK_LIBRARIES: '/usr/local/lib'
    Output:
Change Dir: /build/precommit_macosx/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/local/bin/ninja cmTC_ec99f && [1/2] Building CXX object CMakeFiles/cmTC_ec99f.dir/lapack_check.cpp.o
FAILED: CMakeFiles/cmTC_ec99f.dir/lapack_check.cpp.o 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++  -I. -I/build/precommit_macosx/build -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Wno-long-long -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections  -fvisibility=hidden -fvisibility-inlines-hidden  -O3 -DNDEBUG  -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -fPIE -std=c++11 -MD -MT CMakeFiles/cmTC_ec99f.dir/lapack_check.cpp.o -MF CMakeFiles/cmTC_ec99f.dir/lapack_check.cpp.o.d -o CMakeFiles/cmTC_ec99f.dir/lapack_check.cpp.o -c /build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:11:2: error: Developper needs to figure this out as both LAPACK_GLOBAL and LAPACK_NAME are undefined
#error Developper needs to figure this out as both LAPACK_GLOBAL and LAPACK_NAME are undefined
 ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:15:47: error: use of undeclared identifier 'sgesv'
static char* check_fn1 = (char*)LAPACK_GLOBAL(sgesv,SGESV);
                                              ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:15:53: error: use of undeclared identifier 'SGESV'
static char* check_fn1 = (char*)LAPACK_GLOBAL(sgesv,SGESV);
                                                    ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:16:47: error: use of undeclared identifier 'sposv'
static char* check_fn2 = (char*)LAPACK_GLOBAL(sposv,SPOSV);
                                              ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:16:53: error: use of undeclared identifier 'SPOSV'
static char* check_fn2 = (char*)LAPACK_GLOBAL(sposv,SPOSV);
                                                    ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:17:47: error: use of undeclared identifier 'spotrf'
static char* check_fn3 = (char*)LAPACK_GLOBAL(spotrf,SPOTRF);
                                              ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:17:54: error: use of undeclared identifier 'SPOTRF'
static char* check_fn3 = (char*)LAPACK_GLOBAL(spotrf,SPOTRF);
                                                     ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:18:47: error: use of undeclared identifier 'sgesdd'
static char* check_fn4 = (char*)LAPACK_GLOBAL(sgesdd,SGESDD);
                                              ^
/build/precommit_macosx/4.x/opencv/cmake/checks/lapack_check.cpp:18:54: error: use of undeclared identifier 'SGESDD'
static char* check_fn4 = (char*)LAPACK_GLOBAL(sgesdd,SGESDD);
                                                     ^
9 errors generated.
ninja: build stopped: subcommand failed.

Can you test with the lattest patch I just pushed ?

dwardor · 2021-12-21T22:00:15Z

OK so Mac build fails... suggesting Accelerate implementation does not use LAPACK_xxxx preprocessor macros as does the reference implementation...
---> So I guess this approach is dead... and we are back to #21114

dwardor added 2 commits December 20, 2021 00:26

LAPACK_xxx is the universal way to go for lapack >= 3.40 with c/c++ i…

dd11222

…nterface.

Fix whitespace causing buildbot Docs failure

37008bd

dwardor mentioned this pull request Dec 20, 2021

Fix compile against lapack-3.10.0 - part 1 #21114

Merged

8 tasks

Explain preprocessor tests better and use old assumption if not built…

6108ca2

… using netlib's reference LAPACK implementation version >= 3.4.0

dwardor closed this Dec 21, 2021

dwardor deleted the alternate branch December 22, 2021 14:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

LAPACK_xxx is the universal way to go for lapack >= 3.40 with c/c++ i…#21291

LAPACK_xxx is the universal way to go for lapack >= 3.40 with c/c++ i…#21291
dwardor wants to merge 3 commits intoopencv:4.xfrom
dwardor:alternate

dwardor commented Dec 19, 2021 •

edited

Loading

Uh oh!

alalek commented Dec 21, 2021

Uh oh!

dwardor commented Dec 21, 2021

Uh oh!

alalek commented Dec 21, 2021

Uh oh!

dwardor commented Dec 21, 2021

Uh oh!

dwardor commented Dec 21, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

dwardor commented Dec 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Uh oh!

alalek commented Dec 21, 2021

Uh oh!

dwardor commented Dec 21, 2021

Uh oh!

alalek commented Dec 21, 2021

Uh oh!

dwardor commented Dec 21, 2021

Uh oh!

dwardor commented Dec 21, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dwardor commented Dec 19, 2021 •

edited

Loading