Skip to content

Clean up the obsolete API of Universal Intrinsic#24371

Merged
asmorkalov merged 1 commit intoopencv:4.xfrom
hanliutong:clean-up
Oct 20, 2023
Merged

Clean up the obsolete API of Universal Intrinsic#24371
asmorkalov merged 1 commit intoopencv:4.xfrom
hanliutong:clean-up

Conversation

@hanliutong
Copy link
Copy Markdown
Contributor

@hanliutong hanliutong commented Oct 7, 2023 •
edited by opencv-alalek
Loading

Merge with contrib: opencv/opencv_contrib#3572

This patch cleans up the obsolete API of Universal Intrinsic to ensure that new code will not use them.

Obsolete API of Universal Intrinsic

In order to support scalable vector backends (such as RVV), we introduced some API changes to Universal Intrinsic last year: PR #22179. These changes resulted in following obsolete APIs: they only work with fixed-length backends and are incompatible with scalable vector backend.

Obsolete APIs New APIs
vType::nlanes VTraits<vType>::vlanes()
vType::nlanes (constant) VTraits<vType>::max_nlanes
vType::lane_type VTraits<vType>::lane_type
overloaded operator (vc += va + vb) functions ( vc = v_add(va, vb, vc))
vObj.get0() v_get(vObj)
v_extract_n<vType::nlanes-1>(vObj) v_extract_highest(vObj)
v_broadcast_element<vType::nlanes-1>(vObj) v_broadcast_highest(vObj)

Clean up

To prevent misuse of obsolete APIs, we need to clean up these APIs on at least 1 platform so that the GitHub CI can help with prevent any new code that uses these obsolete APIs from being merged.

This is also the main content of this patch, which mainly includes two aspects:

  1. Rewrite all obsolete APIs to new APIs in the existing code
    • Code block that can be apple to sacalabe platfrom: rewrite completed by PR Rewrite Universal Intrinsic code: float related part #24325 and the mentioned series of "Rewrite..." patchs.
    • Code block that only used for fixed platfrom: Although it cannot be enabled for sacalabe platform, it is still necessary to change to the new API to deprecate the obsolete API. Rewrite via this patch, commit ec15cb9
  2. Clean up the obsolete API on ARM Neon backend.
    • For overloaded operator (+), these APIs have been removed. Attempts to use obsolete overloaded operators will encounter the compiler error error: invalid operands to binary expression
    • For others, these APIs have been marked as private. Attempts to use obsolete API will encounter the compiler error error: '...' is a private member of '...'

Note: this patch is waiting for #24324, for testing purposes, this patch contains 419060d, which will be removed after #24324 is merged

TODO: The Universal Intrinsic documentation needs to be modified, I am woring on it.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@hanliutong hanliutong changed the title Clean up Clean up the obsolete API of Universal Intrinsic Oct 7, 2023
@opencv-alalek opencv-alalek added optimization cleanup Code cleanup (e.g, drop legacy C-API, legacy unmaintained code) labels Oct 7, 2023
@mshabunin mshabunin self-assigned this Oct 7, 2023
@hanliutong hanliutong mentioned this pull request Oct 8, 2023
Merged
6 tasks
@hanliutong
Copy link
Copy Markdown
Contributor Author

hanliutong commented Oct 8, 2023

Related: opencv/opencv_contrib#3572

@hanliutong hanliutong force-pushed the clean-up branch 2 times, most recently from 365dbd9 to df45b85 Compare October 9, 2023 14:41
mshabunin
mshabunin reviewed Oct 9, 2023
View reviewed changes
mshabunin
mshabunin reviewed Oct 10, 2023
View reviewed changes
@hanliutong
Copy link
Copy Markdown
Contributor Author

hanliutong commented Oct 13, 2023

Hello @asmorkalov, I found that the #24324 has been merged and I am going to resolve the conflict through rebase 4.x

Should I Squash the commits into one? or just rebase and keep those commits as is?

@asmorkalov
Copy link
Copy Markdown
Contributor

asmorkalov commented Oct 13, 2023

@hanliutong I recommend to squash commits.

@hanliutong
Copy link
Copy Markdown
Contributor Author

hanliutong commented Oct 13, 2023

squashed

@hanliutong
Copy link
Copy Markdown
Contributor Author

hanliutong commented Oct 13, 2023 •
edited
Loading

The test passed after re-running

I found that there is a Test_Darknet_nets.YOLOv4x_mish/0 failing test cases on macOS-ARM64-Vulkan

2023-10-13T12:25:09.6556050Z [ RUN      ] Test_Darknet_nets.YOLOv4x_mish/0, where GetParam() = VKCOM/VULKAN
2023-10-13T12:25:10.6810170Z /Users/opencv-cn/GHA-OCV-1/_work/opencv/opencv/opencv/modules/dnn/test/test_common.impl.hpp:109: Failure
2023-10-13T12:25:10.6839720Z Value of: testClassIds.empty()
2023-10-13T12:25:10.6851480Z   Actual: true
2023-10-13T12:25:10.6852000Z Expected: false
2023-10-13T12:25:10.6852660Z No detections
2023-10-13T12:25:10.6853170Z Google Test trace:
2023-10-13T12:25:10.6854800Z /Users/opencv-cn/GHA-OCV-1/_work/opencv/opencv/opencv/modules/dnn/test/test_darknet_importer.cpp:977: batch size 1

but I can not reproduce it on my m1 mac :

% ./bin/opencv_test_dnn --gtest_filter="Test_Darknet_nets.YOLOv4x_mish/0"                     
CTEST_FULL_OUTPUT
OpenCV version: 4.8.0-dev
OpenCV VCS version: 4.8.0-313-ga287605c3e
Build type: Release
Compiler: /Library/Developer/CommandLineTools/usr/bin/c++  (ver 14.0.0.14000029)
Parallel framework: gcd (nthreads=10)
CPU features: NEON FP16
TEST: Skip tests with tags: 'mem_6gb', 'verylong', 'dnn_skip_opencv_backend', 'dnn_skip_cpu', 'dnn_skip_cpu_fp16', 'dnn_skip_ocl', 'dnn_skip_ocl_fp16', 'dnn_skip_vulkan', 'dnn_skip_onnx_conformance', 'dnn_skip_parser'
Note: Google Test filter = Test_Darknet_nets.YOLOv4x_mish/0
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from Test_Darknet_nets
[ RUN      ] Test_Darknet_nets.YOLOv4x_mish/0, where GetParam() = VKCOM/VULKAN
[       OK ] Test_Darknet_nets.YOLOv4x_mish/0 (1599 ms)
[----------] 1 test from Test_Darknet_nets (1599 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (1599 ms total)
[  PASSED  ] 1 test.

my cmake config is % cmake ../ -G Ninja -DPYTHON_DEFAULT_EXECUTABLE=/usr/bin/python3 -DOPENCV_ENABLE_NONFREE=ON -DBUILD_ZLIB=OFF -DWITH_OPENCL=OFF -DWITH_VULKAN=ON

Details 
% cmake .
-- Detected processor: arm64
-- Found PythonInterp: /Users/plct/anaconda3/bin/python (found suitable version "3.11.4", minimum required is "2.7") 
-- Looking for ccache - not found
-- libjpeg-turbo: VERSION = 2.1.3, BUILD = opencv-4.8.0-dev-libjpeg-turbo
-- Use full Neon SIMD intrinsics implementation (NEON_INTRINSICS = ON)
-- SIMD extensions: arm64 (WITH_SIMD = 1)
-- Could NOT find OpenJPEG (minimal suitable version: 2.0, recommended version >= 2.3.1). OpenJPEG will be built from sources
-- OpenJPEG: VERSION = 2.5.0, BUILD = opencv-4.8.0-dev-openjp2-2.5.0
-- OpenJPEG libraries will be built from sources: libopenjp2 (version "2.5.0")
-- LAPACK(LAPACK/Apple): LAPACK_LIBRARIES: /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/System/Library/Frameworks/Accelerate.framework;-lm;-ldl
-- LAPACK(LAPACK/Apple): Support is enabled.
-- Could NOT find JNI (missing: JAVA_INCLUDE_PATH JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH) 
-- VTK is not found. Please set -DVTK_DIR in CMake to VTK build directory, or to VTK install subdirectory with VTKConfig.cmake file
-- Allocator metrics storage type: 'int'
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin128.sse2.cpp
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin128.sse3.cpp
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin128.ssse3.cpp
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin128.sse4_1.cpp
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin128.sse4_2.cpp
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin128.avx.cpp
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin128.avx2.cpp
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin128.avx512_skx.cpp
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin256.avx2.cpp
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin256.avx512_skx.cpp
-- Excluding from source files list: <BUILD>/modules/core/test/test_intrin512.avx512_skx.cpp
-- Excluding from source files list: modules/imgproc/src/corner.avx.cpp
-- Excluding from source files list: modules/imgproc/src/imgwarp.avx2.cpp
-- Excluding from source files list: modules/imgproc/src/imgwarp.lasx.cpp
-- Excluding from source files list: modules/imgproc/src/imgwarp.sse4_1.cpp
-- Excluding from source files list: modules/imgproc/src/resize.avx2.cpp
-- Excluding from source files list: modules/imgproc/src/resize.lasx.cpp
-- Excluding from source files list: modules/imgproc/src/resize.sse4_1.cpp
-- Registering hook 'INIT_MODULE_SOURCES_opencv_dnn': /Users/plct/PLCT/hanliutong/opencv/modules/dnn/cmake/hooks/INIT_MODULE_SOURCES_opencv_dnn.cmake
-- opencv_dnn: filter out ocl4dnn source code
-- opencv_dnn: filter out cuda4dnn source code
-- Excluding from source files list: <BUILD>/modules/dnn/layers/layers_common.avx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/layers_common.avx2.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/layers_common.avx512_skx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/layers_common.rvv.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/layers_common.lasx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/int8layers/layers_common.avx2.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/int8layers/layers_common.avx512_skx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/int8layers/layers_common.lasx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/conv_block.avx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/conv_block.avx2.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/conv_depthwise.avx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/conv_depthwise.avx2.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/conv_depthwise.rvv.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/conv_depthwise.lasx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/conv_winograd_f63.avx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/conv_winograd_f63.avx2.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/fast_gemm_kernels.avx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/fast_gemm_kernels.avx2.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/cpu_kernels/fast_gemm_kernels.lasx.cpp
-- Excluding from source files list: modules/features2d/src/fast.avx2.cpp
-- imgcodecs: OpenEXR codec is disabled in runtime. Details: https://github.com/opencv/opencv/issues/21326
-- highgui: using builtin backend: COCOA
-- Found 'misc' Python modules from /Users/plct/PLCT/hanliutong/opencv/modules/python/package/extra_modules
-- Found 'mat_wrapper;utils' Python modules from /Users/plct/PLCT/hanliutong/opencv/modules/core/misc/python/package
-- Found 'gapi' Python modules from /Users/plct/PLCT/hanliutong/opencv/modules/gapi/misc/python/package
-- 
-- General configuration for OpenCV 4.8.0-dev =====================================
--   Version control:               4.8.0-313-ga287605c3e
-- 
--   Platform:
--     Timestamp:                   2023-10-13T13:34:49Z
--     Host:                        Darwin 21.6.0 arm64
--     CMake:                       3.23.0
--     CMake generator:             Ninja
--     CMake build tool:            /opt/homebrew/bin/ninja
--     Configuration:               Release
-- 
--   CPU/HW features:
--     Baseline:                    NEON FP16
-- 
--   C/C++:
--     Built as dynamic libs?:      YES
--     C++ standard:                11
--     C++ Compiler:                /Library/Developer/CommandLineTools/usr/bin/c++  (ver 14.0.0.14000029)
--     C++ flags (Release):         -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -Wno-deprecated-copy -O3 -DNDEBUG  -DNDEBUG
--     C++ flags (Debug):           -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -Wno-deprecated-copy -g  -O0 -DDEBUG -D_DEBUG
--     C Compiler:                  /Library/Developer/CommandLineTools/usr/bin/cc
--     C flags (Release):           -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C flags (Debug):             -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     Linker flags (Release):      -Wl,-dead_strip  
--     Linker flags (Debug):        -Wl,-dead_strip  
--     ccache:                      NO
--     Precompiled headers:         NO
--     Extra dependencies:
--     3rdparty dependencies:
-- 
--   OpenCV modules:
--     To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo stitching ts video videoio
--     Disabled:                    world
--     Disabled by dependency:      -
--     Unavailable:                 java python2 python3
--     Applications:                tests perf_tests apps
--     Documentation:               NO
--     Non-free algorithms:         YES
-- 
--   GUI:                           COCOA
--     Cocoa:                       YES
--     VTK support:                 NO
-- 
--   Media I/O: 
--     ZLib:                        /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/usr/lib/libz.tbd (ver 1.2.11)
--     JPEG:                        build-libjpeg-turbo (ver 2.1.3-62)
--       SIMD Support Request:      YES
--       SIMD Support:              YES
--     WEBP:                        build (ver encoder: 0x020f)
--     PNG:                         build (ver 1.6.37)
--     TIFF:                        build (ver 42 - 4.2.0)
--     JPEG 2000:                   build (ver 2.5.0)
--     OpenEXR:                     build (ver 2.3.0)
--     HDR:                         YES
--     SUNRASTER:                   YES
--     PXM:                         YES
--     PFM:                         YES
-- 
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      NO
--       avcodec:                   NO
--       avformat:                  NO
--       avutil:                    NO
--       swscale:                   NO
--       avresample:                NO
--     GStreamer:                   NO
--     AVFoundation:                YES
-- 
--   Parallel framework:            GCD
-- 
--   Trace:                         YES (with Intel ITT)
-- 
--   Other third-party libraries:
--     Lapack:                      YES (/Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/System/Library/Frameworks/Accelerate.framework -lm -ldl)
--     Eigen:                       NO
--     Custom HAL:                  YES (carotene (ver 0.0.1))
--     Protobuf:                    build (3.19.1)
--     Flatbuffers:                 builtin/3rdparty (23.5.9)
-- 
--   Vulkan:                        YES
--     Include path:                /Users/plct/PLCT/hanliutong/opencv/3rdparty/include
--     Link libraries:              Dynamic load
-- 
--   Python (for build):            /usr/bin/python3
-- 
--   Java:                          
--     ant:                         NO
--     Java:                        YES (ver 18.0.1)
--     JNI:                         NO
--     Java wrappers:               NO
--     Java tests:                  NO
-- 
--   Install to:                    /usr/local
-- -----------------------------------------------------------------
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/plct/PLCT/hanliutong/opencv/.vulkan

Do you have any ideas? What do I need to do further?

@asmorkalov asmorkalov added this to the 4.9.0 milestone Oct 20, 2023
@asmorkalov asmorkalov merged commit 1c0ca41 into opencv:4.x Oct 20, 2023
@asmorkalov asmorkalov mentioned this pull request Nov 3, 2023
Merged
@Kumataro Kumataro mentioned this pull request Dec 29, 2023
Closed
4 tasks
@s-trinh s-trinh mentioned this pull request Jan 16, 2024
Closed
@s-trinh s-trinh mentioned this pull request May 18, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mshabunin mshabunin mshabunin approved these changes

Assignees

@mshabunin mshabunin

Labels

cleanup Code cleanup (e.g, drop legacy C-API, legacy unmaintained code) optimization

Projects

None yet

Milestone

4.9.0

Development

Successfully merging this pull request may close these issues.

4 participants

@hanliutong @asmorkalov @mshabunin @opencv-alalek