Skip to content

Use universal intrinsics in bayer2gray#26868

Merged
asmorkalov merged 6 commits intoopencv:4.xfrom
FantasqueX:bayer2gray-simd-2
Mar 11, 2025
Merged

Use universal intrinsics in bayer2gray#26868
asmorkalov merged 6 commits intoopencv:4.xfrom
FantasqueX:bayer2gray-simd-2

Conversation

@FantasqueX
Copy link
Copy Markdown
Contributor

@FantasqueX FantasqueX commented Feb 2, 2025

I tested this patch on my 14900K desktop with -DCPU_BASELINE=AVX2, performance doesn't go down.

Geometric mean (ms)

                             Name of Test                               old-avx2  avx2      avx2
                                                                                             vs
                                                                                          old-avx2
                                                                                         (x-factor)
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerBG2BGR)      4.325    4.139     1.04
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerBG2BGRA)     4.419    4.393     1.01
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerBG2BGR_VNG) 101.168  101.138    1.00
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerBG2GRAY)     0.950    0.913     1.04
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerGB2BGR)      4.085    4.045     1.01
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerGB2BGRA)     4.354    4.406     0.99
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerGB2BGR_VNG) 102.400  101.853    1.01
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerGB2GRAY)     1.083    1.011     1.07
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerGR2BGR)      4.346    4.076     1.07
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerGR2BGRA)     4.297    4.387     0.98
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerGR2BGR_VNG) 102.164  101.412    1.01
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerGR2GRAY)     0.904    0.949     0.95
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerRG2BGR)      4.101    4.131     0.99
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerRG2BGRA)     4.403    4.416     1.00
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerRG2BGR_VNG) 101.669  103.508    0.98
cvtColorBayer8u::Size_CvtMode_Bayer::(7680x4320, COLOR_BayerRG2GRAY)     0.978    0.861     1.14

I also try to use universal intrinsics on aarch64, however, time becomes twice as previous hand-written intrinsics. I'll find out that later.

For this patch, it should optimize the performance on machine with CV_SIMD_SCALABLE.

To make this patch simple, I only modify bayer2gray and leave bayer2bgr, bayer2bgra and bayer2bgr_ea as is.

Related: #25724 #26012

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@asmorkalov
Copy link
Copy Markdown
Contributor

@fengyuentau @hanliutong Could you try it?

@hanliutong
Copy link
Copy Markdown
Contributor

hanliutong commented Feb 3, 2025

tested on BPI-F3 (Spacemit X60, RVV 1.0)

Unfortunately, it looks like performance has degraded

Geometric mean (ms)

                            Name of Test                               old    new      new    
                                                                                        vs    
                                                                                       old    
                                                                                    (x-factor)
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2BGR)      0.036  0.036     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2BGRA)     0.084  0.084     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2BGR_VNG)  1.353  1.343     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2GRAY)     0.039  0.079     0.49   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2BGR)      0.036  0.036     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2BGRA)     0.087  0.087     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2BGR_VNG)  1.353  1.350     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2GRAY)     0.038  0.079     0.49   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2BGR)      0.036  0.036     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2BGRA)     0.087  0.087     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2BGR_VNG)  1.351  1.346     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2GRAY)     0.038  0.078     0.49   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2BGR)      0.036  0.036     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2BGRA)     0.084  0.084     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2BGR_VNG)  1.347  1.339     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2GRAY)     0.038  0.078     0.49   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2BGR)     0.422  0.424     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2BGRA)    0.768  0.764     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2BGR_VNG) 61.082 60.863    1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2GRAY)    0.304  0.658     0.46   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2BGR)     0.419  0.420     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2BGRA)    0.761  0.754     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2BGR_VNG) 61.457 60.840    1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2GRAY)    0.305  0.655     0.47   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2BGR)     0.430  0.429     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2BGRA)    0.753  0.757     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2BGR_VNG) 61.409 61.029    1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2GRAY)    0.305  0.660     0.46   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2BGR)     0.421  0.417     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2BGRA)    0.759  0.758     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2BGR_VNG) 61.307 60.980    1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2GRAY)    0.304  0.653     0.47

BTW, @FantasqueX , if you are interested in RISC-V devices, you can get cloud access to a device with RVV 1.0 at https://cloud.spacemit.com/

@FantasqueX
Copy link
Copy Markdown
Contributor Author

tested on BPI-F3 (Spacemit X60, RVV 1.0)

Unfortunately, it looks like performance has degraded

Geometric mean (ms)

                            Name of Test                               old    new      new    
                                                                                        vs    
                                                                                       old    
                                                                                    (x-factor)
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2BGR)      0.036  0.036     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2BGRA)     0.084  0.084     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2BGR_VNG)  1.353  1.343     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2GRAY)     0.039  0.079     0.49   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2BGR)      0.036  0.036     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2BGRA)     0.087  0.087     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2BGR_VNG)  1.353  1.350     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2GRAY)     0.038  0.079     0.49   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2BGR)      0.036  0.036     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2BGRA)     0.087  0.087     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2BGR_VNG)  1.351  1.346     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2GRAY)     0.038  0.078     0.49   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2BGR)      0.036  0.036     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2BGRA)     0.084  0.084     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2BGR_VNG)  1.347  1.339     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2GRAY)     0.038  0.078     0.49   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2BGR)     0.422  0.424     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2BGRA)    0.768  0.764     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2BGR_VNG) 61.082 60.863    1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2GRAY)    0.304  0.658     0.46   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2BGR)     0.419  0.420     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2BGRA)    0.761  0.754     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2BGR_VNG) 61.457 60.840    1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2GRAY)    0.305  0.655     0.47   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2BGR)     0.430  0.429     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2BGRA)    0.753  0.757     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2BGR_VNG) 61.409 61.029    1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2GRAY)    0.305  0.660     0.46   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2BGR)     0.421  0.417     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2BGRA)    0.759  0.758     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2BGR_VNG) 61.307 60.980    1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2GRAY)    0.304  0.653     0.47

BTW, @FantasqueX , if you are interested in RISC-V devices, you can get cloud access to a device with RVV 1.0 at https://cloud.spacemit.com/

Hi, @hanliutong. Thank you for testing and your advice. I have a P550 board which haven't setup yet. I'll investigate after investigating aarch64 issue.

@asmorkalov
Copy link
Copy Markdown
Contributor

Warnings:

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:129:40: warning: unused parameter 'bayer' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:129:51: warning: unused parameter 'bayer_step' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:129:70: warning: unused parameter 'dst' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:130:24: warning: unused parameter 'width' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:130:35: warning: unused parameter 'bcoeff' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:130:47: warning: unused parameter 'gcoeff' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:130:59: warning: unused parameter 'rcoeff' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:282:32: warning: unused parameter 'bayer' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:282:43: warning: unused parameter 'bayer_step' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:282:62: warning: unused parameter 'dst' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:282:71: warning: unused parameter 'width' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:282:82: warning: unused parameter 'blue' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:417:33: warning: unused parameter 'bayer' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:417:44: warning: unused parameter 'bayer_step' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:417:63: warning: unused parameter 'dst' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:417:72: warning: unused parameter 'width' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:417:83: warning: unused parameter 'blue' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:417:101: warning: unused parameter 'alpha' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:552:35: warning: unused parameter 'bayer' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:552:46: warning: unused parameter 'bayer_step' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:552:65: warning: unused parameter 'dst' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:552:74: warning: unused parameter 'width' [-Wunused-parameter]

/home/ci/opencv/modules/imgproc/src/demosaicing.cpp:552:85: warning: unused parameter 'blue' [-Wunused-parameter]

@vpisarev
Copy link
Copy Markdown
Contributor

vpisarev commented Mar 7, 2025

@FantasqueX, @hanliutong, the solution for performance regression on RISC-V is to use

#if CV_SIMD
...

instead of

#if CV_SIMD || CV_SIMD_SCALABLE
...

@asmorkalov
Copy link
Copy Markdown
Contributor

asmorkalov commented Mar 11, 2025

No regressions with NEON and RVV now. Desktop performance (AMD Ryzen 7 5700G, Ubuntu 24.04, gcc 13.3):

Geometric mean (ms)

                            Name of Test                              4.x patch  patch  
                                                                                        vs    
                                                                                      4.x   
                                                                                    (x-factor)
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2BGR)      0.003  0.003     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2BGRA)     0.003  0.003     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2BGR_VNG)  0.031  0.032     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerBG2GRAY)     0.003  0.002     1.28   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2BGR)      0.003  0.003     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2BGRA)     0.003  0.003     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2BGR_VNG)  0.032  0.036     0.87   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGB2GRAY)     0.003  0.002     1.29   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2BGR)      0.003  0.003     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2BGRA)     0.003  0.003     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2BGR_VNG)  0.031  0.031     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerGR2GRAY)     0.003  0.002     1.29   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2BGR)      0.003  0.003     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2BGRA)     0.003  0.003     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2BGR_VNG)  0.031  0.031     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(127x61, COLOR_BayerRG2GRAY)     0.003  0.002     1.29   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2BGR)     0.038  0.026     1.46   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2BGRA)    0.020  0.020     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2BGR_VNG) 1.281  1.277     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerBG2GRAY)    0.024  0.021     1.16   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2BGR)     0.026  0.023     1.13   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2BGRA)    0.021  0.020     1.09   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2BGR_VNG) 1.278  1.272     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGB2GRAY)    0.024  0.021     1.13   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2BGR)     0.023  0.023     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2BGRA)    0.020  0.020     1.00   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2BGR_VNG) 1.280  1.270     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerGR2GRAY)    0.023  0.022     1.06   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2BGR)     0.023  0.024     0.95   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2BGRA)    0.020  0.020     0.99   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2BGR_VNG) 1.287  1.271     1.01   
cvtColorBayer8u::Size_CvtMode_Bayer::(640x480, COLOR_BayerRG2GRAY)    0.021  0.022     0.94   

@asmorkalov asmorkalov merged commit 4bb57ce into opencv:4.x Mar 11, 2025
27 of 28 checks passed
@asmorkalov asmorkalov self-assigned this Mar 11, 2025
@asmorkalov asmorkalov added this to the 4.12.0 milestone Mar 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants