Skip to content

(5.x) Merge 4.x#25998

Merged
asmorkalov merged 31 commits intoopencv:5.xfrom
asmorkalov:5.x-merge-4.x
Aug 7, 2024
Merged

(5.x) Merge 4.x#25998
asmorkalov merged 31 commits intoopencv:5.xfrom
asmorkalov:5.x-merge-4.x

Conversation

@asmorkalov
Copy link
Copy Markdown
Contributor

@asmorkalov asmorkalov commented Aug 6, 2024

OpenCV Extra: opencv/opencv_extra#1199

#25199 from olpipi:fix_openvino_2024_0
#25643 from cpoerschke:issue-25635-find-existing-file-tests
#25644 from DaniAffCH:blockwise-quantization
#25786 from plctlab:rvp_3rdparty
#25930 from opencv-pushbot:gitee/alalek/cmake_try_detect_feature_without_flags
#25932 from asmorkalov:as/HAL_cvtColor_aprox
#25940 from Kumataro:fix25928
#25943 from asmorkalov:as/fisheye_distrort_newk
#25952 from alexlyulkov:al/java-orb-test
#25955 from Kumataro:fix25946
#25961 from gblikas:gblikas_4.x_ts_js
#25975 from asmorkalov:as/bmp_error_check
#25978 from chacha21:cuda_stdallocator
#25979 from asmorkalov:as/custom_allocator
#25983 from stepkamipt:4.x
#25987 from AvenSun:fix-namespace
#25991 from MaximSmolskiy:improve-corners-matching-in-ChessBoardDetector-NeighborsFinder-findCornerNeighbor

Previous "Merge 4.x": #25953

ci_branch=pr25998

cpoerschke and others added 30 commits May 25, 2024 08:53
Added ORB detection test for Java and Android
core: FileStorage: detect invalid attribute value
python: prefer cv::Mat over cv::UMat in python binding
modules/js/perf/perf_helpfunc.js and target tests, e.g. perf_gaussianBlur.js contained "const isNodeJs", leading to re-definition when using associated *.html files.
[GSoC] dnn: Blockwise quantization support opencv#25644

This PR introduces blockwise quantization in DNN allowing the parsing of ONNX models quantized in blockwise style. In particular it modifies the `Quantize` and `Dequantize` operations. The related PR opencv/opencv_extra#1181 contains the test data.

Additional notes:
- The original quantization issue has been fixed. Previously, for 1D scale and zero-point, the operation applied was  $y = int8(x/s - z)$ instead of $y = int8(x/s + z)$. Note that the operation was already correctly implemented when the scale and zero-point were scalars. The previous implementation failed the ONNX test cases, but now all have passed successfully.  [Reference](https://github.com/onnx/onnx/blob/main/docs/Operators.md#QuantizeLinear)
- the function `block_repeat` broadcasts scale and zero-point to the input shape. It repeats all the elements of a given axis n times. This function generalizes the behavior of `repeat` from the core module which is defined just for 2 axis assuming `Mat` has 2 dimensions. If appropriate and useful, you might consider moving `block_repeat` to the core module.
- Now, the scale and zero-point can be taken as layer inputs. This increases the ONNX layers' coverage and enables us to run the ONNX test cases (previously disabled) being fully compliant with ONNX standards. Since they are now supported, I have enabled the test cases for: `test_dequantizelinear`, `test_dequantizelinear_axis`, `test_dequantizelinear_blocked`, `test_quantizelinear`, `test_quantizelinear_axis`, `test_quantizelinear_blocked` just in CPU backend. All of them pass successfully.
   
### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
To be on par with `cv::Mat`, let's add `cv::cuda::GpuMat::getStdAllocator()`
This is useful anyway, because when a user wants to use custom allocators, he might want to resort to the standard default allocator behaviour, not some other allocator that could have been set by `setDefaultAllocator()`
Improved error handling in image codecs.
Current code using CMAKE_SOURCE_DIR and it works well if opencv is standalone CMake project,
but in case of building OpenCV as part of a larger CMake project (e.g. one that includes
opencv and opencv_contrib) this path is incorrect, unlike OpenCV_SOURCE_DIR
fix compilation errors caused by namespace
Adding getStdAllocator() to cv::cuda::GpuMat
Set and check allocator pointer for all cv::Mat instances
…tching-in-ChessBoardDetector-NeighborsFinder-findCornerNeighbor

Improve corners matching in ChessBoardDetector::NeighborsFinder::findCornerNeighbor opencv#25991

### Pull Request Readiness Checklist

Idea was mentioned in `Section III-B. New Heuristic for Quadrangle Linking` of `Rufli, Martin & Scaramuzza, Davide & Siegwart, Roland. (2008). Automatic Detection of Checkerboards on Blurred and Distorted Images. 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS. 3121-3126. 10.1109/IROS.2008.4650703` (https://rpg.ifi.uzh.ch/docs/IROS08_scaramuzza_b.pdf):
![Снимок экрана от 2024-08-05 09-51-27](https://github.com/user-attachments/assets/7a090ccc-c24c-4dfb-b0dd-259c8709eb72)
```
* For each candidate pair, focus on the quadrangles they belong to and draw two straight lines passing through the midsections of the respective quadrangle edges (see Fig. 6).
* If the candidate corner and the source corner are on the same side of every of the four straight lines drawn this way (this corresponds to the yellow shaded area in Fig. 6), then the corners are successfully matched.
```

By improving corners matching, we can increase the search radius (`thresh_scale`).

I tested this PR with benchmark
```
python3 objdetect_benchmark.py --configuration=generate_run --board_x=7 --path=res_chessboard --synthetic_object=chessboard
```
PR increases detected chessboards number by `3/7%`:
```
cell_img_size = 100 (default)

before
                                 category  detected chessboard  total detected chessboard  total chessboard  average detected error chessboard
                                      all             0.910417                      13110             14400                           0.599746
Total detected time:  147.50906700000002 sec

after
                                 category  detected chessboard  total detected chessboard  total chessboard  average detected error chessboard
                                      all             0.941667                      13560             14400                           0.596726
Total detected time:  136.68963200000007 sec

----------------------------------------------------------------------------------------------------------------------------------------------

cell_img_size = 10

before
                                 category  detected chessboard  total detected chessboard  total chessboard  average detected error chessboard
                                      all             0.539792                       7773             14400                           4.208237
Total detected time:  2.668964 sec

after
                                 category  detected chessboard  total detected chessboard  total chessboard  average detected error chessboard
                                      all             0.579167                       8340             14400                           4.198448
Total detected time:  2.535998999999999 sec
```

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
3rdparty: NDSRVP - Part 1.5: New Interfaces
…ting-file-tests

replace lena.jpg in find-existing-file tests
Added fisheye::distortPoints with non-identity projection matrix
Added xxxApprox overloads for YUV color conversions in HAL and AlgorithmHint to cvtColor opencv#25932

The xxxApprox to implement HAL functions with less bits for arithmetic of FP.

The hint was introduced in opencv#25792 and opencv#25911

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
@asmorkalov asmorkalov requested a review from fengyuentau August 6, 2024 11:47
@asmorkalov
Copy link
Copy Markdown
Contributor Author

asmorkalov commented Aug 6, 2024

@fengyuentau Could you take a look on the patch? I see test failure in DNN after merge of block quantization:

2024-08-06T11:10:23.7211780Z [ RUN      ] Test_ONNX_conformance.Layer_Test/test_dequantizelinear_blocked_OCV_CPU, where GetParam() = (test_dequantizelinear_blocked, OCV/CPU)
2024-08-06T11:10:23.7213089Z Exception during net.forward() call!
2024-08-06T11:10:23.7213718Z unknown file: Failure
2024-08-06T11:10:23.7217903Z C++ exception with description "OpenCV(5.0.0-pre) /home/ci/opencv/modules/dnn/src/int8layers/quantization_utils.cpp:44: error: (-2:Unspecified error) in function 'void cv::dnn::block_repeat(cv::InputArray, const MatShape&, int, int, cv::OutputArray)'
2024-08-06T11:10:23.7220016Z > Axis out of range:
2024-08-06T11:10:23.7220568Z >     'axis >= 0 && axis < src.dims()'
2024-08-06T11:10:23.7221166Z > where
2024-08-06T11:10:23.7221636Z >     'axis' is 1
2024-08-06T11:10:23.7222157Z " thrown in the test body.
2024-08-06T11:10:23.7223682Z [  FAILED  ] Test_ONNX_conformance.Layer_Test/test_dequantizelinear_blocked_OCV_CPU, where GetParam() = (test_dequantizelinear_blocked, OCV/CPU) (1 ms)

I can disable the test for now or apply a fix, if it's simple.

@asmorkalov
Copy link
Copy Markdown
Contributor Author

The mentioned tests are disabled for new and will be fixed with dedicated PR.

@opencv-alalek
Copy link
Copy Markdown
Contributor

opencv-alalek commented Aug 6, 2024

DNN tests on Linux32 are crashed. Somewhere since this weekly build: https://pullrequest.opencv.org/buildbot/builders/5_x_noOCL-lin32/builds/100632

@fengyuentau
Copy link
Copy Markdown
Member

DNN tests on Linux32 are crashed. Somewhere since this weekly build: https://pullrequest.opencv.org/buildbot/builders/5_x_noOCL-lin32/builds/100632

Looks like memory leaks somewhere in einsum layer.

[ RUN      ] Layer_Einsum_Test.Accuracy_01D/2, where GetParam() = ({}, { 1 }, ",i->i")
*** longjmp causes uninitialized stack frame ***: /build/5_x_noOCL-lin32/build/bin/opencv_test_dnn terminated
======= Backtrace: =========
/lib/i386-linux-gnu/libc.so.6(+0x67387)[0xf67aa387]

@asmorkalov asmorkalov merged commit 7e8f2a1 into opencv:5.x Aug 7, 2024
@asmorkalov
Copy link
Copy Markdown
Contributor Author

Reproduced with my jetson tk1 and submitted bug: #26008

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.