Compensate edge length in ChessBoardDetector::generateQuads by MaximSmolskiy · Pull Request #24833 · opencv/opencv

MaximSmolskiy · 2024-01-08T20:41:27Z

Pull Request Readiness Checklist

Should be merged after #24710

Idea was mentioned in Section III-C. Adaptive Quadrangle Linking Distance of Rufli, Martin & Scaramuzza, Davide & Siegwart, Roland. (2008). Automatic Detection of Checkerboards on Blurred and Distorted Images. 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS. 3121-3126. 10.1109/IROS.2008.4650703 (https://rpg.ifi.uzh.ch/docs/IROS08_scaramuzza_b.pdf):

As mentioned in Section IIA.5, quadrangles only get linked if their corners are less than a certain distance apart. In the original implementation, inaccurately, the shortest edge length of the two involved quadrangles was chosen for this distance limit. If the checkers are large w.r.t erosion, the error introduced is small. But for low resolution images, erosion has a large effect on the overall size of the quadrangle, which may result in a drastic reduction of the smallest edge length. Therefore the distance measure was adapted to incorporate the effect of erosion:
d_limit = shortest_edge_length + 2 · erosion, (1)
where the factor two is due to the erosion acting on both quadrangles

I increased edge length by 2 * dilations because each dilation with 3 x 3 rectangular structuring element as kernel in average case decrease edge length by one pixel but from each of two ends, so total edge length decrease is equal to 2 * dilations

I tested this change (on 3.2.0 code version with some modern patches) on two private datasets and it increased number of found chessboards:

from 262 to 306 ~ 16.8%
from 392 to 536 ~ 36.7%

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

AleksandrPanov

I tested PR with benchmark:
--configuration=generate_run --board_x=7 --path=res_chessboard --synthetic_object=chessboard
before PR fix:

after PR fix:

after PR fix + #24710 PR

There are no regressions, but I was unable to increase the number of chess boards found.
@MaximSmolskiy, is it possible for you to add a test?

What distortions did you fix the improvement on?
Is it a strong distortion of perspective or a strong blur or low resolution or all together?

What distortions should I add to the benchmark?

asmorkalov

👍

MaximSmolskiy · 2024-02-15T12:38:04Z

Thanks for merge!
I will give some comments about situations/examples when this PR increase number of chessboards a little bit later

asmorkalov · 2024-02-15T13:46:41Z

Looks like the patch affects test Calib3d_StereoCalibrate_CPP. It randomly fails in some configurations on CI:

[ RUN      ] Calib3d_StereoCalibrate_CPP.regression
/home/alexander/Projects/OpenCV/opencv-master/modules/ts/src/ts.cpp:618: Failure
Failed

        failure reason: Bad accuracy
        test case #-1
        seed: 0000000000000000
-----------------------------------
        LOG:
The distance between y coordinates is too big(=4.35913) (used uncalibrated stereo), testcase 1

-----------------------------------

[  FAILED  ] Calib3d_StereoCalibrate_CPP.regression (1113 ms)

…pencv#24833

Reverted Compensate edge length in ChessBoardDetector::generateQuads #24833

…n-ChessBoardDetector-generateQuads-attempt-2 Compensate edge length in ChessBoardDetector::generateQuads (attempt 2) #25090 ### Pull Request Readiness Checklist New attempt for #24833, which was reverted as #25036. Locally I fixed `Calib3d_StereoCalibrate_CPP.regression` test by corners refinement using `cornerSubPix` function See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake

MaximSmolskiy · 2024-03-03T13:10:07Z

Thanks for merge! I will give some comments about situations/examples when this PR increase number of chessboards a little bit later

@AleksandrPanov I'm sorry for delay. This PR mainly affects chessboards with fairly small cell sizes, so I tried to run benchmark with decreased cell_img_size (10 instead of default 100) and there were differences in benchmark results.

Commit 6f48cb78b60fbb8ccb4531b31a260b7ee60d00db

python3 objdetect_benchmark.py --configuration=generate_run --board_x=7 --path=res_chessboard --synthetic_object=chessboard --cell_img_size 10

                                 category  detected chessboard  total detected chessboard  total chessboard  average detected error chessboard
                          _none_none_blur             0.988889                        356               360                           4.903708
                    _none_none_gaussNoise             0.666667                        240               360                           5.359727
                          _none_none_none             1.000000                        360               360                           4.943304
                    _none_none_strongBlur             0.833333                        300               360                           3.810527
                   _none_undistorted_blur             0.911111                        328               360                           5.281945
             _none_undistorted_gaussNoise             0.913889                        329               360                           4.673793
                   _none_undistorted_none             0.997222                        359               360                           4.340208
             _none_undistorted_strongBlur             0.161111                         58               360                           5.029701
                   _perspective_none_blur             0.520370                        562              1080                           4.894851
             _perspective_none_gaussNoise             0.966667                       1044              1080                           3.895454
                   _perspective_none_none             0.971296                       1049              1080                           3.920378
             _perspective_none_strongBlur             0.000000                          0              1080                                NaN
            _perspective_undistorted_blur             0.611111                        660              1080                           4.509053
      _perspective_undistorted_gaussNoise             0.999074                       1079              1080                           3.550209
            _perspective_undistorted_none             0.777778                        840              1080                           3.675595
      _perspective_undistorted_strongBlur             0.000000                          0              1080                                NaN
             _strongPerspective_none_blur             0.000000                          0               360                                NaN
       _strongPerspective_none_gaussNoise             0.000000                          0               360                                NaN
             _strongPerspective_none_none             0.166667                         60               360                           2.198370
       _strongPerspective_none_strongBlur             0.000000                          0               360                                NaN
      _strongPerspective_undistorted_blur             0.000000                          0               360                                NaN
_strongPerspective_undistorted_gaussNoise             0.000000                          0               360                                NaN
      _strongPerspective_undistorted_none             0.000000                          0               360                                NaN
_strongPerspective_undistorted_strongBlur             0.000000                          0               360                                NaN
                                      all             0.529444                       7624             14400                           4.201352

Commit 853f7ff904b77f234aeac1a0edc48dc6756738a0

python3 objdetect_benchmark.py --configuration=generate_run --board_x=7 --path=res_chessboard --synthetic_object=chessboard --cell_img_size 10

                                 category  detected chessboard  total detected chessboard  total chessboard  average detected error chessboard
                          _none_none_blur             0.991667                        357               360                           4.905091
                    _none_none_gaussNoise             0.750000                        270               360                           5.215633
                          _none_none_none             1.000000                        360               360                           4.943304
                    _none_none_strongBlur             0.916667                        330               360                           3.806217
                   _none_undistorted_blur             0.994444                        358               360                           5.220915
             _none_undistorted_gaussNoise             0.997222                        359               360                           4.542443
                   _none_undistorted_none             0.997222                        359               360                           4.340208
             _none_undistorted_strongBlur             0.161111                         58               360                           5.024330
                   _perspective_none_blur             0.629630                        680              1080                           4.815848
             _perspective_none_gaussNoise             0.966667                       1044              1080                           3.895454
                   _perspective_none_none             0.971296                       1049              1080                           3.920378
             _perspective_none_strongBlur             0.000000                          0              1080                                NaN
            _perspective_undistorted_blur             0.583333                        630              1080                           4.593498
      _perspective_undistorted_gaussNoise             0.999074                       1079              1080                           3.545573
            _perspective_undistorted_none             0.750000                        810              1080                           3.605107
      _perspective_undistorted_strongBlur             0.000000                          0              1080                                NaN
             _strongPerspective_none_blur             0.000000                          0               360                                NaN
       _strongPerspective_none_gaussNoise             0.000000                          0               360                                NaN
             _strongPerspective_none_none             0.166667                         60               360                           2.198370
       _strongPerspective_none_strongBlur             0.000000                          0               360                                NaN
      _strongPerspective_undistorted_blur             0.000000                          0               360                                NaN
_strongPerspective_undistorted_gaussNoise             0.000000                          0               360                                NaN
      _strongPerspective_undistorted_none             0.000000                          0               360                                NaN
_strongPerspective_undistorted_strongBlur             0.000000                          0               360                                NaN
                                      all             0.541875                       7803             14400                           4.199676

Differences in detected chessboard:

              _none_none_blur: 0.988889 -> 0.991667
        _none_none_gaussNoise: 0.666667 -> 0.750000
        _none_none_strongBlur: 0.833333 -> 0.916667
       _none_undistorted_blur: 0.911111 -> 0.994444
 _none_undistorted_gaussNoise: 0.913889 -> 0.997222
       _perspective_none_blur: 0.520370 -> 0.629630
_perspective_undistorted_blur: 0.611111 -> 0.583333
_perspective_undistorted_none: 0.777778 -> 0.750000

all: 0.529444 -> 0.541875

Differences in detected chessboard for all category:

cell_img_size = 10: 0.529444 -> 0.541875
cell_img_size =  9: 0.477292 -> 0.512292
cell_img_size =  8: 0.412917 -> 0.474444
cell_img_size =  7: 0.324236 -> 0.398889
cell_img_size =  6: 0.197778 -> 0.268472
cell_img_size =  5: 0.118750 -> 0.137500

…ngth-in-ChessBoardDetector-generateQuads-attempt-2 Compensate edge length in ChessBoardDetector::generateQuads (attempt 2) opencv#25090 ### Pull Request Readiness Checklist New attempt for opencv#24833, which was reverted as opencv#25036. Locally I fixed `Calib3d_StereoCalibrate_CPP.regression` test by corners refinement using `cornerSubPix` function See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake

Compensate edge length in ChessBoardDetector::generateQuads

3640217

asmorkalov added the category: calib3d label Jan 9, 2024

asmorkalov added this to the 4.10.0 milestone Jan 9, 2024

asmorkalov requested a review from AleksandrPanov February 6, 2024 12:28

AleksandrPanov requested changes Feb 8, 2024

View reviewed changes

asmorkalov self-requested a review February 15, 2024 10:21

asmorkalov approved these changes Feb 15, 2024

View reviewed changes

asmorkalov merged commit a9f15d7 into opencv:4.x Feb 15, 2024

MaximSmolskiy deleted the compensate-edge-length-in-ChessBoardDetector-generateQuads branch February 15, 2024 11:07

asmorkalov self-assigned this Feb 15, 2024

asmorkalov added a commit to asmorkalov/opencv that referenced this pull request Feb 16, 2024

Reverted Compensate edge length in ChessBoardDetector::generateQuads o…

78a765c

…pencv#24833

asmorkalov mentioned this pull request Feb 16, 2024

Reverted Compensate edge length in ChessBoardDetector::generateQuads #24833 #25036

Merged

6 tasks

asmorkalov added a commit that referenced this pull request Feb 16, 2024

Merge pull request #25036 from asmorkalov:as/revert_#24833

835d6dd

Reverted Compensate edge length in ChessBoardDetector::generateQuads #24833

asmorkalov mentioned this pull request Feb 16, 2024

(5.x) Merge 4.x #25041

Merged

MaximSmolskiy mentioned this pull request Feb 25, 2024

Compensate edge length in ChessBoardDetector::generateQuads (attempt 2) #25090

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Compensate edge length in ChessBoardDetector::generateQuads#24833

Compensate edge length in ChessBoardDetector::generateQuads#24833
asmorkalov merged 1 commit intoopencv:4.xfrom
MaximSmolskiy:compensate-edge-length-in-ChessBoardDetector-generateQuads

MaximSmolskiy commented Jan 8, 2024

Uh oh!

AleksandrPanov left a comment •

edited

Loading

Uh oh!

asmorkalov left a comment

Uh oh!

MaximSmolskiy commented Feb 15, 2024

Uh oh!

asmorkalov commented Feb 15, 2024

Uh oh!

MaximSmolskiy commented Mar 3, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

MaximSmolskiy commented Jan 8, 2024

Pull Request Readiness Checklist

Uh oh!

AleksandrPanov left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asmorkalov left a comment

Choose a reason for hiding this comment

Uh oh!

MaximSmolskiy commented Feb 15, 2024

Uh oh!

asmorkalov commented Feb 15, 2024

Uh oh!

MaximSmolskiy commented Mar 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AleksandrPanov left a comment •

edited

Loading

MaximSmolskiy commented Mar 3, 2024 •

edited

Loading