findChessboardCornersSB: speed improvements by D-Alex · Pull Request #12615 · opencv/opencv

D-Alex · 2018-09-22T21:46:16Z

findChessboardCornersSB: speed up using parallel_for and add support for flags

force_builders=linux,windows,macosx,docs

catree · 2018-09-23T11:24:00Z

modules/calib3d/include/opencv2/calib3d.hpp

-@param flags operation flags for future improvements
+@param flags Various operation flags that can be zero or a combination of the following values:
+-   **CALIB_CB_NORMALIZE_IMAGE** Normalize the image gamma with equalizeHist before detection.
+-   **CALIB_CB_EXAUSTING ** Run an exhausting search to improve detection rate.


Maybe change to CALIB_CB_EXHAUSTIVE?
I understand this option like a brute-force approach. In this case, exhaustive search looks more appropriate in my opinion.

This also fixes a typo

alalek · 2018-09-23T13:34:30Z

modules/calib3d/src/chessboard.cpp

+        flags ^= CALIB_CB_ACCURACY;
    }
+    if(flags)
+        CV_Error(Error::StsOutOfRange, "Invalid remaing flags " + std::to_string(flags));


GCC 4.8 (Android builder) has bad support for std::to_string().

Please use this code:

CV_Error(Error::StsOutOfRange, cv::format("Invalid remaing flags %d", (int)flags));

alalek · 2018-09-23T13:34:33Z

modules/calib3d/src/chessboard.cpp

            }
+        });
+        // check if a good board was found
+        for(auto board : boards)


const auto& ?

alalek

Well done! Thank you 👍

tompollok · 2018-09-26T12:48:12Z

@D-Alex did you check the chessboard detection rate of findChessboardCorners vs findChessboardCornersSB ?

I tested both functions (opencv4.0.0 alpha and not master) and findChessboardCorners seems to outperform findChessboardCornersSB in terms of chessboard detection rate.

Check out this video: https://www.youtube.com/watch?v=ImpxJSIbBdk

I played the sequence twice. first with findChessboardCorners (red) and then again with findChessboardCorners SB (green)

I think about 30% of the chessboard detections are missing. I admit that shirt is not necessarily the best for doing calibrations, but findChessboardCorners had no problems with that.

tompollok · 2018-09-26T13:01:11Z

I tested it with another sequence where i wear a neutral shirt and there i have the same or worse problem. In the conditions of the image below i have zero chessboard detections with or without cv::CALIB_CB_NORMALIZE_IMAGE being set.

D-Alex · 2018-09-26T13:21:17Z

Thx for reporting problems with the detection rate and I am sure there is room for improvements. Have you tested it against the current version merged yesterday?

I am asking because I changed MIN_RESPONSE_RATIO from 0.3F to= 0.1F which was added to speed up detections. However, this filters out points which are less than 30% of the strongest key points. And this is exactly the case in your images.

I tested your last image and it seems to work with the current version if CALIB_CB_NORMALIZE_IMAGE is added as flag. Attached is the repsonse map of the detector which looks very healthy.

tompollok · 2018-09-26T13:35:15Z

Thank you for your fast response. As mentioned i have only tested against 4.0.0-alpha. Do you know if it is possible to download master build artifacts for windows from the opencv buildservers?

alalek · 2018-09-26T13:43:08Z

download master build artifacts

We don't provide such builds on public.

D-Alex · 2018-09-26T13:50:48Z

findChessboardCornersSB part of the alpha build is not really aware of the flags. This was added yesterday. Therefore, to work around the issue you can do the normalisation by yourself calling equalizeHist before calling findChessboardCornersSB.

In addition you can provide 1 as flag to increase the number of tests. However, the alpha version does not take advantage of multiple CPUs for around half of the code base. Also you still have the hard coded 30% threshold which might be ok if you normalise your images.

tompollok · 2018-09-26T13:53:43Z

Alright, thank you @D-Alex and @alalek for your replies.

tompollok · 2018-09-26T14:04:04Z

cv::equalizeHist(mat, mat); before findChessboardCornersSB(...) improved the detection from 0% to about 50% for the sequence of the image of which you made a responsemap for me. So i guess ill have to recompile master with a lower MIN_RESPONSE_RATIO.

providing 1 as a flag for findChessboardCornersSB flags parameter crashes my application. Probably some assertion, but i didnt investigate.

D-Alex · 2018-09-26T14:10:59Z

Yep - the assertion was also fixed when the flags were implemented. I forgot about it. If you upload your sequence somewhere I can run it with the current version. Otherwise you would have to compile it using the current master.

tompollok · 2018-09-26T15:54:54Z

I compiled master and it still crashes with 1 (CALIB_CB_ADAPTIVE_THRESH)

OpenCV(4.0.0-pre) C:\3rdparty\ocv4_master\opencv\modules\calib3d\src\chessboard.cpp:3211: error: (-211:One of arguments' values is out of range) Invalid remaing flags 1 in function 'cv::findChessboardCornersSB'

D-Alex · 2018-09-26T16:09:19Z

Like stated in the header (I know it is hard to read when not compiled ;-)) it only supports:

CALIB_CB_NORMALIZE_IMAGE = 2
CALIB_CB_EXHAUSTING = 16,
CALIB_CB_ACCURACY = 32

tompollok · 2018-09-26T16:12:07Z

I have seen the documentation but you mentioned "In addition you can provide 1 as flag to increase the number of tests." before. I also took a look at the code now and there i clearly see that 1 is not supported :-)

tompollok · 2018-09-26T16:14:22Z

i guess you meant 16 instead of 1

tompollok · 2018-09-26T16:29:27Z

flags = cv::CALIB_CB_NORMALIZE_IMAGE | cv::CALIB_CB_EXHAUSTIVE | cv::CALIB_CB_ACCURACY;

With flags as above the detection rate is much better but i still have many "easy" chessboards that aren't detected. If you want, i can provide you some of them.

Another question: What did you use to enable parallel_for_?

According to the link below a few options are available. Did you just enable openMP?

https://docs.opencv.org/trunk/d7/dff/tutorial_how_to_use_OpenCV_parallel_for_.html

The first precondition is to have OpenCV built with a parallel framework. In OpenCV 3.2, the following parallel frameworks are available in that order:

1. Intel Threading Building Blocks (3rdparty library, should be explicitly enabled)
2. C= Parallel C/C++ Programming Language Extension (3rdparty library, should be explicitly enabled)
3. OpenMP (integrated to compiler, should be explicitly enabled)
4. APPLE GCD (system wide, used automatically (APPLE only))
5. Windows RT concurrency (system wide, used automatically (Windows RT only))
6. Windows concurrency (part of runtime, used automatically (Windows only - MSVC++ >= 10))
7. Pthreads (if available)

D-Alex · 2018-09-26T16:35:03Z

Sure - I am happy to have a look why certain checkerboards are not detected. I am using a Mac therefore I would guess option 4.

tompollok · 2018-09-26T16:49:32Z

Alright, i will send you some images during the next week.

With findChessboardCorners I used cornerSubPix to refine the corner locations. Am I assuming correctly that cornerSubPix would only degrade the corner locations from your algorithm?

* master: (286 commits) Merge pull request opencv#12608 from dmatveev:gapi M_PI changed to CV_PI (opencv#12645) dnn: fix printf format warning Merge pull request opencv#12615 from D-Alex:master Fixed several incorrect printf format specifiers core: fix printf warnings by using c++11 format core: enable printf format warnings for cv::format JS: Support enum properties fix a bug in OpenGL samples: update winpack python samples launcher Merge pull request opencv#12310 from cv3d:chunks/enum_interface Merge pull request opencv#12601 from cv3d:fix/js release: OpenCV 4.0.0-alpha (version++) cuda: move CUDA modules to opencv_contrib cmake: update install paths (Linux) Merge pull request opencv#12570 from alalek:drop_usrtype1 Fix failure to request stddev of non-intrinsics ts: update valgrind test filter build: fix Xcode 10 build problems Enable Myriad device for OpenVINO models test ...

findChessboardCornersSB: speed improvements (opencv#12615) * chessboard: fix do not modify const image * chessboard: speed up scale space using parallel_for * chessboard: small improvements * chessboard: speed up board growing using parallel_for * chessboard: add flags for tuning detection * chessboard: fix compiler warnings * chessborad: change flag name to CALIB_CB_EXHAUSTIVE This also fixes a typo * chessboard: fix const ref + remove to_string

D-Alex added 6 commits September 22, 2018 22:55

chessboard: fix do not modify const image

a2e73eb

chessboard: speed up scale space using parallel_for

6bb2f26

chessboard: small improvements

d473ec3

chessboard: speed up board growing using parallel_for

d5cbfd4

chessboard: add flags for tuning detection

934e721

chessboard: fix compiler warnings

cb81145

catree reviewed Sep 23, 2018

View reviewed changes

chessborad: change flag name to CALIB_CB_EXHAUSTIVE

ea44ddf

This also fixes a typo

alalek reviewed Sep 23, 2018

View reviewed changes

chessboard: fix const ref + remove to_string

3fe6c57

alalek approved these changes Sep 25, 2018

View reviewed changes

alalek merged commit 8811dab into opencv:master Sep 25, 2018

Uh oh!

Conversation

D-Alex commented Sep 22, 2018 • edited by alalek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

catree Sep 23, 2018

Choose a reason for hiding this comment

Uh oh!

alalek Sep 23, 2018

Choose a reason for hiding this comment

Uh oh!

alalek Sep 23, 2018

Choose a reason for hiding this comment

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

tompollok commented Sep 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tompollok commented Sep 26, 2018

Uh oh!

D-Alex commented Sep 26, 2018

Uh oh!

tompollok commented Sep 26, 2018

Uh oh!

alalek commented Sep 26, 2018

Uh oh!

D-Alex commented Sep 26, 2018

Uh oh!

tompollok commented Sep 26, 2018

Uh oh!

tompollok commented Sep 26, 2018

Uh oh!

D-Alex commented Sep 26, 2018

Uh oh!

tompollok commented Sep 26, 2018

Uh oh!

D-Alex commented Sep 26, 2018

Uh oh!

tompollok commented Sep 26, 2018

Uh oh!

tompollok commented Sep 26, 2018

Uh oh!

tompollok commented Sep 26, 2018

Uh oh!

D-Alex commented Sep 26, 2018

Uh oh!

tompollok commented Sep 26, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

D-Alex commented Sep 22, 2018 •

edited by alalek

Loading

tompollok commented Sep 26, 2018 •

edited

Loading