Skip to content

Properly handle empty centers in findCirclesGrid#19498

Merged
alalek merged 3 commits intoopencv:3.4from
mattalvarado:fix_findcirclesgrid
Feb 16, 2021
Merged

Properly handle empty centers in findCirclesGrid#19498
alalek merged 3 commits intoopencv:3.4from
mattalvarado:fix_findcirclesgrid

Conversation

@mattalvarado
Copy link
Copy Markdown
Contributor

@mattalvarado mattalvarado commented Feb 11, 2021

A recent refactor of findCirclesGrid introduced a bug which does not properly handle the case when no centers are detected. This resolves that issue by explicitly checking if the centers result is valid before attempting to transform them.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • There is reference to original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@asmorkalov asmorkalov self-requested a review February 11, 2021 08:01
@asmorkalov asmorkalov added the pr: needs rebase Rebase patch (and squash fixup commits) on the top of target branch label Feb 11, 2021
@asmorkalov
Copy link
Copy Markdown
Contributor

@mattalvarado Thanks for the contribution! The same code exists in branch 3.4 and it looks like the same bug exists there too. Please re-base the patch to 3.4 and re-target it to 3.4 branch. 3.4 is merged to master weekly or biweekly and will be applied to master too. Also it'll some test is required to proof the solution.

@asmorkalov asmorkalov added the pr: needs test New functionality requires minimal tests set label Feb 11, 2021
}

if (!H.empty()) // undone rectification
if (isValid && !H.empty()) // undone rectification
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

relates #19079

The problem is that original code returns transformed (rectified) corners for the "false" result too.
Why do we need to return these rectified corners on detection failure?

Please provide more details about observed issue. What error message did you get?

BTW, you can see this corners problem by launching calib3d test with and without your patch:

./bin/opencv_test_calib3d --gtest_filter=*regression_18713* --test_debug=10

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @alalek, Yep #19079 introduced the issue.

Since the rectification code was removed from the isFound conditional in the main attempts for loop, we need to add back in that check before attempting to rectify the centers.

The error occurs when the attempts loop reaches its last attempt. If no centers are found on the last attempt, but centers were found on one of the previous attempts, H will be valid but centers will be empty. The final !H.empty() conditional will then attempt to transform a empty container of centers which will trigger the following assertion:

opencv/modules/core/src/matmul.dispatch.cpp:439: error: (-215:Assertion failed) scn == m.cols || scn + 1 == m.cols in function 'transform'

I attempted to preserve the original function interface, and figured checking if the result was valid before rectifying the centers was the best way to do that.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps I got this. Probably we should check for corners.empty() instead.

Please capture points values (std::cout << Mat(points) << std::endl) which cause this error, so we can add regression test for that.
(add patternSize / flags arguments values too)

Copy link
Copy Markdown
Contributor Author

@mattalvarado mattalvarado Feb 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debug

I figured isValid was equivalent to !centers.empty(), but I am happy to change this to check the centers directly. I attached the image which failed for reference. I believe the partial circle grid is what throws the logic off.

I am new to the unit testing framework for OpenCV. What is the best way to add the test for detectors like this? Should I hardcode the image contents in the test?

The patternSize is used was 13x8 and I used the default args for flags and blobDetector. Those are CALIB_CB_SYMMETRIC_GRID and SimpleBlobDetector::create() respectively

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just took a look at the regression_18713 test. I port this issue to a similar test

@mattalvarado
Copy link
Copy Markdown
Contributor Author

@mattalvarado Thanks for the contribution! The same code exists in branch 3.4 and it looks like the same bug exists there too. Please re-base the patch to 3.4 and re-target it to 3.4 branch. 3.4 is merged to master weekly or biweekly and will be applied to master too. Also it'll some test is required to proof the solution.

@asmorkalov Happy to help. Which 3.4 branch should I rebase this onto?

@asmorkalov
Copy link
Copy Markdown
Contributor

3.4 is branch name, not particular version.

@alalek
Copy link
Copy Markdown
Member

alalek commented Feb 12, 2021

This patch should go into 3.4 branch first.
We will merge changes from 3.4 into master regularly (weekly/bi-weekly).

Please:

  • change "base" branch of this PR: master => 3.4 (use "Edit" button near PR title)
  • rebase your commits from master onto 3.4 branch. For example:
    git rebase -i --onto upstream/3.4 upstream/master
    (check list of your commits, save and quit (Esc + "wq" + Enter)
    where upstream is configured by following this GitHub guide and fetched (git fetch upstream).
  • push rebased commits into source branch of your fork (with --force option)

Note: no needs to re-open PR, apply changes "inplace".

@mattalvarado mattalvarado changed the base branch from master to 3.4 February 16, 2021 07:34
@mattalvarado
Copy link
Copy Markdown
Contributor Author

@alalek @asmorkalov Apologies for the confusion on 3.4. I was accidentally looking at tags and not branches. I added a test for this regression, and should have rebased this onto 3.4. Please let me know if I need to make any other changes.

@asmorkalov asmorkalov removed pr: needs rebase Rebase patch (and squash fixup commits) on the top of target branch pr: needs test New functionality requires minimal tests set labels Feb 16, 2021
Copy link
Copy Markdown
Member

@alalek alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done! Thank you for contribution 👍

@mattalvarado
Copy link
Copy Markdown
Contributor Author

Well done! Thank you for contribution

Happy to help! Thanks for your patience.

@alalek alalek merged commit 0a7a54f into opencv:3.4 Feb 16, 2021
@alalek alalek mentioned this pull request Apr 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

findCirclesGrid() Assertion failed in function 'transform'

3 participants