Skip to content

aruco: fix subpixel coordinates in interpolateCornersCharuco#3024

Merged
alalek merged 2 commits intoopencv:masterfrom
andy-held:charuco-selectAndRefineChessboardCorners
Sep 17, 2021
Merged

aruco: fix subpixel coordinates in interpolateCornersCharuco#3024
alalek merged 2 commits intoopencv:masterfrom
andy-held:charuco-selectAndRefineChessboardCorners

Conversation

@andy-held
Copy link
Copy Markdown
Contributor

@andy-held andy-held commented Aug 23, 2021 •
edited
Loading

The two possible paths of interpolateCornersCharuco, _interpolateCornersCharucoApproxCalib and _interpolateCornersCharucoLocalHom, assume that the top-left corner of the top-left pixel of an image has the coordinate (0,0) whereas for cornerSubpix that is the center of that pixel and the top-left corner is actually (-0.5, -0.5). To check this I rendered a ChArUco board in a way that has the board corners (not the marker corners) exactly at integer pixel coordinates. Both _interpolateCornersCharucoApproxCalib and _interpolateCornersCharucoLocalHom return exactly these coordinates already, however, cornerSubpix then optimizes that towards the top-left.

In the test interpolateCornersCharuco is once called with the cameraMatrix parameter, once without, so both paths (_interpolateCornersCharucoApproxCalib and _interpolateCornersCharucoLocalHom) are tested.

This seems to be an incarnation of undocumented subpixel coodinates, see opencv/opencv#10130

The test also needs the data contributed here: opencv/opencv_extra#900

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • There is reference to original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@andy-held andy-held mentioned this pull request Aug 23, 2021
Merged
@andy-held andy-held force-pushed the charuco-selectAndRefineChessboardCorners branch from d2b4669 to 1b53fa7 Compare August 24, 2021 07:31
@andy-held
Copy link
Copy Markdown
Contributor Author

andy-held commented Aug 24, 2021

I could just take corners & ids and discard the actual image, that would make this pull a little less complex.

@andy-held andy-held force-pushed the charuco-selectAndRefineChessboardCorners branch from 1b53fa7 to 942dbf3 Compare September 9, 2021 09:58
@andy-held andy-held force-pushed the charuco-selectAndRefineChessboardCorners branch from 942dbf3 to b56097f Compare September 9, 2021 11:00
@paroj
Copy link
Copy Markdown
Contributor

paroj commented Sep 9, 2021

one should probably note that _interpolateCornersCharucoApproxCalib generates candidates using projectPoints.

projectPoints and cornerSubPix disagree on subpixel coordinates

it would help, if you could mention what the disagreement actually is.

also, did you check, whether this affects the _interpolateCornersCharucoLocalHom path?

@andy-held
Copy link
Copy Markdown
Contributor Author

andy-held commented Sep 9, 2021 •
edited
Loading

_interpolateCornersCharucoApproxCalib and _interpolateCornersCharucoLocalHom assume that the top-left corner of the top-left pixel of an image has the coordinate (0,0) whereas for cornerSubpix that is the center of that pixel and the top-left corner is actually (-0.5, -0.5). To check this I rendered a ChArUco board in a way that has the board corners (not the marker corners) exactly at integer pixel coordinates. Both _interpolateCornersCharucoApproxCalib and _interpolateCornersCharucoLocalHom return exactly these coordinates already, however, cornerSubpix then optimizes that towards the top-left.

In the test interpolateCornersCharuco is once called with the cameraMatrix parameter, once without, so both paths (_interpolateCornersCharucoApproxCalib and _interpolateCornersCharucoLocalHom) are tested.

@alalek alalek merged commit a1fe9c8 into opencv:master Sep 17, 2021
@alalek alalek mentioned this pull request Oct 15, 2021
Merged
@andy-held andy-held mentioned this pull request Jul 22, 2022
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@paroj paroj paroj approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@paroj paroj

Labels

category: aruco

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@andy-held @paroj @alalek