Skip to content

Fix getPerspectiveTransform for singular case#26926

Merged
asmorkalov merged 5 commits intoopencv:4.xfrom
MaximSmolskiy:fix-getPerspectiveTransform-for-singular-case
Mar 2, 2025
Merged

Fix getPerspectiveTransform for singular case#26926
asmorkalov merged 5 commits intoopencv:4.xfrom
MaximSmolskiy:fix-getPerspectiveTransform-for-singular-case

Conversation

@MaximSmolskiy
Copy link
Copy Markdown
Contributor

Pull Request Readiness Checklist

Fix #26916

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@MaximSmolskiy MaximSmolskiy marked this pull request as draft February 16, 2025 23:55
@asmorkalov asmorkalov added this to the 4.12.0 milestone Feb 17, 2025
@asmorkalov
Copy link
Copy Markdown
Contributor

The fix overrides user selection of solver and it's not obvious behaviour. Alternatives I see:

  • Add recommendation to documentation how to check the function result (estimate error) and add recommendations.
  • Change default method value to make the step-by-step approach more obvious.

Let me talk to colleagues and I'll return back.

@MaximSmolskiy
Copy link
Copy Markdown
Contributor Author

The fix overrides user selection of solver and it's not obvious behaviour. Alternatives I see:

* Add recommendation to documentation how to check the function result (estimate error) and add recommendations.

* Change default method value to make the step-by-step approach more obvious.

Let me talk to colleagues and I'll return back.

@asmorkalov This isn't correct fix (because of that I moved PR to draft state). Problem isn't about solver. I guess, that it is a fundamental problem. In getPerspectiveTransform we search perspective transform only with h_33 != 0 (because use h_33 = 1 normalization), but for several cases (including issue case) solution should have h_33 = 0, because there is no proper solution with h_33 != 0. In issue reproducer correct inv_M* (obtained by inverse and findHomography) matrices have h_33 = 0.

I will try to prepare a correct fix in the near future

@MaximSmolskiy MaximSmolskiy marked this pull request as ready for review February 17, 2025 18:50
@MaximSmolskiy
Copy link
Copy Markdown
Contributor Author

@asmorkalov In

* \anchor lecture_16 1. [Lecture 16: Planar Homographies](http://www.cse.psu.edu/~rtc12/CSE486/lecture16.pdf), Robert Collins
on slide 33:

A More General Approach

What might be wrong with setting h33 = 1?

If h33 actually = 0, we can’t get the right answer.

So, that's exactly the problem. We can't limit ourselves only with h33 = 1 normalization case where h33 != 0. In case of failure we must consider h11^2 + h12^2 + h13^2 + h21^2 + h22^2 + h23^2 + h31^2 + h32^2 + h33^2 = 1 normalization case where h33 can be zero.

For implementation of new normalization case I used last 3 slides of the same source

@MaximSmolskiy
Copy link
Copy Markdown
Contributor Author

@asmorkalov Do you have any ideas why the error on macOS or ARM64 is significantly larger?

/Users/opencv-cn/GHA-OCV-2/_work/opencv/opencv/opencv/modules/imgproc/test/test_imgwarp.cpp:1796: Failure
Expected: (cvtest::ocl::TestUtils::checkNorm2(obtained_homogeneous_dst_points, expected_homogeneous_dst_points)) <= (1e-8), actual: 0.00404862 vs 1e-08

/home/ci/opencv/modules/imgproc/test/test_imgwarp.cpp:1796: Failure
Expected: (cvtest::ocl::TestUtils::checkNorm2(obtained_homogeneous_dst_points, expected_homogeneous_dst_points)) <= (1e-8), actual: 0.0620923 vs 1e-08

@MaximSmolskiy
Copy link
Copy Markdown
Contributor Author

Tried to use double in test - to increase precision

@asmorkalov
Copy link
Copy Markdown
Contributor

@MaximSmolskiy Friendly reminder.

@MaximSmolskiy
Copy link
Copy Markdown
Contributor Author

@asmorkalov Do you have any ideas why the error on macOS or ARM64 is significantly larger?

/Users/opencv-cn/GHA-OCV-2/_work/opencv/opencv/opencv/modules/imgproc/test/test_imgwarp.cpp:1796: Failure
Expected: (cvtest::ocl::TestUtils::checkNorm2(obtained_homogeneous_dst_points, expected_homogeneous_dst_points)) <= (1e-8), actual: 0.00404862 vs 1e-08

/home/ci/opencv/modules/imgproc/test/test_imgwarp.cpp:1796: Failure
Expected: (cvtest::ocl::TestUtils::checkNorm2(obtained_homogeneous_dst_points, expected_homogeneous_dst_points)) <= (1e-8), actual: 0.0620923 vs 1e-08

@asmorkalov I found that the significant difference in accuracy was due to the cv::eigen function (which depends on the Eigen library being included):

  1. without Eigen - good accuracy
  2. with Eigen - bad accuracy

I used cv::SVDecomp instead of cv::eigen to improve accuracy and get rid of this rather unexpected behavior

@asmorkalov asmorkalov merged commit dbd3ef9 into opencv:4.x Mar 2, 2025
28 checks passed
@MaximSmolskiy MaximSmolskiy deleted the fix-getPerspectiveTransform-for-singular-case branch March 2, 2025 09:47
@asmorkalov asmorkalov mentioned this pull request Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

getPerspectiveTransform produces wrong output on certain data

3 participants