[GSoC] robust function for affine transform estimation by hrnr · Pull Request #6615 · opencv/opencv

hrnr · 2016-06-01T08:13:39Z

implements estimateAffine2D. estimates affine transformation using robust RANSAC method.

uses RANSAC framework in calib3d
includes accuracy test
uses SVD decomposition for solving 3 point equation

things to consider:

is solving through SVD fast enough? should we make it symmetric and use EIGEN val decomposition?
is checking for coplanar points robust enough? (it seems so)

I plan to write perf test to see if make sense to do some tricks and solve with EIGEN val decomp. Checking for coplanar points seems to work well for me. I adapted the check from estimateAffine3D, estimateRigidTransform have some addition checks (checking if points are not too close), but I'm not sure if it is worth to implement those.

I will implement also estimateAffinePartial2D that will estimate transformation using only rotation, translation an scaling (similar to estimateRigidTransform). I'm only not sure about the name estimateAffinePartial2D, any ideas welcome.

cc: @prclibo

Save method should save, instead trying to run inkview

hrnr · 2016-06-01T08:59:40Z

I don't have any idea why this fails OpenCL perf test. It does not make any sense to me. Is that some bug in OCL_PyrLKOpticalFlowFixture_PyrLKOpticalFlow.PyrLKOpticalFlow?

hrnr · 2016-06-01T13:48:58Z

added estimateAffinePartial2D and rebased.

prclibo · 2016-06-01T14:35:18Z

modules/calib3d/include/opencv2/calib3d.hpp

+between 0.95 and 0.99 is usually good enough. Values too close to 1 can slow down the estimation
+significantly. Values lower than 0.8-0.9 can result in an incorrectly estimated transformation.
+
+The function estimates an optimal 2D affine transformation with 5 degrees of freedom limited to


do you mean 4 dof? 1 rotation + 1 scale + 2 translation?

yes. I will fix it.

hrnr · 2016-06-01T14:48:02Z

added perf test for both functions. Here are my results:

Time compensation is 0
CTEST_FULL_OUTPUT
OpenCV version: 3.1.0-dev
OpenCV VCS version: 3.1.0-663-g9e32c5c
Build type: release
Parallel framework: pthreads
CPU features: mmx sse sse2 sse3
OpenCL is disabled
Note: Google Test filter = *Estimate*
[==========] Running 18 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 9 tests from EstimateAffine_EstimateAffine2D
[ RUN      ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/0
[ PERFSTAT ]    (samples = 10, mean = 2.93, median = 2.91, stddev = 0.07 (2.5%))
[ VALUE    ]    (5000, 0.99)
[       OK ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/0 (30 ms)
[ RUN      ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/1
[ PERFSTAT ]    (samples = 10, mean = 1.87, median = 1.87, stddev = 0.02 (0.9%))
[ VALUE    ]    (5000, 0.95)
[       OK ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/1 (19 ms)
[ RUN      ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/2
[ PERFSTAT ]    (samples = 10, mean = 1.46, median = 1.45, stddev = 0.02 (1.2%))
[ VALUE    ]    (5000, 0.9)
[       OK ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/2 (14 ms)
[ RUN      ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/3
[ PERFSTAT ]    (samples = 10, mean = 0.73, median = 0.72, stddev = 0.01 (1.5%))
[ VALUE    ]    (1000, 0.99)
[       OK ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/3 (8 ms)
[ RUN      ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/4
[ PERFSTAT ]    (samples = 10, mean = 0.48, median = 0.47, stddev = 0.01 (2.9%))
[ VALUE    ]    (1000, 0.95)
[       OK ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/4 (5 ms)
[ RUN      ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/5
[ PERFSTAT ]    (samples = 10, mean = 0.41, median = 0.41, stddev = 0.01 (2.4%))
[ VALUE    ]    (1000, 0.9)
[       OK ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/5 (4 ms)
[ RUN      ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/6
[ PERFSTAT ]    (samples = 13, mean = 0.24, median = 0.24, stddev = 0.01 (2.1%))
[ VALUE    ]    (100, 0.99)
[       OK ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/6 (3 ms)
[ RUN      ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/7
[ PERFSTAT ]    (samples = 17, mean = 0.16, median = 0.15, stddev = 0.00 (3.0%))
[ VALUE    ]    (100, 0.95)
[       OK ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/7 (3 ms)
[ RUN      ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/8
[ PERFSTAT ]    (samples = 13, mean = 0.13, median = 0.13, stddev = 0.00 (1.2%))
[ VALUE    ]    (100, 0.9)
[       OK ] EstimateAffine_EstimateAffine2D.EstimateAffine2D/8 (2 ms)
[----------] 9 tests from EstimateAffine_EstimateAffine2D (88 ms total)

[----------] 9 tests from EstimateAffine_EstimateAffinePartial2D
[ RUN      ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/0
[ PERFSTAT ]    (samples = 10, mean = 1.07, median = 1.05, stddev = 0.03 (2.9%))
[ VALUE    ]    (5000, 0.99)
[       OK ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/0 (10 ms)
[ RUN      ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/1
[ PERFSTAT ]    (samples = 10, mean = 0.70, median = 0.69, stddev = 0.01 (1.6%))
[ VALUE    ]    (5000, 0.95)
[       OK ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/1 (8 ms)
[ RUN      ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/2
[ PERFSTAT ]    (samples = 10, mean = 0.54, median = 0.53, stddev = 0.01 (2.0%))
[ VALUE    ]    (5000, 0.9)
[       OK ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/2 (5 ms)
[ RUN      ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/3
[ PERFSTAT ]    (samples = 25, mean = 0.25, median = 0.25, stddev = 0.01 (2.7%))
[ VALUE    ]    (1000, 0.99)
[       OK ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/3 (7 ms)
[ RUN      ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/4
[ PERFSTAT ]    (samples = 13, mean = 0.17, median = 0.16, stddev = 0.00 (2.8%))
[ VALUE    ]    (1000, 0.95)
[       OK ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/4 (2 ms)
[ RUN      ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/5
[ PERFSTAT ]    (samples = 13, mean = 0.13, median = 0.13, stddev = 0.00 (2.9%))
[ VALUE    ]    (1000, 0.9)
[       OK ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/5 (2 ms)
[ RUN      ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/6
[ PERFSTAT ]    (samples = 36, mean = 0.07, median = 0.07, stddev = 0.00 (3.0%))
[ VALUE    ]    (100, 0.99)
[       OK ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/6 (2 ms)
[ RUN      ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/7
[ PERFSTAT ]    (samples = 25, mean = 0.05, median = 0.05, stddev = 0.00 (2.9%))
[ VALUE    ]    (100, 0.95)
[       OK ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/7 (1 ms)
[ RUN      ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/8
[ PERFSTAT ]    (samples = 25, mean = 0.04, median = 0.04, stddev = 0.00 (2.8%))
[ VALUE    ]    (100, 0.9)
[       OK ] EstimateAffine_EstimateAffinePartial2D.EstimateAffinePartial2D/8 (1 ms)
[----------] 9 tests from EstimateAffine_EstimateAffinePartial2D (38 ms total)

[----------] Global test environment tear-down
[==========] 18 tests from 2 test cases ran. (126 ms total)
[  PASSED  ] 18 tests.

estimateAffinePartial2D is about 3 times faster. This is expected result since it needs only 2 points for model (instead of 3) and the system to solve is also smaller.

hrnr · 2016-06-01T15:41:57Z

Is it normal that perf test fails on a first run? It fails with No regression data for aff_est argument, that must be because of SANITY_CHECK macro.

It does not happen locally.

hrnr · 2016-06-02T08:38:35Z

added regression data in opencv/opencv_extra#296. Hope that it will make build bot happy.

hrnr · 2016-06-02T09:01:51Z

I have experimented with using eigenval decomposition instead of SVD in eba30a8 (+ trick to make the matrix symmetric), but it is not any faster.

From my POV I consider this feature complete. If we convice the buildbot to use the new generated regression data I can rebase and this could be the first thing to go in.

Since this blocks #6609, I will focus on parallel feature finding subtask for now and continue with affine registration in sttiching pipeline once this will go in.

prclibo · 2016-06-03T14:36:10Z

Basically looks ok to me.

the test size of (100000, 5000, 100) is different from what you have in the sanity sheet in opencv/opencv_extra#296. Do you need to regenerate it?

hrnr · 2016-06-03T15:06:27Z

Thanks for noticing, I have regenerated the data. Hope that buildbot will be happy now.

I have rebased my commits.

hrnr · 2016-06-03T15:26:26Z

Buildbot is failing to merge opencv_extra with my regression data. Is there anything I can do to convice buildbot to use the generated data?

alalek · 2016-06-03T15:32:22Z

You should use the same name for branch in opencv_extra: estimate_affine_2d

hrnr · 2016-06-03T18:01:51Z

Thanks, reopened as opencv/opencv_extra#297 with correct branch name.

Rebased against newest master to force builtbot run.

hrnr · 2016-06-06T14:05:36Z

All green. For me this is good to go in.

* resolve opencv#5183

…reshold

hrnr · 2016-07-04T15:06:33Z

incorporated fix from #6768

EDIT: I pushed a commit into my branch, but this PR is not updating. Hopefully it will update soon.

estimates affine transformation using robust RANSAC method. * uses RANSAC framework in calib3d * includes accuracy test * uses SVD decomposition for solving 3 point equation

estimates limited affine transformation * includes accuracy test

test Calib3d_EstimateAffineTransform rename to Calib3d_EstimateAffine3D. This is more descriptive and prevents confusion with estimateAffine2D tests.

tests both estimateAffine2D and estimateAffinePartial2D

* incorporates fix from opencv#6768

vpisarev · 2016-07-08T13:22:25Z

this PR will be reopened because of CI issues

phryniszak · 2017-01-23T16:05:42Z

estimateAffinePartial2D and estimateAffine2D seems to be buggy. Fails for int input values. Returned values are cast to int so results makes not sens.
More details here:
http://answers.opencv.org/question/123476/estimateaffinepartial2d-and-estimateaffine2d/

hrnr · 2017-01-23T16:26:29Z

In case estimateAffine* functions get something different than floats, inputs are converted to floats and then algorithm works the same. Returned matrix is always doubles.

estimateAffine* uses Levenberg-Marquart as final refinement step. Do you get expected results when you call it with refineIters=0?

phryniszak · 2017-01-23T18:08:11Z

I tried with refineIters=0 and results are still the same and test:

#include "opencv2/highgui.hpp"
#include "opencv2/video/tracking.hpp"
#include "opencv2/calib3d.hpp"


#include <iostream>
#include <vector>

int main(int argc, char *argv[])
{
    std::vector<cv::Point> p1s,p2s;

    p1s.push_back(cv::Point2f( 100, 0));
    p1s.push_back(cv::Point2f( 0, 100));
    p1s.push_back(cv::Point2f(-100, 0));
    p1s.push_back(cv::Point2f( 0,-100));

    p2s.push_back(cv::Point2f(71, -71));
    p2s.push_back(cv::Point2f(71, 71));
    p2s.push_back(cv::Point2f(-71, 71));
    p2s.push_back(cv::Point2f(-71, -71));

    // 1.
    cv::Mat t_false = cv::estimateRigidTransform(p1s,p2s,false);
    std::cout << "estimateRigidTransform false: " << t_false << "\n" << std::flush;

    // 2.
    cv::Mat t_true = cv::estimateRigidTransform(p1s,p2s,true);
    std::cout << "estimateRigidTransform true:" << t_true << "\n" << std::flush;

    // 3.
    std::vector<uchar> inliers(p1s.size(), 0);
    cv::Mat affine1 = cv::estimateAffine2D(p1s, p2s, inliers, cv::RANSAC, 3, 2000, 0.99, 0);
    std::cout << "estimateAffine2D" << affine1 << "\n" << std::flush;

    // 4.
    cv::Mat affine2 = cv::estimateAffinePartial2D(p1s, p2s, inliers, cv::RANSAC, 3, 2000, 0.99, 0);
    std::cout << "estimateAffinePartial2D" << affine2 << "\n" << std::flush;
}

hrnr · 2017-01-23T18:30:46Z

Thanks for report, I will take a look on that. There might be some degenerate points configuration not catched by tests.

I see you are now storing cv::Point2f into cv::Point. Are you sure conversion does what you expect?

phryniszak · 2017-01-23T18:36:52Z

std::vectorcv::Point p1s,p2s and results are:

estimateRigidTransform false: [0.7100000000000001, 0.7100000000000001, 0;
 -0.7100000000000001, 0.7100000000000001, 0]
estimateRigidTransform true:[0.7100000000000001, 0.7100000000000001, 0;
 -0.7100000000000001, 0.7100000000000001, 0]
estimateAffine2D[1, -0, -0;
 -0, 1, -0]
estimateAffinePartial2D[1, -0, 0;
 0, 1, 0]

and std::vectorcv::Point2f p1s,p2s; and correct results:


stimateRigidTransform false: [0.7100000000000001, 0.7100000000000001, 0;
 -0.7100000000000001, 0.7100000000000001, 0]
estimateRigidTransform true:[0.7100000000000001, 0.7100000000000001, 0;
 -0.7100000000000001, 0.7100000000000001, 0]
estimateAffine2D[0.7100000000000001, 0.7100000000000001, -0;
 -0.7100000000000001, 0.7100000000000001, -0]
estimateAffinePartial2D[0.7100000000000001, 0.7100000000000001, 0;
 -0.7100000000000001, 0.7100000000000001, 0]

hrnr · 2017-01-23T19:40:17Z

You are right, there is a stupid bug in estimateAffine* affecting conversion of inputs.

I will fix that (please give me a few days, I'm quite busy now). Thank you for reporting this, we should have probably more tests to cover also conversions.

IshankGulati and others added 5 commits December 23, 2015 23:37

png-image-compatibility

e607a85

Added HAL cvtColor interface

5e91922

Update gen_pattern.py

7f52a55

Save method should save, instead trying to run inkview

HAL math interfaces: fastAtan2, magnitude, sqrt, invSqrt, log, exp

1e667de

core: prevent changes in 'std' namespace

ff0601c

hrnr force-pushed the estimate_affine_2d branch from 4fc37fe to 9e32c5c Compare June 1, 2016 13:48

prclibo reviewed Jun 1, 2016
View reviewed changes

hrnr mentioned this pull request Jun 2, 2016

add regression data for EstimateAffine test opencv/opencv_extra#296

Closed

louis-sportlogiq and others added 3 commits June 2, 2016 11:24

This fixes the seeking in h264 B-Frame enabled video issue. opencv#4890

c03d778

Implement internal HAL for GEMM and matrix decompositions

a2d0cc8

Update HAL samples

901e1e4

hrnr force-pushed the estimate_affine_2d branch from e4c38a7 to 480eea3 Compare June 3, 2016 15:04

hrnr force-pushed the estimate_affine_2d branch from 480eea3 to 3dde7fe Compare June 3, 2016 18:02

mshabunin added 2 commits June 6, 2016 09:18

Merge pull request opencv#6628 from sovrasov:hal_samples_update

89a1e80

Merge pull request opencv#6607 from alalek:issue_6604

532885c

tomoaki0705 and others added 3 commits June 7, 2016 14:53

fix typo

d1c1c01

Change names of some internal functions

055f5c7

Add an extended version of CalibrateCamera function

46fb46c

vpisarev and others added 9 commits June 30, 2016 17:17

Merge pull request opencv#6697 from wiryls:FileStorageBase64

7fea7e0

Merge pull request opencv#6734 from PkLab:doc_videoio_enums

330e4bb

cvtColor: fix inplace processing

8b05a4b

Merge pull request opencv#6745 from alalek:viz_samples

7dc5332

build: fix compiler warnings (GCC 5.3.1)

3844ee7

Merge pull request opencv#6761 from alalek:fix_warnings

9691915

calib3d: compare error and threshold both in square

0e380ff

* resolve opencv#5183

Merge pull request opencv#6768 from tomoaki0705:fixEstimateAffine3DTh…

da69cd0

…reshold

Merge pull request opencv#6760 from alalek:issue_6653

22682f9

hrnr added 5 commits July 4, 2016 17:14

implement estimateAffine2D

ddbc887

estimates affine transformation using robust RANSAC method. * uses RANSAC framework in calib3d * includes accuracy test * uses SVD decomposition for solving 3 point equation

implement estimateAffinePartial2D

89763ac

estimates limited affine transformation * includes accuracy test

calib3d: rename estimateAffine3D test

40da6ad

test Calib3d_EstimateAffineTransform rename to Calib3d_EstimateAffine3D. This is more descriptive and prevents confusion with estimateAffine2D tests.

added perf test for estimateAffine functions

00abefe

tests both estimateAffine2D and estimateAffinePartial2D

calib3d: compare error in square in estimateAffine2D

dbce5f3

* incorporates fix from opencv#6768

hrnr force-pushed the estimate_affine_2d branch from 72719d8 to dbce5f3 Compare July 4, 2016 15:14

mshabunin added broken-pr and removed broken-pr labels Jul 8, 2016

vpisarev closed this Jul 8, 2016

vpisarev mentioned this pull request Jul 8, 2016

[GSoC] robust function for affine transform estimation #6817

Closed

mshabunin added the broken-pr label Jul 14, 2016

hrnr mentioned this pull request Aug 18, 2016

[GSOC] New camera model for stitching pipeline #6933

Merged

24 tasks

hrnr mentioned this pull request Jan 27, 2017

fix wrong conversion in estimateAffine2D* functions #8089

Merged

Uh oh!

Conversation

hrnr commented Jun 1, 2016

Uh oh!

hrnr commented Jun 1, 2016

Uh oh!

hrnr commented Jun 1, 2016

Uh oh!

prclibo Jun 1, 2016

Choose a reason for hiding this comment

Uh oh!

hrnr Jun 1, 2016

Choose a reason for hiding this comment

Uh oh!

hrnr commented Jun 1, 2016

Uh oh!

hrnr commented Jun 1, 2016

Uh oh!

hrnr commented Jun 2, 2016

Uh oh!

hrnr commented Jun 2, 2016

Uh oh!

prclibo commented Jun 3, 2016

Uh oh!

hrnr commented Jun 3, 2016

Uh oh!

hrnr commented Jun 3, 2016

Uh oh!

alalek commented Jun 3, 2016

Uh oh!

hrnr commented Jun 3, 2016

Uh oh!

hrnr commented Jun 6, 2016

Uh oh!

hrnr commented Jul 4, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vpisarev commented Jul 8, 2016

Uh oh!

phryniszak commented Jan 23, 2017

Uh oh!

hrnr commented Jan 23, 2017

Uh oh!

phryniszak commented Jan 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hrnr commented Jan 23, 2017

Uh oh!

phryniszak commented Jan 23, 2017

Uh oh!

hrnr commented Jan 23, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

hrnr commented Jul 4, 2016 •

edited

Loading

phryniszak commented Jan 23, 2017 •

edited

Loading