Main PnP pose computation doc page#21320
Conversation
|
|
||
| Pose refinement consists in estimating the rotation and translation that minimizes the reprojection error using a non-linear minimization method and starting from an initial estimate of the solution. OpenCV proposes cv::solvePnPRefineLM() and cv::solvePnPRefineVVS() for this problem. | ||
|
|
||
| cv::solvePnPRefineLM() uses a non-linear Levenberg-Marquardt minimization scheme @cite Madsen04 @cite Eade13 and the current implementation computes the rotation update as a perturbation and not on SO(3). |
There was a problem hiding this comment.
Is it correct for the current implementation that the rotation update is not done on SO(3)? So it should only be valid for small rotation perturbation.
What about the future LM implementation?
There was a problem hiding this comment.
I thought about it when reimplementing LevMarq. Of course, it's better to do it in true way with SO(3) and exponential oplus but:
(0) The current code works
(1) AFAIK the difference between the true method and the existing method has more influence on a convergence speed than on a result correctness
(2) LevMarq is a local optimization algorithm, it doesn't have to work well on large perturbations
The ongoing LevMarq implementation will let a user (OK, maybe not an OpenCV user but at least an OpenCV contributor) to implement objective functions which accept SE(3) among other things.
There was a problem hiding this comment.
Thanks for your answer. This makes sense.
(1) AFAIK the difference between the true method and the existing method has more influence on a convergence speed than on a result correctness
Maybe some related discussion: https://forum.opencv.org/t/slow-convergence-of-solvepnprefinelm/5486
Wondering if the new implementation would let adding additional constraints such as line of sight or similar ideas into the solving framework?
There was a problem hiding this comment.
Thanks for the link to the discussion. Now it's obvious to me that sometimes bad convergence speed means bad accuracy.
I'm not sure what exactly this line of sight constraint is. If it can be done as a part of cost function in a regularization manner then even old LevMarq is able to work with them.
There was a problem hiding this comment.
Thanks for the info and the work done 👍
On another topic, there is this paper: A compact formula for the derivative of a 3-D rotation in exponential coordinates. Not sure if it can be useful pratically.
There was a problem hiding this comment.
Regarding the 3D rotation derivative: usually an exponential derivative near 0 is used in local optimization, see this example.
b8811e7 to
032a61b
Compare
alalek
left a comment
There was a problem hiding this comment.
Thank you for the contribution 👍
Use a single PnP pose computation page similar to the Bayer conversion page in order to remove duplicated doc and avoid potential mistakes when updating the doc in the future.
Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.