Skip to content

added FAST for neighborhoods of size 8 and 12#6

Closed
vrabaud wants to merge 1 commit intoopencv:masterfrom
vrabaud:master
Closed

added FAST for neighborhoods of size 8 and 12#6
vrabaud wants to merge 1 commit intoopencv:masterfrom
vrabaud:master

Conversation

@vrabaud
Copy link
Copy Markdown
Contributor

@vrabaud vrabaud commented Jul 31, 2012

We need those for the BRISK implementation that I will provide next, as BRISK looks at FAST at different scales.

I am not too happy about breaking the ABI for the FAST constructor but I don't see any other way.

@vrabaud vrabaud closed this Aug 7, 2012
ozantonkal referenced this pull request in ozantonkal/opencv Jun 11, 2013
allnes added a commit to allnes/opencv that referenced this pull request Mar 16, 2018
allnes added a commit to allnes/opencv that referenced this pull request Mar 18, 2018
@lhxy1991 lhxy1991 mentioned this pull request Jun 26, 2019
philsc added a commit to philsc/opencv that referenced this pull request Aug 31, 2023
The address sanitizer highlighted this issue in our code base. It
looks like the code is currently grabbing a pointer to a temporary
object and then performing operations on it.

I printed some information right before the asan crash:

    eigensolver address: 0x7f0ad95032f0
    eigensolver size: 4528
    eig_vecs_ ptr: 0x7f0ad95045e0
    eig_vecs_ offset: 4848

This shows that `eig_vecs_` points past the end of `eigensolver`. In
other words, it points at the temporary object created by the
`eigensolver.eigenvectors()` call.

Compare the docs for `.eigenvalues()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a0f507ad7ab14797882f474ca8f2773e7
to the docs for `.eigenvectors()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a66288022802172e3ee059283b26201d7

The difference in return types is interesting. `.eigenvalues()`
returns a reference. But `.eigenvectors()` returns a matrix.

This patch here fixes the problem by saving the temporary object and
then grabbing a pointer into it.

This is a curated snippet of the original asan failure:

    ==12==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fc633704640 at pc 0x7fc64f7f1593 bp 0x7ffe8875fc90 sp 0x7ffe8875fc88
    READ of size 8 at 0x7fc633704640 thread T0
        #0 0x7fc64f7f1592 in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:181:48
        opencv#1 0x7fc64f915d92 in cv::usac::EssentialEstimatorImpl::estimateModels(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/estimator.cpp:110:46
        opencv#2 0x7fc64fa74fb0 in cv::usac::Ransac::run(cv::Ptr<cv::usac::RansacOutput>&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:152:58
        opencv#3 0x7fc64fa6cd8e in cv::usac::run(cv::Ptr<cv::usac::Model const> const&, cv::_InputArray const&, cv::_InputArray const&, int, cv::Ptr<cv::usac::RansacOutput>&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:1010:16
        opencv#4 0x7fc64fa6fb46 in cv::usac::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:527:9
        opencv#5 0x7fc64f3b5522 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, int, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:437:16
        opencv#6 0x7fc64f3b7e00 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:486:12
        ...

    Address 0x7fc633704640 is located in stack of thread T0 at offset 17984 in frame
        #0 0x7fc64f7ed4ff in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:36

      This frame has 63 object(s):
        [32, 56) 'coefficients' (line 38)
        [96, 384) 'ee' (line 55)
        ...
        [13040, 17568) 'eigensolver' (line 142)
        [17824, 17840) 'ref.tmp518' (line 143)
        [17856, 17872) 'ref.tmp523' (line 144)
        [17888, 19488) 'ref.tmp524' (line 144) <== Memory access at offset 17984 is inside this variable
        [19616, 19640) 'ref.tmp532' (line 169)
        ...

The crash report says that we're accessing a temporary object from
line 144 when we shouldn't be. Line 144 looks like this:
https://github.com/opencv/opencv/blob/4.6.0/modules/calib3d/src/usac/essential_solver.cpp#L144

    const auto * const eig_vecs_ = (double *) eigensolver.eigenvectors().real().data();

We are using version 4.6.0 for this, but the problem is present on the
4.x branch.
philsc added a commit to philsc/opencv that referenced this pull request Aug 31, 2023
The address sanitizer highlighted this issue in our code base. It
looks like the code is currently grabbing a pointer to a temporary
object and then performing operations on it.

I printed some information right before the asan crash:

    eigensolver address: 0x7f0ad95032f0
    eigensolver size: 4528
    eig_vecs_ ptr: 0x7f0ad95045e0
    eig_vecs_ offset: 4848

This shows that `eig_vecs_` points past the end of `eigensolver`. In
other words, it points at the temporary object created by the
`eigensolver.eigenvectors()` call.

Compare the docs for `.eigenvalues()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a0f507ad7ab14797882f474ca8f2773e7
to the docs for `.eigenvectors()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a66288022802172e3ee059283b26201d7

The difference in return types is interesting. `.eigenvalues()`
returns a reference. But `.eigenvectors()` returns a matrix.

This patch here fixes the problem by saving the temporary object and
then grabbing a pointer into it.

This is a curated snippet of the original asan failure:

    ==12==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fc633704640 at pc 0x7fc64f7f1593 bp 0x7ffe8875fc90 sp 0x7ffe8875fc88
    READ of size 8 at 0x7fc633704640 thread T0
        #0 0x7fc64f7f1592 in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:181:48
        opencv#1 0x7fc64f915d92 in cv::usac::EssentialEstimatorImpl::estimateModels(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/estimator.cpp:110:46
        opencv#2 0x7fc64fa74fb0 in cv::usac::Ransac::run(cv::Ptr<cv::usac::RansacOutput>&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:152:58
        opencv#3 0x7fc64fa6cd8e in cv::usac::run(cv::Ptr<cv::usac::Model const> const&, cv::_InputArray const&, cv::_InputArray const&, int, cv::Ptr<cv::usac::RansacOutput>&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:1010:16
        opencv#4 0x7fc64fa6fb46 in cv::usac::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:527:9
        opencv#5 0x7fc64f3b5522 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, int, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:437:16
        opencv#6 0x7fc64f3b7e00 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:486:12
        ...

    Address 0x7fc633704640 is located in stack of thread T0 at offset 17984 in frame
        #0 0x7fc64f7ed4ff in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:36

      This frame has 63 object(s):
        [32, 56) 'coefficients' (line 38)
        [96, 384) 'ee' (line 55)
        ...
        [13040, 17568) 'eigensolver' (line 142)
        [17824, 17840) 'ref.tmp518' (line 143)
        [17856, 17872) 'ref.tmp523' (line 144)
        [17888, 19488) 'ref.tmp524' (line 144) <== Memory access at offset 17984 is inside this variable
        [19616, 19640) 'ref.tmp532' (line 169)
        ...

The crash report says that we're accessing a temporary object from
line 144 when we shouldn't be. Line 144 looks like this:
https://github.com/opencv/opencv/blob/4.6.0/modules/calib3d/src/usac/essential_solver.cpp#L144

    const auto * const eig_vecs_ = (double *) eigensolver.eigenvectors().real().data();

We are using version 4.6.0 for this, but the problem is present on the
4.x branch.
philsc added a commit to philsc/opencv that referenced this pull request Sep 1, 2023
The address sanitizer highlighted this issue in our code base. It
looks like the code is currently grabbing a pointer to a temporary
object and then performing operations on it.

I printed some information right before the asan crash:

    eigensolver address: 0x7f0ad95032f0
    eigensolver size: 4528
    eig_vecs_ ptr: 0x7f0ad95045e0
    eig_vecs_ offset: 4848

This shows that `eig_vecs_` points past the end of `eigensolver`. In
other words, it points at the temporary object created by the
`eigensolver.eigenvectors()` call.

Compare the docs for `.eigenvalues()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a0f507ad7ab14797882f474ca8f2773e7
to the docs for `.eigenvectors()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a66288022802172e3ee059283b26201d7

The difference in return types is interesting. `.eigenvalues()`
returns a reference. But `.eigenvectors()` returns a matrix.

This patch here fixes the problem by saving the temporary object and
then grabbing a pointer into it.

This is a curated snippet of the original asan failure:

    ==12==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fc633704640 at pc 0x7fc64f7f1593 bp 0x7ffe8875fc90 sp 0x7ffe8875fc88
    READ of size 8 at 0x7fc633704640 thread T0
        #0 0x7fc64f7f1592 in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:181:48
        opencv#1 0x7fc64f915d92 in cv::usac::EssentialEstimatorImpl::estimateModels(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/estimator.cpp:110:46
        opencv#2 0x7fc64fa74fb0 in cv::usac::Ransac::run(cv::Ptr<cv::usac::RansacOutput>&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:152:58
        opencv#3 0x7fc64fa6cd8e in cv::usac::run(cv::Ptr<cv::usac::Model const> const&, cv::_InputArray const&, cv::_InputArray const&, int, cv::Ptr<cv::usac::RansacOutput>&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:1010:16
        opencv#4 0x7fc64fa6fb46 in cv::usac::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:527:9
        opencv#5 0x7fc64f3b5522 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, int, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:437:16
        opencv#6 0x7fc64f3b7e00 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:486:12
        ...

    Address 0x7fc633704640 is located in stack of thread T0 at offset 17984 in frame
        #0 0x7fc64f7ed4ff in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:36

      This frame has 63 object(s):
        [32, 56) 'coefficients' (line 38)
        [96, 384) 'ee' (line 55)
        ...
        [13040, 17568) 'eigensolver' (line 142)
        [17824, 17840) 'ref.tmp518' (line 143)
        [17856, 17872) 'ref.tmp523' (line 144)
        [17888, 19488) 'ref.tmp524' (line 144) <== Memory access at offset 17984 is inside this variable
        [19616, 19640) 'ref.tmp532' (line 169)
        ...

The crash report says that we're accessing a temporary object from
line 144 when we shouldn't be. Line 144 looks like this:
https://github.com/opencv/opencv/blob/4.6.0/modules/calib3d/src/usac/essential_solver.cpp#L144

    const auto * const eig_vecs_ = (double *) eigensolver.eigenvectors().real().data();

We are using version 4.6.0 for this, but the problem is present on the
4.x branch.
philsc added a commit to philsc/opencv that referenced this pull request Sep 4, 2023
The address sanitizer highlighted this issue in our code base. It
looks like the code is currently grabbing a pointer to a temporary
object and then performing operations on it.

I printed some information right before the asan crash:

    eigensolver address: 0x7f0ad95032f0
    eigensolver size: 4528
    eig_vecs_ ptr: 0x7f0ad95045e0
    eig_vecs_ offset: 4848

This shows that `eig_vecs_` points past the end of `eigensolver`. In
other words, it points at the temporary object created by the
`eigensolver.eigenvectors()` call.

Compare the docs for `.eigenvalues()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a0f507ad7ab14797882f474ca8f2773e7
to the docs for `.eigenvectors()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a66288022802172e3ee059283b26201d7

The difference in return types is interesting. `.eigenvalues()`
returns a reference. But `.eigenvectors()` returns a matrix.

This patch here fixes the problem by saving the temporary object and
then grabbing a pointer into it.

This is a curated snippet of the original asan failure:

    ==12==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fc633704640 at pc 0x7fc64f7f1593 bp 0x7ffe8875fc90 sp 0x7ffe8875fc88
    READ of size 8 at 0x7fc633704640 thread T0
        #0 0x7fc64f7f1592 in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:181:48
        opencv#1 0x7fc64f915d92 in cv::usac::EssentialEstimatorImpl::estimateModels(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/estimator.cpp:110:46
        opencv#2 0x7fc64fa74fb0 in cv::usac::Ransac::run(cv::Ptr<cv::usac::RansacOutput>&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:152:58
        opencv#3 0x7fc64fa6cd8e in cv::usac::run(cv::Ptr<cv::usac::Model const> const&, cv::_InputArray const&, cv::_InputArray const&, int, cv::Ptr<cv::usac::RansacOutput>&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:1010:16
        opencv#4 0x7fc64fa6fb46 in cv::usac::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:527:9
        opencv#5 0x7fc64f3b5522 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, int, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:437:16
        opencv#6 0x7fc64f3b7e00 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:486:12
        ...

    Address 0x7fc633704640 is located in stack of thread T0 at offset 17984 in frame
        #0 0x7fc64f7ed4ff in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:36

      This frame has 63 object(s):
        [32, 56) 'coefficients' (line 38)
        [96, 384) 'ee' (line 55)
        ...
        [13040, 17568) 'eigensolver' (line 142)
        [17824, 17840) 'ref.tmp518' (line 143)
        [17856, 17872) 'ref.tmp523' (line 144)
        [17888, 19488) 'ref.tmp524' (line 144) <== Memory access at offset 17984 is inside this variable
        [19616, 19640) 'ref.tmp532' (line 169)
        ...

The crash report says that we're accessing a temporary object from
line 144 when we shouldn't be. Line 144 looks like this:
https://github.com/opencv/opencv/blob/4.6.0/modules/calib3d/src/usac/essential_solver.cpp#L144

    const auto * const eig_vecs_ = (double *) eigensolver.eigenvectors().real().data();

We are using version 4.6.0 for this, but the problem is present on the
4.x branch.

Note that I am dropping the .real() call here. I think that is safe because
of the code further down (line 277 in the most recent version):

    const int eig_i = 20 * i + 12; // eigen stores imaginary values too

The code appears to expect to have to skip doubles for the imaginary parts
of the complex numbers.

Admittedly, I couldn't find a test case that exercised this code path to
validate correctness.
AsyaPronina pushed a commit to AsyaPronina/opencv that referenced this pull request Sep 11, 2023
The address sanitizer highlighted this issue in our code base. It
looks like the code is currently grabbing a pointer to a temporary
object and then performing operations on it.

I printed some information right before the asan crash:

    eigensolver address: 0x7f0ad95032f0
    eigensolver size: 4528
    eig_vecs_ ptr: 0x7f0ad95045e0
    eig_vecs_ offset: 4848

This shows that `eig_vecs_` points past the end of `eigensolver`. In
other words, it points at the temporary object created by the
`eigensolver.eigenvectors()` call.

Compare the docs for `.eigenvalues()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a0f507ad7ab14797882f474ca8f2773e7
to the docs for `.eigenvectors()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a66288022802172e3ee059283b26201d7

The difference in return types is interesting. `.eigenvalues()`
returns a reference. But `.eigenvectors()` returns a matrix.

This patch here fixes the problem by saving the temporary object and
then grabbing a pointer into it.

This is a curated snippet of the original asan failure:

    ==12==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fc633704640 at pc 0x7fc64f7f1593 bp 0x7ffe8875fc90 sp 0x7ffe8875fc88
    READ of size 8 at 0x7fc633704640 thread T0
        #0 0x7fc64f7f1592 in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:181:48
        #1 0x7fc64f915d92 in cv::usac::EssentialEstimatorImpl::estimateModels(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/estimator.cpp:110:46
        opencv#2 0x7fc64fa74fb0 in cv::usac::Ransac::run(cv::Ptr<cv::usac::RansacOutput>&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:152:58
        opencv#3 0x7fc64fa6cd8e in cv::usac::run(cv::Ptr<cv::usac::Model const> const&, cv::_InputArray const&, cv::_InputArray const&, int, cv::Ptr<cv::usac::RansacOutput>&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:1010:16
        opencv#4 0x7fc64fa6fb46 in cv::usac::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:527:9
        opencv#5 0x7fc64f3b5522 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, int, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:437:16
        opencv#6 0x7fc64f3b7e00 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:486:12
        ...

    Address 0x7fc633704640 is located in stack of thread T0 at offset 17984 in frame
        #0 0x7fc64f7ed4ff in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:36

      This frame has 63 object(s):
        [32, 56) 'coefficients' (line 38)
        [96, 384) 'ee' (line 55)
        ...
        [13040, 17568) 'eigensolver' (line 142)
        [17824, 17840) 'ref.tmp518' (line 143)
        [17856, 17872) 'ref.tmp523' (line 144)
        [17888, 19488) 'ref.tmp524' (line 144) <== Memory access at offset 17984 is inside this variable
        [19616, 19640) 'ref.tmp532' (line 169)
        ...

The crash report says that we're accessing a temporary object from
line 144 when we shouldn't be. Line 144 looks like this:
https://github.com/opencv/opencv/blob/4.6.0/modules/calib3d/src/usac/essential_solver.cpp#L144

    const auto * const eig_vecs_ = (double *) eigensolver.eigenvectors().real().data();

We are using version 4.6.0 for this, but the problem is present on the
4.x branch.

Note that I am dropping the .real() call here. I think that is safe because
of the code further down (line 277 in the most recent version):

    const int eig_i = 20 * i + 12; // eigen stores imaginary values too

The code appears to expect to have to skip doubles for the imaginary parts
of the complex numbers.

Admittedly, I couldn't find a test case that exercised this code path to
validate correctness.
savuor pushed a commit to nickyu-zhu/opencv that referenced this pull request Oct 27, 2023
FantasqueX pushed a commit to FantasqueX/opencv that referenced this pull request Dec 21, 2023
Co-authored-by: mingfa.yang <mingfa.yang@aqrose.com>
thewoz pushed a commit to thewoz/opencv that referenced this pull request Jan 4, 2024
The address sanitizer highlighted this issue in our code base. It
looks like the code is currently grabbing a pointer to a temporary
object and then performing operations on it.

I printed some information right before the asan crash:

    eigensolver address: 0x7f0ad95032f0
    eigensolver size: 4528
    eig_vecs_ ptr: 0x7f0ad95045e0
    eig_vecs_ offset: 4848

This shows that `eig_vecs_` points past the end of `eigensolver`. In
other words, it points at the temporary object created by the
`eigensolver.eigenvectors()` call.

Compare the docs for `.eigenvalues()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a0f507ad7ab14797882f474ca8f2773e7
to the docs for `.eigenvectors()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a66288022802172e3ee059283b26201d7

The difference in return types is interesting. `.eigenvalues()`
returns a reference. But `.eigenvectors()` returns a matrix.

This patch here fixes the problem by saving the temporary object and
then grabbing a pointer into it.

This is a curated snippet of the original asan failure:

    ==12==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fc633704640 at pc 0x7fc64f7f1593 bp 0x7ffe8875fc90 sp 0x7ffe8875fc88
    READ of size 8 at 0x7fc633704640 thread T0
        #0 0x7fc64f7f1592 in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:181:48
        opencv#1 0x7fc64f915d92 in cv::usac::EssentialEstimatorImpl::estimateModels(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/estimator.cpp:110:46
        opencv#2 0x7fc64fa74fb0 in cv::usac::Ransac::run(cv::Ptr<cv::usac::RansacOutput>&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:152:58
        opencv#3 0x7fc64fa6cd8e in cv::usac::run(cv::Ptr<cv::usac::Model const> const&, cv::_InputArray const&, cv::_InputArray const&, int, cv::Ptr<cv::usac::RansacOutput>&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:1010:16
        opencv#4 0x7fc64fa6fb46 in cv::usac::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:527:9
        opencv#5 0x7fc64f3b5522 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, int, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:437:16
        opencv#6 0x7fc64f3b7e00 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:486:12
        ...

    Address 0x7fc633704640 is located in stack of thread T0 at offset 17984 in frame
        #0 0x7fc64f7ed4ff in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:36

      This frame has 63 object(s):
        [32, 56) 'coefficients' (line 38)
        [96, 384) 'ee' (line 55)
        ...
        [13040, 17568) 'eigensolver' (line 142)
        [17824, 17840) 'ref.tmp518' (line 143)
        [17856, 17872) 'ref.tmp523' (line 144)
        [17888, 19488) 'ref.tmp524' (line 144) <== Memory access at offset 17984 is inside this variable
        [19616, 19640) 'ref.tmp532' (line 169)
        ...

The crash report says that we're accessing a temporary object from
line 144 when we shouldn't be. Line 144 looks like this:
https://github.com/opencv/opencv/blob/4.6.0/modules/calib3d/src/usac/essential_solver.cpp#L144

    const auto * const eig_vecs_ = (double *) eigensolver.eigenvectors().real().data();

We are using version 4.6.0 for this, but the problem is present on the
4.x branch.

Note that I am dropping the .real() call here. I think that is safe because
of the code further down (line 277 in the most recent version):

    const int eig_i = 20 * i + 12; // eigen stores imaginary values too

The code appears to expect to have to skip doubles for the imaginary parts
of the complex numbers.

Admittedly, I couldn't find a test case that exercised this code path to
validate correctness.
thewoz pushed a commit to thewoz/opencv that referenced this pull request May 29, 2024
The address sanitizer highlighted this issue in our code base. It
looks like the code is currently grabbing a pointer to a temporary
object and then performing operations on it.

I printed some information right before the asan crash:

    eigensolver address: 0x7f0ad95032f0
    eigensolver size: 4528
    eig_vecs_ ptr: 0x7f0ad95045e0
    eig_vecs_ offset: 4848

This shows that `eig_vecs_` points past the end of `eigensolver`. In
other words, it points at the temporary object created by the
`eigensolver.eigenvectors()` call.

Compare the docs for `.eigenvalues()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a0f507ad7ab14797882f474ca8f2773e7
to the docs for `.eigenvectors()`:
https://eigen.tuxfamily.org/dox/classEigen_1_1EigenSolver.html#a66288022802172e3ee059283b26201d7

The difference in return types is interesting. `.eigenvalues()`
returns a reference. But `.eigenvectors()` returns a matrix.

This patch here fixes the problem by saving the temporary object and
then grabbing a pointer into it.

This is a curated snippet of the original asan failure:

    ==12==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fc633704640 at pc 0x7fc64f7f1593 bp 0x7ffe8875fc90 sp 0x7ffe8875fc88
    READ of size 8 at 0x7fc633704640 thread T0
        #0 0x7fc64f7f1592 in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:181:48
        opencv#1 0x7fc64f915d92 in cv::usac::EssentialEstimatorImpl::estimateModels(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/estimator.cpp:110:46
        opencv#2 0x7fc64fa74fb0 in cv::usac::Ransac::run(cv::Ptr<cv::usac::RansacOutput>&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:152:58
        opencv#3 0x7fc64fa6cd8e in cv::usac::run(cv::Ptr<cv::usac::Model const> const&, cv::_InputArray const&, cv::_InputArray const&, int, cv::Ptr<cv::usac::RansacOutput>&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:1010:16
        opencv#4 0x7fc64fa6fb46 in cv::usac::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/ransac_solvers.cpp:527:9
        opencv#5 0x7fc64f3b5522 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, int, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:437:16
        opencv#6 0x7fc64f3b7e00 in cv::findEssentialMat(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, int, double, double, cv::_OutputArray const&) /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/five-point.cpp:486:12
        ...

    Address 0x7fc633704640 is located in stack of thread T0 at offset 17984 in frame
        #0 0x7fc64f7ed4ff in cv::usac::EssentialMinimalSolverStewenius5ptsImpl::estimate(std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) const /proc/self/cwd/external/com_github_opencv_opencv/modules/calib3d/src/usac/essential_solver.cpp:36

      This frame has 63 object(s):
        [32, 56) 'coefficients' (line 38)
        [96, 384) 'ee' (line 55)
        ...
        [13040, 17568) 'eigensolver' (line 142)
        [17824, 17840) 'ref.tmp518' (line 143)
        [17856, 17872) 'ref.tmp523' (line 144)
        [17888, 19488) 'ref.tmp524' (line 144) <== Memory access at offset 17984 is inside this variable
        [19616, 19640) 'ref.tmp532' (line 169)
        ...

The crash report says that we're accessing a temporary object from
line 144 when we shouldn't be. Line 144 looks like this:
https://github.com/opencv/opencv/blob/4.6.0/modules/calib3d/src/usac/essential_solver.cpp#L144

    const auto * const eig_vecs_ = (double *) eigensolver.eigenvectors().real().data();

We are using version 4.6.0 for this, but the problem is present on the
4.x branch.

Note that I am dropping the .real() call here. I think that is safe because
of the code further down (line 277 in the most recent version):

    const int eig_i = 20 * i + 12; // eigen stores imaginary values too

The code appears to expect to have to skip doubles for the imaginary parts
of the complex numbers.

Admittedly, I couldn't find a test case that exercised this code path to
validate correctness.
asmorkalov pushed a commit that referenced this pull request Dec 23, 2024
Use size_t when calculating size of all_points #26650

Closes: #26642 

Asan log
```
=================================================================
==41401==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fc55a02a3fc at pc 0x7fc58e304131 bp 0x7ffd54787b00 sp 0x7ffd54787af8
WRITE of size 4 at 0x7fc55a02a3fc thread T0
    #0 0x7fc58e304130 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&) /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726
    #1 0x7fc58e3054b0 in cv::QRDetectMulti::localization() /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3829
    #2 0x7fc58e308020 in cv::ImplContour::detectMulti(cv::_InputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3987
    #3 0x7fc58e30b5b1 in cv::ImplContour::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:4176
    #4 0x7fc58e28922f in cv::GraphicalCodeDetector::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/graphical_code_detector.cpp:42
    #5 0x5954e8 in Body /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:48
    #6 0x594fc0 in TestBody /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:42
    #7 0x67ee6a in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919
    #8 0x6734a4 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955
    #9 0x641fe8 in testing::Test::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3993
    #10 0x6431ac in testing::TestInfo::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4169
    #11 0x643d15 in testing::TestCase::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4287
    #12 0x659ff3 in testing::internal::UnitTestImpl::RunAllTests() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6662
    #13 0x681205 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919
    #14 0x675127 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955
    #15 0x65734c in testing::UnitTest::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6271
    #16 0x5907f0 in RUN_ALL_TESTS() /home/fanta/source/opencv/modules/ts/include/opencv2/ts/ts_gtest.h:22240
    #17 0x590cdd in main (/home/fanta/source/opencv-build-4.x-clang/bin/opencv_test_objdetect+0x590cdd) (BuildId: a9363fc788d57c48225fc0559ac9199d07d415db)
    #18 0x7fc58ab242ad in __libc_start_call_main (/lib64/libc.so.6+0x2a2ad) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7)
    #19 0x7fc58ab24378 in __libc_start_main@@GLIBC_2.34 (/lib64/libc.so.6+0x2a378) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7)
    #20 0x417014 in _start ../sysdeps/x86_64/start.S:115

0x7fc55a02a3fc is located 0 bytes after 2938510332-byte region [0x7fc4aadc8800,0x7fc55a02a3fc)
allocated by thread T0 here:
    #0 0x7fc58e590298 in operator new(unsigned long) (/lib64/libasan.so.8+0xfd298) (BuildId: da72ee674d801ced58193987786b90646d94ff8d)
    #1 0x7fc58e34d010 in std::__new_allocator<cv::Vec<int, 3> >::allocate(unsigned long, void const*) /usr/include/c++/14/bits/new_allocator.h:151

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&)

Shadow bytes around the buggy address:
  0x7fc55a02a100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x7fc55a02a380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04]
  0x7fc55a02a400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==41401==ABORTING
```

`(true_points_group[i].size()` is 1794 and `(true_points_group[i].size() - 2 ) * (true_points_group[i].size() - 1) * true_points_group[i].size())` is 5764222464 which overflows `int`

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
shyama7004 pushed a commit to shyama7004/opencv that referenced this pull request Jan 16, 2025
* Prepared OpenCV iOS Framework build

* Added iOS Framework build for 3.4 branch

* Added a functionality to run jobs on workflow_dispatch, linting
shyama7004 pushed a commit to shyama7004/opencv that referenced this pull request Jan 20, 2025
Use size_t when calculating size of all_points opencv#26650

Closes: opencv#26642 

Asan log
```
=================================================================
==41401==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fc55a02a3fc at pc 0x7fc58e304131 bp 0x7ffd54787b00 sp 0x7ffd54787af8
WRITE of size 4 at 0x7fc55a02a3fc thread T0
    #0 0x7fc58e304130 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&) /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726
    opencv#1 0x7fc58e3054b0 in cv::QRDetectMulti::localization() /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3829
    opencv#2 0x7fc58e308020 in cv::ImplContour::detectMulti(cv::_InputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3987
    opencv#3 0x7fc58e30b5b1 in cv::ImplContour::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:4176
    opencv#4 0x7fc58e28922f in cv::GraphicalCodeDetector::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/graphical_code_detector.cpp:42
    opencv#5 0x5954e8 in Body /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:48
    opencv#6 0x594fc0 in TestBody /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:42
    opencv#7 0x67ee6a in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919
    opencv#8 0x6734a4 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955
    opencv#9 0x641fe8 in testing::Test::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3993
    opencv#10 0x6431ac in testing::TestInfo::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4169
    opencv#11 0x643d15 in testing::TestCase::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4287
    opencv#12 0x659ff3 in testing::internal::UnitTestImpl::RunAllTests() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6662
    opencv#13 0x681205 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919
    opencv#14 0x675127 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955
    opencv#15 0x65734c in testing::UnitTest::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6271
    opencv#16 0x5907f0 in RUN_ALL_TESTS() /home/fanta/source/opencv/modules/ts/include/opencv2/ts/ts_gtest.h:22240
    opencv#17 0x590cdd in main (/home/fanta/source/opencv-build-4.x-clang/bin/opencv_test_objdetect+0x590cdd) (BuildId: a9363fc788d57c48225fc0559ac9199d07d415db)
    opencv#18 0x7fc58ab242ad in __libc_start_call_main (/lib64/libc.so.6+0x2a2ad) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7)
    opencv#19 0x7fc58ab24378 in __libc_start_main@@GLIBC_2.34 (/lib64/libc.so.6+0x2a378) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7)
    opencv#20 0x417014 in _start ../sysdeps/x86_64/start.S:115

0x7fc55a02a3fc is located 0 bytes after 2938510332-byte region [0x7fc4aadc8800,0x7fc55a02a3fc)
allocated by thread T0 here:
    #0 0x7fc58e590298 in operator new(unsigned long) (/lib64/libasan.so.8+0xfd298) (BuildId: da72ee674d801ced58193987786b90646d94ff8d)
    opencv#1 0x7fc58e34d010 in std::__new_allocator<cv::Vec<int, 3> >::allocate(unsigned long, void const*) /usr/include/c++/14/bits/new_allocator.h:151

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&)

Shadow bytes around the buggy address:
  0x7fc55a02a100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x7fc55a02a380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04]
  0x7fc55a02a400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==41401==ABORTING
```

`(true_points_group[i].size()` is 1794 and `(true_points_group[i].size() - 2 ) * (true_points_group[i].size() - 1) * true_points_group[i].size())` is 5764222464 which overflows `int`

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
NanQin555 pushed a commit to NanQin555/opencv that referenced this pull request Feb 24, 2025
Use size_t when calculating size of all_points opencv#26650

Closes: opencv#26642 

Asan log
```
=================================================================
==41401==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fc55a02a3fc at pc 0x7fc58e304131 bp 0x7ffd54787b00 sp 0x7ffd54787af8
WRITE of size 4 at 0x7fc55a02a3fc thread T0
    #0 0x7fc58e304130 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&) /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726
    opencv#1 0x7fc58e3054b0 in cv::QRDetectMulti::localization() /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3829
    opencv#2 0x7fc58e308020 in cv::ImplContour::detectMulti(cv::_InputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3987
    opencv#3 0x7fc58e30b5b1 in cv::ImplContour::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:4176
    opencv#4 0x7fc58e28922f in cv::GraphicalCodeDetector::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/graphical_code_detector.cpp:42
    opencv#5 0x5954e8 in Body /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:48
    opencv#6 0x594fc0 in TestBody /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:42
    opencv#7 0x67ee6a in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919
    opencv#8 0x6734a4 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955
    opencv#9 0x641fe8 in testing::Test::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3993
    opencv#10 0x6431ac in testing::TestInfo::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4169
    opencv#11 0x643d15 in testing::TestCase::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4287
    opencv#12 0x659ff3 in testing::internal::UnitTestImpl::RunAllTests() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6662
    opencv#13 0x681205 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919
    opencv#14 0x675127 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955
    opencv#15 0x65734c in testing::UnitTest::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6271
    opencv#16 0x5907f0 in RUN_ALL_TESTS() /home/fanta/source/opencv/modules/ts/include/opencv2/ts/ts_gtest.h:22240
    opencv#17 0x590cdd in main (/home/fanta/source/opencv-build-4.x-clang/bin/opencv_test_objdetect+0x590cdd) (BuildId: a9363fc788d57c48225fc0559ac9199d07d415db)
    opencv#18 0x7fc58ab242ad in __libc_start_call_main (/lib64/libc.so.6+0x2a2ad) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7)
    opencv#19 0x7fc58ab24378 in __libc_start_main@@GLIBC_2.34 (/lib64/libc.so.6+0x2a378) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7)
    opencv#20 0x417014 in _start ../sysdeps/x86_64/start.S:115

0x7fc55a02a3fc is located 0 bytes after 2938510332-byte region [0x7fc4aadc8800,0x7fc55a02a3fc)
allocated by thread T0 here:
    #0 0x7fc58e590298 in operator new(unsigned long) (/lib64/libasan.so.8+0xfd298) (BuildId: da72ee674d801ced58193987786b90646d94ff8d)
    opencv#1 0x7fc58e34d010 in std::__new_allocator<cv::Vec<int, 3> >::allocate(unsigned long, void const*) /usr/include/c++/14/bits/new_allocator.h:151

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&)

Shadow bytes around the buggy address:
  0x7fc55a02a100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x7fc55a02a380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04]
  0x7fc55a02a400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==41401==ABORTING
```

`(true_points_group[i].size()` is 1794 and `(true_points_group[i].size() - 2 ) * (true_points_group[i].size() - 1) * true_points_group[i].size())` is 5764222464 which overflows `int`

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
@coderabbitai coderabbitai bot mentioned this pull request Sep 28, 2025
6 tasks
SrimanPadmanabanCB added a commit to karthiknaveene/opencv that referenced this pull request Oct 12, 2025
SrimanPadmanabanCB added a commit to karthiknaveene/opencv that referenced this pull request Oct 12, 2025
SrimanPadmanabanCB added a commit to karthiknaveene/opencv that referenced this pull request Oct 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant