Use size_t when calculating size of all_points#26650
Conversation
c5adbaf to
00f8feb
Compare
|
We cannot always use |
|
Yes, Assert is very good idea here. |
|
I propose to add |
|
I'm unfamiliar with adding a commit to a pre-existing fork/PR, so I'll just leave this here. It appears that the main reason for the long duration is the number of A quick summary of the approach is this. The Maybe something like |
|
@JSpencerPittman Tnank you for your thoughts. I'm not super familiar with this block of code, could you provide a rough suggestion of what you think this code change would look like? Are you suggesting changing this line, modifying the I can try it out and see if it addresses the slowdown. I'm not sure how to tell if that would otherwise degrade the performance of the detector. |
|
Howdy @marcreichman-pfi, I was referring to this line, right below it. I believe kmeans assigns a higher priority to the directly passed in number of attempts over the one specified in the termination critiera, kmeans(all_contours_points, count_contours, qrcode_labels,
TermCriteria( TermCriteria::EPS + TermCriteria::COUNT, 10, 0.1),
std::min(count_contours, 10), KMEANS_PP_CENTERS, clustered_localization_points);Although, the main choking point is on this line this line with a possible fix being: kmeans(list_lines_y, num_points, labels,
TermCriteria( TermCriteria::EPS + TermCriteria::COUNT, 10, 0.1),
std::min(num_points, 10), KMEANS_PP_CENTERS, tmp_localization_points);This alteration sped it up on my end to almost instant rather than the 10 minutes of waiting. These changes may cause an error, but I believe it's a further downstream issue that needs to be fixed. I've been getting a Performance ComparisonI went ahead and wrote a custom benchmark using 1000 images from this dataset. On it the results before and after the change were identical on images that had a QR code within them. The code was quite confusing, but it seems like when it finds a prominent tag it quits its search and moves on with it. So often the problem is it has too few contours which is why its search is quick, despite leaving out other QR codes. However, when there's no QR code it's grabbing onto anything it can leaving a pile of 1000 contours too sift through on a singular image.
Here the green boxes are the ground truth and the red square is the detected QR code. ResultsAgainst the
I went ahead and also benchmarked the ArUco detector to see how it compares. |
|
Hi @JSpencerPittman , Thank you so much for this additional information and testing. I have made your suggested changes locally and feel I'm still doing something wrong as things are still hanging. I am fine to take this to another location if this PR is no longer the most appropriate, or provide an email. Just let me know. Working at this fork commit My test code is this: TEST(QRCppTest, testQRHangCases) {
std::cout << "Testing hang-crash1.jpg" << std::endl << std::flush;
{
bfs::path testFile = (bfs::path(COMMON_DATA_PATH) / "qrDetectPy" / "hang-crash1.jpg").make_preferred();
cv::Mat testImage = cv::imread(testFile.string());
cv::QRCodeDetector qrCodeDetector;
std::vector<std::string> decodedInfo;
bool retVal = qrCodeDetector.detectAndDecodeMulti(testImage, decodedInfo);
ASSERT_FALSE(retVal);
}
std::cout << "Finished hang-crash1.jpg" << std::endl << std::flush;
std::cout << "Testing hang-crash2.jpg" << std::endl << std::flush;
{
bfs::path testFile = (bfs::path(COMMON_DATA_PATH) / "qrDetectPy" / "hang-crash2.jpg").make_preferred();
cv::Mat testImage = cv::imread(testFile.string());
cv::QRCodeDetector qrCodeDetector;
std::vector<std::string> decodedInfo;
bool retVal = qrCodeDetector.detectAndDecodeMulti(testImage, decodedInfo);
ASSERT_FALSE(retVal);
}
std::cout << "Finished hang-crash2.jpg" << std::endl << std::flush;
}My test run is this: I am still working to be 100% certain I'm running this test against the correct code but I believe this is the case. I wonder if the same detection method is being called in my test vs yours? ##Small followup I modified the test to call I'm not clear on what I'm doing differently than your changes. Thanks again for your continued time and attention. |
Use size_t when calculating size of all_points opencv#26650 Closes: opencv#26642 Asan log ``` ================================================================= ==41401==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fc55a02a3fc at pc 0x7fc58e304131 bp 0x7ffd54787b00 sp 0x7ffd54787af8 WRITE of size 4 at 0x7fc55a02a3fc thread T0 #0 0x7fc58e304130 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&) /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726 opencv#1 0x7fc58e3054b0 in cv::QRDetectMulti::localization() /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3829 opencv#2 0x7fc58e308020 in cv::ImplContour::detectMulti(cv::_InputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3987 opencv#3 0x7fc58e30b5b1 in cv::ImplContour::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:4176 opencv#4 0x7fc58e28922f in cv::GraphicalCodeDetector::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/graphical_code_detector.cpp:42 opencv#5 0x5954e8 in Body /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:48 opencv#6 0x594fc0 in TestBody /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:42 opencv#7 0x67ee6a in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919 opencv#8 0x6734a4 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955 opencv#9 0x641fe8 in testing::Test::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3993 opencv#10 0x6431ac in testing::TestInfo::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4169 opencv#11 0x643d15 in testing::TestCase::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4287 opencv#12 0x659ff3 in testing::internal::UnitTestImpl::RunAllTests() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6662 opencv#13 0x681205 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919 opencv#14 0x675127 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955 opencv#15 0x65734c in testing::UnitTest::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6271 opencv#16 0x5907f0 in RUN_ALL_TESTS() /home/fanta/source/opencv/modules/ts/include/opencv2/ts/ts_gtest.h:22240 opencv#17 0x590cdd in main (/home/fanta/source/opencv-build-4.x-clang/bin/opencv_test_objdetect+0x590cdd) (BuildId: a9363fc788d57c48225fc0559ac9199d07d415db) opencv#18 0x7fc58ab242ad in __libc_start_call_main (/lib64/libc.so.6+0x2a2ad) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7) opencv#19 0x7fc58ab24378 in __libc_start_main@@GLIBC_2.34 (/lib64/libc.so.6+0x2a378) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7) opencv#20 0x417014 in _start ../sysdeps/x86_64/start.S:115 0x7fc55a02a3fc is located 0 bytes after 2938510332-byte region [0x7fc4aadc8800,0x7fc55a02a3fc) allocated by thread T0 here: #0 0x7fc58e590298 in operator new(unsigned long) (/lib64/libasan.so.8+0xfd298) (BuildId: da72ee674d801ced58193987786b90646d94ff8d) opencv#1 0x7fc58e34d010 in std::__new_allocator<cv::Vec<int, 3> >::allocate(unsigned long, void const*) /usr/include/c++/14/bits/new_allocator.h:151 SUMMARY: AddressSanitizer: heap-buffer-overflow /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&) Shadow bytes around the buggy address: 0x7fc55a02a100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7fc55a02a180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7fc55a02a200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7fc55a02a280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7fc55a02a300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 =>0x7fc55a02a380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04] 0x7fc55a02a400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7fc55a02a480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7fc55a02a500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7fc55a02a580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7fc55a02a600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==41401==ABORTING ``` `(true_points_group[i].size()` is 1794 and `(true_points_group[i].size() - 2 ) * (true_points_group[i].size() - 1) * true_points_group[i].size())` is 5764222464 which overflows `int` ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake
Use size_t when calculating size of all_points opencv#26650 Closes: opencv#26642 Asan log ``` ================================================================= ==41401==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fc55a02a3fc at pc 0x7fc58e304131 bp 0x7ffd54787b00 sp 0x7ffd54787af8 WRITE of size 4 at 0x7fc55a02a3fc thread T0 #0 0x7fc58e304130 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&) /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726 opencv#1 0x7fc58e3054b0 in cv::QRDetectMulti::localization() /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3829 opencv#2 0x7fc58e308020 in cv::ImplContour::detectMulti(cv::_InputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3987 opencv#3 0x7fc58e30b5b1 in cv::ImplContour::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:4176 opencv#4 0x7fc58e28922f in cv::GraphicalCodeDetector::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/graphical_code_detector.cpp:42 opencv#5 0x5954e8 in Body /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:48 opencv#6 0x594fc0 in TestBody /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:42 opencv#7 0x67ee6a in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919 opencv#8 0x6734a4 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955 opencv#9 0x641fe8 in testing::Test::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3993 opencv#10 0x6431ac in testing::TestInfo::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4169 opencv#11 0x643d15 in testing::TestCase::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4287 opencv#12 0x659ff3 in testing::internal::UnitTestImpl::RunAllTests() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6662 opencv#13 0x681205 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919 opencv#14 0x675127 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955 opencv#15 0x65734c in testing::UnitTest::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6271 opencv#16 0x5907f0 in RUN_ALL_TESTS() /home/fanta/source/opencv/modules/ts/include/opencv2/ts/ts_gtest.h:22240 opencv#17 0x590cdd in main (/home/fanta/source/opencv-build-4.x-clang/bin/opencv_test_objdetect+0x590cdd) (BuildId: a9363fc788d57c48225fc0559ac9199d07d415db) opencv#18 0x7fc58ab242ad in __libc_start_call_main (/lib64/libc.so.6+0x2a2ad) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7) opencv#19 0x7fc58ab24378 in __libc_start_main@@GLIBC_2.34 (/lib64/libc.so.6+0x2a378) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7) opencv#20 0x417014 in _start ../sysdeps/x86_64/start.S:115 0x7fc55a02a3fc is located 0 bytes after 2938510332-byte region [0x7fc4aadc8800,0x7fc55a02a3fc) allocated by thread T0 here: #0 0x7fc58e590298 in operator new(unsigned long) (/lib64/libasan.so.8+0xfd298) (BuildId: da72ee674d801ced58193987786b90646d94ff8d) opencv#1 0x7fc58e34d010 in std::__new_allocator<cv::Vec<int, 3> >::allocate(unsigned long, void const*) /usr/include/c++/14/bits/new_allocator.h:151 SUMMARY: AddressSanitizer: heap-buffer-overflow /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&) Shadow bytes around the buggy address: 0x7fc55a02a100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7fc55a02a180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7fc55a02a200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7fc55a02a280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7fc55a02a300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 =>0x7fc55a02a380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04] 0x7fc55a02a400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7fc55a02a480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7fc55a02a500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7fc55a02a580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7fc55a02a600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==41401==ABORTING ``` `(true_points_group[i].size()` is 1794 and `(true_points_group[i].size() - 2 ) * (true_points_group[i].size() - 1) * true_points_group[i].size())` is 5764222464 which overflows `int` ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake



Closes: #26642
Asan log
(true_points_group[i].size()is 1794 and(true_points_group[i].size() - 2 ) * (true_points_group[i].size() - 1) * true_points_group[i].size())is 5764222464 which overflowsintPull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.