Detection and decoding of curved QR-codes by APrigarina · Pull Request #18003 · opencv/opencv

APrigarina · 2020-07-31T14:16:03Z

These changes add detection and decoding of curved QR-codes

modules/objdetect/src/qrcode.cpp

Co-authored-by: Nesterov Alexander <nesoldr@gmail.com>

modules/objdetect/src/qrcode.cpp

allnes · 2020-10-07T13:01:29Z

modules/objdetect/src/qrcode.cpp

+        }
+        num_points_at_side += it->second.size();
+    }
+    mean_step /= num_points_at_side;


please add test for check zero divided

allnes · 2020-10-07T13:02:33Z

modules/objdetect/src/qrcode.cpp

+        int count = -1;
+        for(size_t j = 0; j < it->second.size() - 1; j++)
+        {
+            count++;


please replace near with j++

allnes · 2020-10-07T13:07:14Z

modules/objdetect/src/qrcode.cpp

+    for (it=complete_curved_sides.begin(); it!=complete_curved_sides.end(); ++it)
+    {
+        int count = -1;
+        for(size_t j = 0; j < it->second.size() - 1; j++)


please replace all constants outside cycles because cycle call size() function every iteration

allnes · 2020-10-07T13:15:16Z

modules/objdetect/src/qrcode.cpp

+    return true;
+}
+
+vector<vector<float> > QRDecode::computeSpline(vector<int> &x_arr, vector<int> &y_arr)


Suggested change

vector<vector<float> > QRDecode::computeSpline(vector<int> &x_arr, vector<int> &y_arr)

vector<vector<float> > QRDecode::computeSpline(const vector<int> &x_arr, const vector<int> &y_arr)

allnes · 2020-10-07T13:15:41Z

modules/objdetect/src/qrcode.cpp

+
+vector<vector<float> > QRDecode::computeSpline(vector<int> &x_arr, vector<int> &y_arr)
+{
+    int n = (int)y_arr.size();


Suggested change

int n = (int)y_arr.size();

const int n = (int)y_arr.size();

allnes · 2020-10-07T13:39:41Z

modules/objdetect/src/qrcode.cpp

+                {
+                    if ((x > x_arr[i]) && (x <= x_arr[i + 1]))
+                    {
+                        float y = S[i][0] + S[i][1] * (x - x_arr[i]) + S[i][2] * pow((x - x_arr[i]), 2)


please use multiplication instead pow

allnes · 2020-10-07T13:43:04Z

modules/objdetect/src/qrcode.cpp

+            x_arr.push_back(cvRound(spline_points[j].x));
+            y_arr.push_back(cvRound(spline_points[j].y));
+        }
+        if (abs(x_arr.front() - x_arr.back()) > abs(y_arr.front() - y_arr.back()))


please use swap between x and y for calculate only one condition branch

allnes · 2020-10-07T13:46:54Z

modules/objdetect/src/qrcode.cpp

+
+bool QRDecode::divideIntoEvenSegments(vector<vector<Point2f> > &segments_points)
+{
+    vector<vector<Point2f> > spline_lines(NUM_SIDES);


please add NUM_SIDES into class

allnes · 2020-10-07T13:54:58Z

modules/objdetect/src/qrcode.cpp

+        mean_num_points_in_line += spline_lines[i].size();
+    }
+    mean_num_points_in_line /= NUM_SIDES;
+    int min_num_points = 1, max_num_points = cvRound(mean_num_points_in_line / 2.0);


Suggested change

int min_num_points = 1, max_num_points = cvRound(mean_num_points_in_line / 2.0);

const int min_num_points = 1, max_num_points = cvRound(mean_num_points_in_line / 2.0);

allnes · 2020-10-07T13:57:33Z

modules/objdetect/src/qrcode.cpp

+            mean_of_two_sides    += mean_dist_in_segment;
+        }
+        mean_of_two_sides /= NUM_SIDES;
+        if (mean_of_two_sides < 0.5)


please name magic number

rayonnant14 · 2020-10-15T08:38:26Z

modules/objdetect/src/qrcode.cpp

+        if (abs(temp_point_start.x - temp_point_end.x) >
+            abs(temp_point_start.y - temp_point_end.y))
+        {
+            if(segments_points[i].front().y > segments_points[(i+1)%2].front().y)
+            {
+                current_curved_side  = segments_points[i];
+                opposite_curved_side = segments_points[(i+1)%2];
+            }
+        }
+        else
+        {
+            if(segments_points[i].front().x < segments_points[(i+1)%2].front().x)
+            {
+                current_curved_side  = segments_points[i];
+                opposite_curved_side = segments_points[(i+1)%2];
+            }
+        }
+    }


please revise conditions (maybe you can do it simpler)

rayonnant14 · 2020-10-15T08:55:38Z

modules/objdetect/src/qrcode.cpp

+    if (complete_curved_sides.size() < 2)
+    {
+        for (size_t i = 0; i < NUM_SIDES; i++)
+        {
+            if(complete_curved_sides.count(curved_indexes[i]) == 0)
+            {
+                int idx_second_cur_side = curved_indexes[i];
+                complete_curved_sides.insert(std::pair<int,vector<Point> >(idx_second_cur_side, sides_points[idx_second_cur_side]));
+            }
+        }
+    }
+    std::map<int,vector<Point> >::iterator it;
+    for (it = complete_curved_sides.begin(); it != complete_curved_sides.end(); ++it)
+    {
+        Point p1 = it->second.front();
+        Point p2 = it->second.back();
+        if (abs(p1.x - p2.x) > abs(p1.y - p2.y))
+        {
+            sort(it->second.begin(), it->second.end(), sortPointsByX());
+        }
+        else
+        {
+            sort(it->second.begin(), it->second.end(), sortPointsByY());
+        }
+    }


please add it into new method

rayonnant14 · 2020-10-15T09:02:19Z

modules/objdetect/src/qrcode.cpp

+    CV_TRACE_FUNCTION();
+    const double multiplyingFactor = (version < 3)  ? 1 :
+                                     (version == 3) ? 1.5 :
+                                     version * (5 + version - 4);


Suggested change

version * (5 + version - 4);

version * (version + 1);

alalek · 2020-08-02T09:01:49Z

modules/objdetect/src/qrcode.cpp

+                              ((a1.x * a2.y  -  a1.y * a2.x) * (b1.x - b2.x) -
+                               (b1.x * b2.y  -  b1.y * b2.x) * (a1.x - a2.x)) /
+                              ((a1.x - a2.x) * (b1.y - b2.y) -
+                               (a1.y - a2.y) * (b1.x - b2.x)),


divisor should be pre-calculated and checked for 0 (eps tolerance) or NaNs.

Or ensure that caller works well with Inf/NaN values and filter them out later.

alalek · 2020-10-19T11:16:31Z

modules/objdetect/src/qrcode.cpp

+    if (sqrt(A*A + B*B) == 0) return 0;
+    result = abs((A * a.x - B * a.y + C)) / sqrt(A*A + B*B);


Avoid calculation of sqrt() expression twice.

alalek · 2020-10-19T11:17:52Z

modules/objdetect/src/qrcode.cpp

+        }
+        num_points_at_side += num_points_at_pattern;
+    }
+    if (num_points_at_side == 0) { return false; }


Please use separate line for if body, even for simple cases.
It is important to see the real code coverage.

ismael-benito · 2023-08-16T15:08:30Z

Hi! Last year, I defended my PhD in Engineering and Applied Science, regarding a thesis which involved huge use of QR Codes, at the time we didn't realize OpenCV had included a curved QR Code decoder, it is amazing the work the community has been done in recent years. In the thesis, we tackled the recovery of QR Codes from curved surfaces using Thin Plate Splines (Chapter 3 of https://diposit.ub.edu/dspace/bitstream/2445/185845/1/IBA_PhD_THESIS.pdf). Is there any paper regarding the implementation here presented of the use of splines to extract the QR Code? If we understand correctly, the above presented method uses the splines to fit the outer contour of the QR Code? We would like to be able to compare our method and the one presented here, and if able to present the research of the comparison and later attempt to make a pull request.

TL;DR: Looking for a paper related with this method.

APrigarina added 2 commits July 31, 2020 16:55

temp changes for curved qrcodes

2a5162f

added api for curved qr code decoding

c69d515

allnes reviewed Sep 23, 2020

View reviewed changes

APrigarina marked this pull request as ready for review September 27, 2020 09:17

APrigarina and others added 2 commits September 28, 2020 11:11

fixed prototypes

a90192e

Co-authored-by: Nesterov Alexander <nesoldr@gmail.com>

refactored curved qr code decoding

79a37c5

APrigarina force-pushed the curved_qrcodes_decoding branch from bb90559 to 79a37c5 Compare September 30, 2020 12:34

rayonnant14 reviewed Sep 30, 2020

View reviewed changes

APrigarina force-pushed the curved_qrcodes_decoding branch from 89e1815 to f67ef55 Compare October 7, 2020 09:56

refactored curved qr code decoding 2nd part

a243657

APrigarina force-pushed the curved_qrcodes_decoding branch from f67ef55 to a243657 Compare October 7, 2020 10:52

allnes reviewed Oct 7, 2020

View reviewed changes

refactored curved qr code decoding 3rd part

5ff79b2

rayonnant14 reviewed Oct 15, 2020

View reviewed changes

refactored curved qr code decoding 4th part

da91274

APrigarina force-pushed the curved_qrcodes_decoding branch from 872654c to 5d3f886 Compare October 16, 2020 10:00

added tests for curved qr code decoding

7380ea4

APrigarina force-pushed the curved_qrcodes_decoding branch from 5d3f886 to 7380ea4 Compare October 19, 2020 08:07

alalek reviewed Oct 19, 2020

View reviewed changes

APrigarina force-pushed the curved_qrcodes_decoding branch 2 times, most recently from a06167d to 064142a Compare October 23, 2020 10:39

refactored curved qr code decoding 5th part

8d05d93

APrigarina force-pushed the curved_qrcodes_decoding branch from 064142a to 8d05d93 Compare October 23, 2020 11:29

allnes approved these changes Oct 23, 2020

View reviewed changes

APrigarina changed the title ~~WIP: detection and decoding of curved QR-codes~~ Detection and decoding of curved QR-codes Oct 23, 2020

alalek assigned allnes Oct 23, 2020

alalek merged commit c71f271 into opencv:3.4 Oct 23, 2020

This was referenced Oct 26, 2020

Merge 3.4 #18662

Merged

objdetect: sporadic crashes in QR code tests #18669

Closed

PeterBowman mentioned this pull request Nov 2, 2020

Develop QR detection module roboticslab-uc3m/vision#6

Closed

alalek mentioned this pull request Nov 27, 2020

Merge 4.x #18951

Merged

AleksandrPanov mentioned this pull request Jun 15, 2023

WIP: Fix segfault in QRCodeDetector.detectAndDecodeCurved #23802

Closed

6 tasks

	vector<vector<float> > QRDecode::computeSpline(vector<int> &x_arr, vector<int> &y_arr)
	vector<vector<float> > QRDecode::computeSpline(const vector<int> &x_arr, const vector<int> &y_arr)

	int min_num_points = 1, max_num_points = cvRound(mean_num_points_in_line / 2.0);
	const int min_num_points = 1, max_num_points = cvRound(mean_num_points_in_line / 2.0);

		if (sqrt(AA + BB) == 0) return 0;
		result = abs((A * a.x - B * a.y + C)) / sqrt(AA + BB);

Uh oh!

Conversation

APrigarina commented Jul 31, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ismael-benito commented Aug 16, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants