IPC-D-356A testpoint data support plus RS274-X2 attributes.#161
IPC-D-356A testpoint data support plus RS274-X2 attributes.#161eyal0 wants to merge 16 commits intogerbv:mainfrom
Conversation
|
I run this command to see if each individual commit is passing tests: git rebase main -x "./autogen.sh && ./configure --prefix=/home/esoha/.local && make -j 20 clean && make -j 20 && (make check || (cd test ; ./run_tests.sh --regen ; cd .. ; make check))" |
|
The first failure is in 0bc88c7 found by valgrind. |
| text_color = x2attr_get_field_or_last(0, | ||
| x2attr_get_project_attr_or_default(gerbvProject, "text-color", "")); |
There was a problem hiding this comment.
The memory leak is around here. Maybe text_color needs to be freed?
| return g_strdup(""); | ||
|
|
||
| for (i = 0; i < utf8_array->len; ++i) | ||
| l += _utf8_to_file(((char **)utf8_array->data)[i], NULL); |
There was a problem hiding this comment.
x2attr.c: In function ‘x2attr_utf8_array_to_file’:
x2attr.c:965:11: error: ‘l’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
965 | l += _utf8_to_file(((char **)utf8_array->data)[i], NULL);
| ^~
| // layer. It also indicates how many layers to expect. E.g. if --layers=1,2,3,4 then we know it's a 4-layer | ||
| // board and the bottom layer is #4. | ||
|
|
||
| if (layertype == GERBV_LAYERTYPE_RS274X) { |
There was a problem hiding this comment.
gerbv.c: In function ‘gerbv_open_image’:
gerbv.c:587:8: error: ‘layertype’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
587 | if (layertype == GERBV_LAYERTYPE_RS274X) {
| ^
|
@eyal0 The memory leak is probably not real. For attribute keys and data, I am relying on GLib interning the strings, hence everything is a const char * without needing to free. To quieten valgrind, I will need to see if GLib has an API to tell it to free all the interned strings. We'll have to call that at the end of main(). From a pure library perspective, we might need to define a "cleanup" function, although the lifetime of the interned strings must remain for the entire application, since we can't pick and choose which ones to clean up. The uninit variables are a fair catch. Easy to fix. |
|
BTW, the source line numbers don't seem to line up in the valgrind output. And there's not even any instance of calling g_strndup() from gerbv_set_render_options_for_file(). Not directly, anyway. |
|
Argh! There's no default layertype enum to set! Have to use a cast. |
|
Looks like GLib interning uses reference counting. So technically I would have to unref each string instead of leaving it to a global cleanup function. But maybe there's a deeper problem with that one orphaned byte. I'll run valgrind myself to see if I can't clean it up a bit more. |
|
the memory leak and errors seem real. Here's a solution: diff --git a/src/gerbv.c b/src/gerbv.c
index 3d0b21a..37d4793 100644
--- a/src/gerbv.c
+++ b/src/gerbv.c
@@ -562,6 +562,10 @@ gerbv_open_image(gerbv_project_t *gerbvProject, gchar const* filename, int idx,
else if (gerber_is_rs274d_p(fd)) {
ftype = FILE_TYPE_RS274D;
layertype = GERBV_LAYERTYPE_RS274X;
+ } else {
+ GERB_COMPILE_ERROR(_("Unrecognized layertype in file \"%s\": %s"),
+ filename, strerror(errno));
+ return -1;
}
@@ -1231,10 +1235,10 @@ gerbv_render_all_layers_to_cairo_target_for_vector_output (
void
gerbv_set_render_options_for_file (gerbv_project_t *gerbvProject, gerbv_fileinfo_t *fileInfo, gerbv_render_info_t *renderInfo)
{
- const char * text_min;
- const char * text_max;
- const char * text_mils;
- const char * text_color;
+ char * text_min;
+ char * text_max;
+ char * text_mils;
+ char * text_color;
gfloat tmin, tmax, tmils, pts;
text_min = x2attr_get_field_or_last(0,
@@ -1250,6 +1254,9 @@ gerbv_set_render_options_for_file (gerbv_project_t *gerbvProject, gerbv_fileinfo
sscanf(text_min, "%g", &tmin);
sscanf(text_max, "%g", &tmax);
sscanf(text_mils, "%g", &tmils);
+ g_free(text_min);
+ g_free(text_max);
+ g_free(text_mils);
// Set the target text size (em square) in inches.
renderInfo->textSizeInch = 0.001 * tmils;
@@ -1278,7 +1285,7 @@ gerbv_set_render_options_for_file (gerbv_project_t *gerbvProject, gerbv_fileinfo
text_color,
FALSE,
"text");
-
+ g_free(text_color);
}We need to |
|
There's a few other instances where I forgot to g_free. grep for all |
|
This is the fix for all the valgrind and compiler warnings so far: diff --git a/src/gerbv.c b/src/gerbv.c
index bd5028f..24e5e0f 100644
--- a/src/gerbv.c
+++ b/src/gerbv.c
@@ -565,6 +565,10 @@ gerbv_open_image(gerbv_project_t *gerbvProject, gchar const* filename, int idx,
else if (gerber_is_rs274d_p(fd)) {
ftype = FILE_TYPE_RS274D;
layertype = GERBV_LAYERTYPE_RS274X;
+ } else {
+ GERB_COMPILE_ERROR(_("Unrecognized layertype in file \"%s\": %s"),
+ filename, strerror(errno));
+ return -1;
}
@@ -587,6 +591,7 @@ gerbv_open_image(gerbv_project_t *gerbvProject, gchar const* filename, int idx,
// layer. It also indicates how many layers to expect. E.g. if --layers=1,2,3,4 then we know it's a 4-layer
// board and the bottom layer is #4.
+ attrbuf[0] = 0;
if (layertype == GERBV_LAYERTYPE_RS274X) {
/* Get *all* layers defined, so can map 'b' to max (bottom) layer. 't' is always 1.
Copper layer number is always first char: digit(2) 1..9 or t or b. (Others are not copper).
@@ -684,7 +689,6 @@ gerbv_open_image(gerbv_project_t *gerbvProject, gchar const* filename, int idx,
g_free(ff);
// Now that we know what 'b' layer means, go ahead and parse this instance.
ff = x2attr_get_field_or_default(instance, file_functions, "");
- attrbuf[0] = 0;
ffp = ff;
switch (tolower(*ffp++)) {
case '1': case '2': case '3': case '4': case '5': case '6': case '7': case '8': case '9':
@@ -1256,10 +1260,10 @@ gerbv_set_render_options_for_file (gerbv_project_t *gerbvProject,
gerbv_fileinfo_t *fileInfo,
gerbv_render_info_t *renderInfo)
{
- const char * text_min;
- const char * text_max;
- const char * text_mils;
- const char * text_color;
+ char * text_min;
+ char * text_max;
+ char * text_mils;
+ char * text_color;
gfloat tmin, tmax, tmils, pts;
text_min = x2attr_get_field_or_last(0,
@@ -1275,6 +1279,9 @@ gerbv_set_render_options_for_file (gerbv_project_t *gerbvProject,
sscanf(text_min, "%g", &tmin);
sscanf(text_max, "%g", &tmax);
sscanf(text_mils, "%g", &tmils);
+ g_free(text_min);
+ g_free(text_max);
+ g_free(text_mils);
// Set the target text size (em square) in inches.
renderInfo->textSizeInch = 0.001 * tmils;
@@ -1306,7 +1313,7 @@ gerbv_set_render_options_for_file (gerbv_project_t *gerbvProject,
text_color,
FALSE,
"text");
-
+ g_free(text_color);
}
diff --git a/src/x2attr.c b/src/x2attr.c
index 3df76fe..c8167e7 100644
--- a/src/x2attr.c
+++ b/src/x2attr.c
@@ -960,6 +960,7 @@ x2attr_utf8_array_to_file(const GArray * utf8_array)
if (!utf8_array->len)
return g_strdup("");
+ l = 0;
for (i = 0; i < utf8_array->len; ++i)
l += _utf8_to_file(((char **)utf8_array->data)[i], NULL);
l += utf8_array->len; // for the commas and null term |
2838d44 to
922bcdc
Compare
It's okay, no problem! You did a big job, it's expected to have some fixes to it! |
|
I also needed to set |
922bcdc to
39d3688
Compare
|
Tests pass now I think. |
|
Great! - but what was the issue with |
|
@eyal0 Is there somewhere I can pull the patches you made? I would like to continue to create more test cases for the stuff I was working on. It would be nice to have the most solid base. |
|
Sure, you could build off this PR. Or compare yours to this. This one is like yours but with small changes to make it pass tests. |
|
@eyal0 Great! So, from my Linux box, what's the magic git command to merge this PR? |
|
Nevermind, I created a remote to your branch. For some reason it came up with all conflicts rather than a fast forward. Anyway, trying to run valgrind tests, but the -s option to valgrind is not understood. What is that supposed to do? Can't find it in the valgrind documentation. |
|
Nevermind again, I just upgraded to the latest valgrind. |
|
I have been testing export in RS274-X format. It seems there is a problem with layer states not getting set correctly. For example, the Input: This gets written out as: So ignoring the x2 attributes (and metric units) which I probably should not be inserting, it has reverted to positive polarity. |
|
IPNEG! Oh man. Here's the doc on IPNEG: You're not supposed to make images that are just negatives. gerbv is supporting it, I guess. |
|
@eyal0 could you give an overview of the state of this PR. Are there blockers before merging? |
|
I only did the rebase. But this thing is so massive that I don't even know where to begin! I'm able to look at it more next week, though. Is there a way to break this up into multiple commits so that the review is easier? |
|
@meantaipan I agree with eyal0. It would be great if we could merge the commits in smaller chunks. For example, is 143ae2bfe16bf03725ea1ebd5bc10ba268afccd1 a self contained feature? If so, I would prefer working on getting that merged |
|
Yes, I tried to commit at reasonable points, so hopefully it would make your life easier to merge commits that way. Unfortunately, I did not run the regression tests at each step so no guarantee that you'll be able to test each step that way. So is the main problem that the change is so big that you are getting numerous conflicts with other changes (which I was not aware of, because I based from the main branch)? I feel your pain. I do recall asking up front about how often I should commit my own development steps, and got the impression you all don't want it too granular, so I tried to keep it to fairly complete chunks of work. With the benefit of hindsight, I now see that I should have made it a little finer grained. Sorry about that. Put it down to being a newcomer to open source cooperation. I'm used to having a fairly linear development cycle, since it's normally just me. |
|
@meantaipan not having a separate commit for every little step is fine, we should just try to merge one feature after the other :-) Let's split 143ae2b into a separate pull request and continue from there. |
| _find_value(const x2attr_state_t * s, const char * key) | ||
| { | ||
| unsigned i; | ||
| if (s) |
There was a problem hiding this comment.
I prefer to always use braces. Apple had a very serious bug when they didn't do this: https://nakedsecurity.sophos.com/2014/02/24/anatomy-of-a-goto-fail-apples-ssl-bug-explained-plus-an-unofficial-patch/ . But I realize that some of the code here doesn't do it so...
| *******************/ | ||
|
|
||
| static const char * | ||
| _find_value(const x2attr_state_t * s, const char * key) |
There was a problem hiding this comment.
Place this function after _find_index below and write:
int index = _find_index(s, key);
return index == -1 ? NULL : s->t[i].value;That's shorter, right?
| const char * key; // Pointer to interned UTF-8 null terminated string. | ||
| // Sort by key if 'sorted'. | ||
| const char * value; // Pointer to interned UTF-8 null terminated string | ||
| } t[1]; // 1st of 'size' elements. |
There was a problem hiding this comment.
Why is it called t? Could we be more descriptive with the name? Like attributes or something?
eyal0
left a comment
There was a problem hiding this comment.
That was pretty great! How about we put just this single commit into a PR on its own? I have reviewed it and and I don't see any problems though I have some suggestions that you could consider.
I think that if we commit this one first, on its own, as it's a big one, maybe we can chip away at this PR little-by-little?
| _populate(gpointer key, gpointer value, gpointer user_data) | ||
| { | ||
| x2attr_state_t * s = (x2attr_state_t *)user_data; | ||
| s->t[s->size].key = (const char *)key; |
There was a problem hiding this comment.
We're counting on the pointers for key and value to exist beyond the call of this function, right? Is that a safe bet?
c++ can solve a lot of these problems but we are stuck with c. :-(
There was a problem hiding this comment.
Yes. As it mentions in the struct definition, key and value are both interned strings, so their lifetime will persist until GLib is terminated. Interning makes string compare very fast since just compare pointers, and I'm too lazy to use strdup all over the place.
|
|
||
| if (source) { | ||
| int size = g_hash_table_size(source); | ||
| s = (x2attr_state_t *)g_malloc0(sizeof(x2attr_state_t) + sizeof(const char *)*2*(size-1)); |
There was a problem hiding this comment.
Better to use calculation, like:
s = (x2attr_state_t *)g_malloc0(sizeof(x2attr_state_t) + sizeof(s->t[0])*(size-1));Then you don't need that *2, which is like, a magic number in there.
There was a problem hiding this comment.
Agreed. Don't know why I didn't do that.
|
|
||
| if (source) { | ||
| chain = source->next ? _new_from_copy(source->next) : NULL; | ||
| n = (x2attr_state_t *)g_memdup(source, sizeof(x2attr_state_t) + sizeof(const char *)*2*(source->size-1)); |
There was a problem hiding this comment.
Again the magic number. Seeing as we are doing this calculation a lot, how about making a function that computes the size of the struct based on size. You could call it size_of_x2attr_state_t(int size) or something like that. It just does the calculation. You can use it a few places and if you change it, you will only need to change it in one place.
| if (i < 0) | ||
| return NULL; | ||
| if (i+1 < s->size) | ||
| memmove(s->t + i, s->t + (i+1), sizeof(s->t[0])*(s->size-i-1)); |
There was a problem hiding this comment.
Why not just copy the last element into this position and set sorted to false? It would be a little faster sometimes.
There was a problem hiding this comment.
Yes, I may have overestimated the expense of simply resorting. And looky there: I used sizeof(s->t[0]). Must have wised up...
| x2attr_get_image_attr_or_default(const gerbv_image_t * image, | ||
| const char * key, const char * dflt) | ||
| { | ||
| const char * v = _find_value(image->attrs, g_intern_string(key)); |
There was a problem hiding this comment.
const char * v = x2attr_get_image_attr(image, key);You already defined that function, you might as well use it!
| x2attr_get_aperture_attr_or_default(const gerbv_aperture_t * aperture, | ||
| const char * key, const char * dflt) | ||
| { | ||
| const char * v = _find_value(aperture->attrs, g_intern_string(key)); |
| x2attr_get_net_attr_or_default(const gerbv_net_t * net, | ||
| const char * key, const char * dflt) | ||
| { | ||
| const char * v = _find_chained_value(net->attrs, g_intern_string(key)); |
There was a problem hiding this comment.
same as above.
How come this one is chained and the rest of them aren't? Why don't we always use a chained version?
There was a problem hiding this comment.
The chaining is only used in this case, according to the standard, i.e. for "net" attributes. In some cases, after looking for a net attribute, also look for aperture attributes. IIRC, this applies to polygon regions.
| fprintf(fd, "G01X%07ldY%07ld",xVal,yVal); | ||
| if ((!insidePolygon) && (currentNet->aperture_state == GERBV_APERTURE_STATE_ON)) | ||
| fprintf(fd, "G01%sD02*\n",_format_pair(&xud, currentNet->start_x, currentNet->start_y, FALSE)); | ||
| fprintf(fd, "G01%s",_format_pair(&xud, currentNet->stop_x, currentNet->stop_y, FALSE)); |
There was a problem hiding this comment.
This is a great refactor and I am in favor of it but it could be in its own commit.
…d as RS274-X2. Note that this is only a single image, so tracks will appear crossed since they are possibly taken from multiple copper layers. This will be fixed in next commit. Also, there is currently no annotation of normal gerbers. Only the image struct from the IPC file has the net, refdes, pin etc. attributes. These will need to be matched up by coordinate location to other gerber images.
This allows visual splitting of layer info. e.g. gerbv --ipcd356a-layers=0,1,2 foo.ipc foo.ipc foo.ipc --ipcd356a-tracks=y,y,y will open foo.ipc into three distinct images, for board outline, top and bottom layers. Tracks are shown for all three. Added x2 attributes to the project structure. This is an easily expandable way of passing arbitrary options to file readers etc.
Also added some functionality for text (label) display. Command line args now allow text color and font size ranges to be entered. On zooming in, the text will change sizes within bounds so as to maintain visibility without excessive clutter. Command line arg --layers added to allow specifying file function and related attributes for each RS274-X/D file read in.
This is basically a demo of the new search API. When mouse is near an object, its standard attributes (or label) will be shown in the status bar, next to the selected object (if any).
Process tracks as well as pads; Don't mix track and pad annotations; Properly handle access=00 in IPC by passing bottom layer number. Ignore polygon pours unless from aperture macro.
Mouse hover selection of net object will cause other objects in the same image (layer) to be XOR highlighted if they have matching netname or refdes. Obviously, this needs a source of netname and refdes information, such as IPC-D-356A. Known issues: XOR highlighting defeats itself if e.g. a track is generated with an even number of overlapping strokes. Proper solution requires using Cairo instead of deprecated GDK API.
Also added some test cases for IPC-D-356A files. Changed some golden PNG files to account for pixel differences. May be because I'm on old OS level (Ubuntu 16.04).
39d3688 to
2ec9199
Compare
|
Following up on @eyal0's and @ooxi's earlier requests to break this into smaller pieces — here's an analysis of how the 16 commits decompose into independent, reviewable sub-PRs. Commit Grouping AnalysisThe commits fall into 7 logical groups with clear dependency boundaries: Group 1: RS274-X2 Attribute InfrastructureCommits: Implements the data model and parser for %TF, %TA, %TO, %TD attribute commands. Modifies the RS274-X exporter to emit attributes. Foundation that everything else builds on. This would also close #362. Group 2: IPC-D-356A File ParserCommits: Full parser for IPC-D-356A netlist test data files with CLI options. Addresses #157. Group 3: Geometric Search APICommits: Spatial search over rendered geometry — find objects at a given coordinate. Used for hover identification and annotation matching. Group 4: File Type Detection RefactorFiles: Extracts file type detection into its own module. Standalone cleanup. Group 5: IPC-to-Gerber AnnotationCommits: Matches IPC-D-356A testpoint data to gerber features by coordinate, annotating objects with net names, refdes, and pin data. Group 6: UI EnhancementsCommits: Mouse hover object info in status bar, net name and refdes highlighting across layers. Group 7: Examples, Build System, Man PageCommits: Should be distributed across the relevant sub-PRs rather than bundled separately. Dependency GraphProposed Sub-PR Order
Note: #162 (SEGV on export) is already addressed by PR #379. I'm filing a tracking issue to coordinate this work and give @meantaipan credit for the original implementation. Each sub-PR would be rebased onto current |
From #160 which I accidentally closed and cannot re-open, oops. @meantaipan