metrics: nest json results in unique subsections#785
metrics: nest json results in unique subsections#785GabyCT merged 4 commits intokata-containers:masterfrom
Conversation
Separate the test result data out from the generic system level
data by placing all result data into a subsection named by
${TEST_NAME} setting. This also isolates each set of test data
from other tests, which should prevent any future name/type clashes.
Internally the fix is mildly hacky, in that we record the array index
where the system data ends, and later use that index to inject the
new section into the final generation loop. This was the easiest way,
as the library does not support generation of nested fragments
directly through its API. The alternatives would be to either
support nesting (complex), or add a separate API/array to store
system info separate from test info. Right now, this was the easiest
fix.
Fixes: kata-containers#778
Signed-off-by: Graham Whaley <graham.whaley@intel.com>
Now we are nesting JSON results inside a section with the name of the tests, we need to update the checkmetrics `jq` pattern matches to match. Signed-off-by: Graham Whaley <graham.whaley@intel.com>
Now we nest the test results into uniquely named sub-structs of the JSON, we need to un-nest it in the report processing in order to get at the data. We specifically un-nest, rather than just educate the scripts about the sub-section names, as then the inner processing loops can stay fairly generic across all the test results. It just makes the inner code a bit less complex. Signed-off-by: Graham Whaley <graham.whaley@intel.com>
Loading plyr to do the DUT tables was spitting out a warning which ended up in the report. Silence it... Signed-off-by: Graham Whaley <graham.whaley@intel.com>
|
This is related to how we store our data into elasticsearch over on kata-containers/ci#60 Note, marked as DNM right now, because as soon as this lands the metrics CI configs will need updating to take into account the new data format - so we will potentially have a few metrics glitches. As I'm landing this on a Friday evening (yay, that old 🌰 @jodh-intel ;-) ), I thought I'd go for DNM until it gets a bit of review, and then we can do the fast-hand land/fix in a sync'd manner to reduce the fallout. @GabyCT - have a look. The final results (such as the genreport output) do not change, just the internal structure of the final JSON files. Nobody should notice - apart from anybody 'looking' in the data, which is mostly you and I ;-) |
|
lgtm It might be interesting if you could paste an example of the "before" and "after" formats here inside |
|
Will do Monday with some real data. Basically, it goes from... generic system stuff here....
,
"Results" : {
blah specific to the test in question
}to generic system stuff here....
,
"test-name" : {
"Results" : {
blah specific to the test in question
}
} |
|
@grahamwhaley , let me take a look :) |
GabyCT
left a comment
There was a problem hiding this comment.
tested this PR and is looking good
|
@jodh-intel , here it is Before this PR After this PR I did not paste the full |
|
Thanks @grahamwhaley and @GabyCT. |
|
dropping the DNM. The nesting works (I see it in elastic), and I believe is necessary. |
|
even though I don't believe the tests will run this code.... I'm going to fire them so we get a nice clear BIG GREEN BUTTON ;-) |
|
/test |
To avoid any JSON name space/value type clashes, we nest each different tests
results into a section named by the test name.
This also then needs any code or examples that reference that data to be updated.