Conversation
| // [all | test_tag] - (test_tag_skip - test_tag_enable) + test_tag_force | ||
|
|
||
| #define CV_TEST_TAGS_PARAMS \ | ||
| "{ test_tag | |run tests with specified 'tag' markers only (comma ',' separated list) }" \ |
There was a problem hiding this comment.
Aren't there too many controlling options? I believe one or two should be enough for most usecases:
--test_tag_filter=* # default value - all tests enabled
--test_tag_filter=*:-type_64f:size_4k # disable some tags
--test_tag_filter=type_64f:size_4k # enable only specific tagsI also think that we should not disable any tags by default (e.g. 2Gb test on 32-bit platforms). It's up to end user to determine platform limitations and choose appropriate test case set.
There was a problem hiding this comment.
It is due its "default values" for different parameters. Automatic disabling of 2Gb+ (plus mean "more") on 32-bit platforms is useful feature.
It's up to end user to determine platform limitations and choose appropriate test case set.
--gtest_filter can specify tests cases set too. Idea of this proposal to make it flexible.
Implied tags:
mem_6gbimpliesmem_2gb,mem_1gb,mem_512Mb, ...verylongtest implieslongtest tag toosize_fullhdimpliessize_hd- see
applyTestTagImpl()builtins
I don't like gtest filter format - it is not friendly for simple appending (need to parse first on positive and negative parts).
There was a problem hiding this comment.
As I can see implied/direct tags distinction is needed only for printing. We can simplify code greatly if there would be no such separation and all implied tags would be printed too. We could store tags in single std::set or std::map.
Can the default tag filter be set in CommandLineParser?
# no parameter - default values, e.g. mem_2gb
--skip_tags=mem_6gb,verylong # override default values
--skip_tags=none,mem_6gb # override default values, 'none' matches non-tagged tests- added functionality to collect memory usage of OpenCL sybsystem - memory usage of fastMalloc() (disabled by default): * It is not accurate sometimes - external memory profiler is required. - specify common `CV_TEST_TAG_` macros - added applyTestTag() function - write memory usage / enabled tags into Google Tests output file (.xml)
|
👍 |
--gtest_filteris not enough for flexible tests filtering on multiple platforms in multiple configurations: 32-bit / debug / valgrind / coverage / etc.resolves #12710
related #12502 #11523
Patch includes:
fastMalloc()(disabled by default):CV_TEST_TAG_macrosapplyTestTag()functionTODO:
perf_dnn--test_tag_file