-
-
Notifications
You must be signed in to change notification settings - Fork 56.5k
OPENCV_PYTEST_FILTER fails to filter as expected; runs too many python tests #19402
Description
OPENCV_PYTEST_FILTER fails to filter as expected due to a logic bug. It runs too many tests due to an errant wildcard.
For example, OPENCV_PYTEST_FILTER=test_cudaa runs 4 tests. I expect it to run 0 tests.
System information (version)
- OpenCV => 4.5.1
- Operating System / Platform => Microsoft Windows [Version 10.0.19042.746]
- Compiler => VS2019 v16.8.4 Community
Detailed description
Too many tests are run due to an errant wildcard in the test.py python test runner. There is also a minor output bug.
opencv/modules/python/test/test.py
Lines 37 to 39 in 1363496
| tests_pattern = os.environ.get('OPENCV_PYTEST_FILTER', 'test_') + '*.py' | |
| if tests_pattern != 'test_*py': | |
| print('Tests filter: {}'.format(tests_pattern)) |
Line 37 has an errant wildcard. It should not be present. Instead, the user should include a wildcard in OPENCV_PYTEST_FILTER if they want such behavior. Line 38 has the minor display bug. The coder intended to check for the default case but failed to include the .
It is easily possible to fix the python code to support both the intended default case and expectant wildcard behavior. Then OPENCV_PYTEST_FILTER would match the wildcard behavior of --gtest_filter. FYI: to achieve full test case name filtering, Python 3.7 is needed and use of unittest.testloader.testNamePatterns.
Steps to reproduce
- Setup Windows and Python 2.7
- Build a release version of OpenCV 4.5.1
- set env var
OPENCV_PYTEST_FILTER=test_cudaa c:\python27\python opencv\modules\ts\misc\run.py . -a -t python2
Result
4 test found and 4 test run
...
Discovering python tests from: C:\repos-nobackup\opencv_contrib\modules\cudaarithm\misc\python\test
found 4 tests
...
test_arithmetic (test_cudaarithm.cudaarithm_test) ... ok
test_convolution (test_cudaarithm.cudaarithm_test) ... ERROR
test_cudaarithm (test_cudaarithm.cudaarithm_test) ... ok
test_logical (test_cudaarithm.cudaarithm_test) ... ok
Expected
No tests found and no tests run.
Issue submission checklist
- I report the issue, it's not a question
- I checked the problem with documentation, FAQ, open issues,
forum.opencv.org, Stack Overflow, etc and have not found solution - I updated to latest OpenCV version and the issue is still there
- There is reproducer code and related data files: videos, images, onnx, etc