Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
How to use Python/pytest to organize unit and system tests?
I have a big system which I want to test using Python scripts: using pytest or any other infrastructure.
Some of my tests are of the "pass/fail" type; pytest has great support for these, like collecting, executing, summarizing, etc. Let's call them unit tests.
However, I have different tests, which produce some data about the system, like speed, quality, etc. I want to run these tests manually and examine the results (which are in csv or HTML format). Let's call them system tests.
Currently, I use shell-scripts for them. That means that any common part between these and unit tests has two versions: one written in shell language and another in Python.
How could I organize all my tests using Python?
I run my shell-scripts like this:
test/speed_test_1/run.sh
test/speed_test_2/run_this.sh
test/speed_test_2/run_that.sh
test/quality_test/run.sh
Naively, I'd like to rewrite them in Python and run like this:
test/speed_test_1/run.py
test/speed_test_2/run_this.py
test/speed_test_2/run_that.py
test/quality_test/run.py
However, running Python scripts in a subdirectory is not recommended because of problems with imports. That is — where would I put pytest fixtures and how would the run scripts access them? Not sure there is a good solution for that.
Could I use pytest to run my system tests? I imagine I'd need to change my command lines to something like this:
python -m pytest -k speed_test_1
python -m pytest -k speed_test_2_this
python -m pytest -k speed_test_2_that
python -m pytest -k quality_test
which looks ugly (e.g. there is no tab-completion for test names). And in addition, does pytest really support tests which have output other than pass/fail? I imagine there is something equally ugly I need to do to access my generated reports.
Is there a way I could use pytest or just any Python code to write system tests which could be invoked manually using shell-like syntax?
1 answer
It works fine to have tests in a parallel directory, as long as the Python environment is properly configured. When Pytest is properly installed in an environment, it will also provide a wrapper script; so as long as that environment is active, a pytest command is active and it's not necessary to use python -m pytest.
Suppose we have a directory structure like
.
├── LICENSE.txt
├── pyproject.toml
├── README.md
├── src
│ └── package
│ ├── __init__.py
│ └── code.py
└── test
├── speed_tests
| ├── test_1.py
| └── test_2.py
└── quality_tests
└── test_1.py
To set up testing, we can:
- Create a virtual environment.
- Install pytest in the virtual environment.
- Do an editable install of our own project into the virtual environment.
- Activate the virtual environment.
- Run
pytestfrom the project root.
When writing the test code, make sure to use absolute imports to refer to the code being tested. (Modules that are part of the project can and should continue to refer to each other with relative imports.)
Pytest will be available because the virtual environment is active. It will discover the tests automatically because they are found within the current working directory. The test code will be able to use absolute imports of the main code because the main project is also installed in the same environment. (Relative imports will not work, because the project root will not become a package directory; the entire point of this setup is to not have to care about how the test and package folders are positioned relative to each other on disk.)
This could look like (on Linux, using pip — of course, other options are possible as well):
$ python -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip install -e .
(.venv) $ pip install pytest
(.venv) $ pytest
This bootstraps a separate copy of pip into the virtual environment rather than using the system pip (if any); working around this is possible (and saves installation time and disk space) but requires additional effort.
Writing the actual Python code for testing (and understanding the results, getting results other than pass/fail etc.) is out of scope here; please consult the guides and examples offered in the documentation and then ask separate, more specific questions if necessary.
0 comment threads
0 comment threads