Exclude some tests by default

Question

I wish to configure pytest such that it excludes some tests by default; but it should be easy to include them again with some command line option. I only found -k, and I have the impression that that allows complex specifications, but am not sure how to go about my specific need...

The exclusion should be part of the source or a config file (it's permanent - think about very long-running tests which should only be included as conscious choice, certainly never in a build pipeline...).

Bonus question: if that is not possible, how would I use -k to exclude specific tests? Again, I saw hints in the documentation about a way to use not as a keyword, but that doesn't seem to work for me. I.e., -k "not longrunning" gives an error about not being able to find a file "notrunning", but does not exclude anything...

daggett · Accepted Answer · 2021-09-14 23:54:46Z

6

goal: by default skip tests marked as @pytest.mark.integration

conftest.py

import pytest

# function executed right after test items collected but before test run
def pytest_collection_modifyitems(config, items):
    if not config.getoption('-m'):
        skip_me = pytest.mark.skip(reason="use `-m integration` to run this test")
        for item in items:
            if "integration" in item.keywords:
                item.add_marker(skip_me)

pytest.ini

[pytest]
markers =
    integration: integration test that requires environment

now all tests marked as @pytest.mark.integration are skipped unless you use

pytest -m integration

answered Sep 14, 2021 at 23:54

daggett

29k3 gold badges52 silver badges61 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

gold_cy · Accepted Answer · 2021-02-22 12:10:22Z

1

What I have done in the past is create custom markers for these tests that way I can exclude them using the -m command line flag for running the tests. So for example in your pytest.ini file place the following content:

[pytest]
markers =
    longrunning: marks a test as longrunning

Then we just have to mark our long running tests with this marker.

@pytest.mark.longrunning
def test_my_long_test():
    time.sleep(100000)

Then when we run the tests we would do pytest -m "not longrunning" tests/ to exclude them and pytest tests to run everything as intended.

answered Feb 22, 2021 at 12:10

gold_cy

14.2k4 gold badges27 silver badges55 bronze badges

Comments

glades · Accepted Answer · 2024-12-24 09:10:48Z

There's another option which I'd consider more idiomatic:

Just mark all tests that you do want to include with something like auto:

@pytest.mark.auto
def test_function():
   ...

Then place a pytest config file in project root (new version uses pyproject.toml) to run pytest using the -m auto option:

# pyproject.toml
[tool.pytest.ini_options]
minversion = "6.0"
addopts = "-ra -q -m auto"
testpaths = [
    "tests",
]

Now only tests marked auto are picked up. This also avoids the problem that vscode was not able to debug the tests in @dagget's answer, because every GUI invocation of the debugger would also skip the tests (at least that's what I assume was going on).

https://docs.pytest.org/en/latest/reference/customize.html

Sreeram TP · Accepted Answer · 2021-02-22 12:09:35Z

-2

You can use pytest to mark some tests and use -k arg to skip or include them.

For example consider following tests,

import pytest

def test_a():
    assert True

@pytest.mark.never_run
def test_b():
    assert True


def test_c():
    assert True
    
@pytest.mark.never_run
def test_d():
    assert True

you can run pytest like this to run all the tests

pytest

To skip the marked tests you can run pytest like this,

pytest -m "not never_run"

If you want to run the marked tests alone,

pytest -m "never_run"

answered Feb 22, 2021 at 12:09

Sreeram TP

12k8 gold badges65 silver badges121 bronze badges

2 Comments

Drako Over a year ago

May be was not an explicit point in an original question but still - usually you want tests that delete something to be never run unless explicitly called - with this approach if someone will call pytest without args - all destructive tests will be called. Obviously the correct answer is provided by @daggett that ensures the correct behavior and saves you from problems of someone running destructive tests by accident.

Mr. Terminix Oct 28, 2025 at 9:52

I think the OP was asking for a default solution.

Collectives™ on Stack Overflow

Exclude some tests by default

4 Answers 4

Comments

Comments

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related