How to Xfail multi-parametrised test definitions in py.test?

Question

if anyone knows how do deal with the following issue:

@pytest.mark.parametrize("paramA, paramB", params_variable)
@pytest.mark.parametrize("bool", (True, False))
@pytest.mark.parametrize("size", sizes_variable)
def test_definition(paramA, paramB, boolA, boolB, size):
    assert something

Lets imagine that tests results can be like this:

FAILED test_definition[paramA_1-paramB_2-bool_true-size_4] - AssertionError: assert False

FAILED test_definition[paramA_4-paramB_6-bool_false-size_1] - AssertionError: assert False

How to wrap a specific constellation of parameters to marked them as Xfail? Is it possible?

When I put xfail to some specific parameters, I will marked all Test Cases with this parameter as Xfail - but I only want to mark specific constellation (in the case above - only two cases from a lot of)

billion points for someone who will resolve this problem

MrBean Bremen · Accepted Answer · 2020-08-03 11:22:06Z

3

If you know the parameter combinations where you want to have the xfail marker, you can add the marker in pytest_collection_modifyitems based on the function name:

conftest.py

def pytest_collection_modifyitems(config, items):
    for item in items:
        if item.name in ("test_definition[paramA_1-paramB_2-bool_true-size_4]",
                         "test_definition[paramA_4-paramB_6-bool_false-size_1]"):
            item.add_marker("xfail")

This has to be done in the conftest.py in the same directory as the test, or in a directory above.

answered Aug 3, 2020 at 11:22

MrBean Bremen

17.2k3 gold badges36 silver badges59 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ummagumma Over a year ago

Yeah, you won 1000points. This works perfectly, thanks!

Collectives™ on Stack Overflow

How to Xfail multi-parametrised test definitions in py.test?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related