We are currently using

which shows every test that is not passing in the summary. Meaning, skips and xfails are also shown although I'd would wager a guess that no one looks at them. Thus, this PR changes this to only show failures, errors, and xpasses.

Without this, adding markers to the parametrization will render the summary almost useless, since they all get a distinct mention:

import pytest


@pytest.mark.xfail
def test_foo():
    assert False


@pytest.mark.parametrize("baz", [pytest.param(i, marks=[pytest.mark.xfail]) for i in range(10)])
def test_bar(baz):
    assert False

• -ra:

  ======================================== short test summary info =========================================
  XFAIL main.py::test_foo
  XFAIL main.py::test_bar[0]
  XFAIL main.py::test_bar[1]
  XFAIL main.py::test_bar[2]
  XFAIL main.py::test_bar[3]
  XFAIL main.py::test_bar[4]
  XFAIL main.py::test_bar[5]
  XFAIL main.py::test_bar[6]
  XFAIL main.py::test_bar[7]
  XFAIL main.py::test_bar[8]
  XFAIL main.py::test_bar[9]
  ========================================== 11 xfailed in 0.73s ===========================================
  

• -rfEX:

  ========================================== 11 xfailed in 0.72s ===========================================
  

AFAIK, there is currently no occurrence of such pattern in our test suite yet, but will be soon #6653.