While working on #221, I noticed that when passing multiple --rerun-except arguments (or, now when passing a list to @pytest.mark.flaky(rerun_except=[...])), all failures are rerun unless they match all of the patterns.

With this test:

def test_fail():
    raise AssertionError()

Running with the example --rerun-except args from the readme gives:

$ pytest --reruns 5 --rerun-except AssertionError --rerun-except OSError
======================== test session starts =========================
platform linux -- Python 3.9.2, pytest-7.4.0, pluggy-1.2.0
rootdir: /tmp/test_rerun
plugins: rerunfailures-11.1.2
collected 1 item

test_rerun.py R                                                [100%]R [100%]R [100%]R [100%]R [100%]F [100%]

============================== FAILURES ==============================
_____________________________ test_fail ______________________________

    def test_fail():
>       raise AssertionError()
E       AssertionError

test_rerun.py:2: AssertionError
====================== short test summary info =======================
FAILED test_rerun.py::test_fail - AssertionError
===================== 1 failed, 5 rerun in 0.02s =====================

I think the intention in this case (as implied by the docs) is that the test should not be rerun, since the failure matches one of the --rerun-except arguments.

Currently the logic in _should_hard_fail_on_error is:

• If neither --only-rerun nor --rerun-except are used, all failures are rerun
• Else, if one or more --only-rerun args is given, a failure is rerun if it matches any --only-rerun pattern
• Else, if one or more --rerun-except args is given, a failure rerun if it fails to match any --rerun-except pattern

The fix for this isn't exactly obvious, as the intended relationship between --only-rerun and --rerun-except is a bit unclear when they are used together. My proposal would be:

• If neither --only-rerun nor --rerun-except are used, all failures are rerun (no change)
• Else, if --only-rerun is used alone, a failure is rerun if it matches any --only-rerun pattern (no change)
• Else, if --rerun-except is used alone, a failure is rerun if it matches none of the --rerun-except patterns (new)
• Else, if --only-rerun and --rerun-except are used together, a failure is only rerun if it matches at least one --only-rerun pattern, AND none of the --rerun-except patterns (new)