When there is flaky pytest plugin installed the testing of pytest-rerunfailures fails miserably (almost all tests fails). It is unclear whether the problem is in the pytest-rerunfailures test file or in the pytest-rerunfailures itself, or even in the flaky plugin.