Potentially, we can avoid using a database using the pytest_report_from_serializable hook in the controller to count failures, and re-configuring the failure history for tests. However, the number of reruns needs to be communicated to the controller before the test run is started. Otherwise, the controller will not be able to determine whether the crash should be re-run or not.

About the db approach: it's as good as it could be, but for the fact that it doesn't work when the nodes are executed on a remote server (as is possible with xdist).

This PR fails in all configurations, could you please have a look.

The reason for the failure is that it uses hooks from pytest-xdist.

The reason for the failure is that it uses hooks from pytest-xdist.

So a release of pytest-xdist is needed? But on the other hand pytest-xdist is not a dependency for pytest-rerunfailures and I do not think that it should become one.

I still plan to complete this, but it will be delayed for a bit. The main problem is that the xdist controller needs to know whether a worker crash should be reported as a rerun or a failure, which requires the workers to communicate this information to the controller before running the test setup (so that if the test setup crashes, the controller can rerun it). Currently this is done through a file, but that appears to only work on windows and it doesn't work if the workers are remote. The new plan is set-up a simple key-value database on the controller that's stored in-memory.

xdist will not be a dependency in the final PR, but this functionality will be included if xdist is detected.

@xoviat Thank you for the clarification.

new plan: start socket server on controller, use as db:

https://stackoverflow.com/questions/7749341/basic-python-client-socket-example

@icemac I believe this may be close to complete. Do you have any questions about this, or should I clean up the commits?

@xoviat The last CI run was a crash. Each and every test failed and had to be killed by GHA after 6 hours. (This does not increase my confidence in this PR.)
What went wrong there and how are you planning to fix it?

The latest run passed (https://github.com/xoviat/pytest-rerunfailures/actions/runs/903077274)

2bff11b wasn't ready yet, I was just pushing changes so that I could see the diffs on github.

Additionally the linter tests fail.

There is one skipped test: Could it be that this is your newly written test?

There is one skipped test: Could it be that this is your newly written test?

The test is skipped because it requires pytest-xdist development branch. What would you like to do about this?

Again, to clarify, this adds new functionality that reruns a test that crashes a worker with pytest-xdist. If a crash occurs without pytest-xdist, there is nothing that can be done, therefore this code is a no-op without pytest-xdist (the next release as of this writing). However, this functionality is important enough to add because it allows a timed-out test (via pytest-timeout) to be rerun (which cannot be currently done).

The test is skipped because it requires pytest-xdist development branch. What would you like to do about this?

Probably this PR could use that development branch for now until it is merged/released.

@icemac Is this a good summary of required changes?

1. add explanation for set/get.
2. add xdist development branch to tests

@xoviat additionally I'd request:

3. fix the linting issues
4. in case of not using xdist make the StatusDB as minimal as possible

Anything else?

The linter still fails, see https://github.com/pytest-dev/pytest-rerunfailures/pull/158/checks?check_run_id=2780766837
You can run the linter tests locally by installing pre-commit.

cc @icemac

ping @icemac

The checks are green now, currently I do not have the time and energy for a final review of this PR. (Maybe @sallner can take over.)
Additionally we need a release of pytest-xdist before this PR can be merged.

@icemac Is there an update on this? I'd be willing to resolve the merge conflicts if I could get an indication that this was moving forward.

@xoviat Thank you for coming back to this PR. Has the needed xdist release already been created?

Yes

Additionally there is a problem the linter shows up.

@icemac Fixes appear to work on my branch. To be clear, pytest-xdist is not required with this pr, but it is required if you want to run all of the tests (in this repository, not in general).

@icemac Done.

Thank you for this PR. 😃

I just created a release including this PR: https://pypi.org/project/pytest-rerunfailures/10.2/