Skip to content

actions: Use pytest-rerunfailures for pytest-xdist worker crash#1273

Merged
gentoo-bot merged 1 commit intogentoo:masterfrom
zmedico:bug_924416_pytest-rerunfailures
Feb 14, 2024
Merged

actions: Use pytest-rerunfailures for pytest-xdist worker crash#1273
gentoo-bot merged 1 commit intogentoo:masterfrom
zmedico:bug_924416_pytest-rerunfailures

Conversation

@zmedico
Copy link
Member

@zmedico zmedico commented Feb 14, 2024

Since pytest-xdist workers crash intermittently for the multiprocessing spawn start method, use pytest-rerunfailures to detect and handle this case. Only use pytest-rerunfailures for the spawn start-method since that is the only case where we've observed intermittent pytest-xdist worker crashes, and use --only-rerun 'worker .* crashed while running' to ensure that rerun only triggers for worker crashes.

Bug: https://bugs.gentoo.org/924416

@thesamesam
Copy link
Member

thesamesam commented Feb 14, 2024

bleh. I'm not hard-against it, just worried we're papering over the problem with this. But I suppose it's better than flaky CI which may-or-may not fail for unrelated changes...

I haven't looked at what the actual crash is - do you have any ideas for it yet? (Are we to blame?)

@zmedico
Copy link
Member Author

zmedico commented Feb 14, 2024

bleh. I'm not hard-against it, just worried we're papering over the problem with this. But I suppose it's better than flaky CI which may-or-may not fail for unrelated changes...

It's a pytest-xdist problem though, since it doesn't trigger without it, and pytest-rerunfailures is designed to handle it since pytest-dev/pytest-rerunfailures#158.

I haven't looked at what the actual crash is - do you have any ideas for it yet? (Are we to blame?)

I haven't been able to trigger it locally, so that makes it extra mysterious. Since using a private multiprocessing context for the spawn start method didn't help in #1270, I think we can safely say it's not our fault. It might be pytest-dev/execnet#96 since that's an "intermittent" issue.

The --only-rerun 'worker .* crashed while running' bit makes the workaround not so bad, since rerun is only triggered when absolutely necessary.

@zmedico zmedico force-pushed the bug_924416_pytest-rerunfailures branch from 4cab84d to 5c66967 Compare February 14, 2024 18:46
@zmedico zmedico marked this pull request as ready for review February 14, 2024 18:46
@zmedico zmedico requested a review from thesamesam February 14, 2024 18:47
@thesamesam
Copy link
Member

AH, thank you, sorry, I thought it was an interaction between one of our setup fixtures and multiprocessing, just exposed by pytest-xdist.

Since pytest-xdist workers crash intermittently for the multiprocessing
spawn start method, use pytest-rerunfailures to detect and handle this
case. Only use pytest-rerunfailures for the spawn start-method since
that is the only case where we've observed intermittent pytest-xdist
worker crashes, and use --only-rerun 'worker .* crashed while running'
to ensure that rerun only triggers for worker crashes.

Bug: https://bugs.gentoo.org/924416
See: pytest-dev/execnet#96
Signed-off-by: Zac Medico <zmedico@gentoo.org>
@zmedico zmedico force-pushed the bug_924416_pytest-rerunfailures branch from 5c66967 to ffcb1eb Compare February 14, 2024 19:08
@gentoo-bot gentoo-bot merged commit ffcb1eb into gentoo:master Feb 14, 2024
@zmedico zmedico deleted the bug_924416_pytest-rerunfailures branch February 14, 2024 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants