-
-
Notifications
You must be signed in to change notification settings - Fork 767
Investigate flakey system tests #18597
Copy link
Copy link
Open
Labels
component/system-testingp3https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priorityhttps://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#prioritytriagedHas been triaged, issue is waiting for implementation.Has been triaged, issue is waiting for implementation.
Metadata
Metadata
Assignees
Labels
component/system-testingp3https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priorityhttps://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#prioritytriagedHas been triaged, issue is waiting for implementation.Has been triaged, issue is waiting for implementation.
Related issues, PRs or discussions
What is the current state of the codebase?
A stress test was run on our system tests, running each test 40 times:
The following failures occurred:
Generic:
With 14 separate test failures in 40 runs, its expected that the average PR build will fail roughly 1/3 of the time
Why are changes required?
Tests should pass regularly without intermittent failures
What technical changes are required?
There might be an actual issue with NVDA crash handling.
Since only the first test for notepad is failing, starting notepad prior to testing might help.
Speech not finishing before timeout usually involves a loss of focus or freeze, but it could just need a timeout increase.
It seems like a CMD shell spawned by GitHub actions can hijack the focus from notepad and Chrome
Are the proposed technical changes API breaking?
No
Are there potential risks or issues with the proposed implementation?
No