Skip to content

Fix tests: "NVDA restarts on quit" and first time opening notepad failing#18631

Merged
SaschaCowley merged 3 commits intomasterfrom
fix-tests
Aug 8, 2025
Merged

Fix tests: "NVDA restarts on quit" and first time opening notepad failing#18631
SaschaCowley merged 3 commits intomasterfrom
fix-tests

Conversation

@seanbudd
Copy link
Copy Markdown
Member

@seanbudd seanbudd commented Aug 6, 2025

Link to issue number:

Fixes some issues from #18597

Summary of the issue:

  • Our timeout for waiting for NVDA exit when a crash is triggered is only 3 seconds. Our timeout is 10 seconds for when testing for testing NVDA to restart normally. This is causing the test to fail regularly
  • Starting notepad is fragile. If notepad doesn't load before we open task switcher, then it may fail to get focus. This results in the first notepad test regularly failing.

Description of user facing changes:

none

Description of developer facing changes:

none

Description of development approach:

  • The timeout for waiting for NVDA to exit after a crash is triggered is increased.
  • When starting notepad, we first wait to see if it gets focus normally. If not, we try to task switch to it. This gives ample time for notepad to load, and avoids task switching if its not necessary

Testing strategy:

Ran each test 50 times

results:
17 failures out of 50 runs of 4 test suites.
This is around the same rate as #18597, however there is an increase to timeout issues (likely due to reduced resources for 50x4 matrix jobs).
It is likely that these timeout issues will occur with a lower frequency on normal builds.

Test failures:

  • "Speech did not finish before timeout" - 7
  • "Timed out waiting for key to be processed" - 4
  • "Timed out waiting for core to sleep again" - 2
  • "Unable to get chrome window" - 1
  • "Unable to get notepad window" - 1
  • "Unable to connect to nvdaSpyLib" - 1
  • "Focus mode: report content editable with details braille Actual != Expected: Focus mode != The word hlght has cmnt cat hlght end has a comment tied to it." - 1

Note that the failures noted in #18597 and targeted by this fix do not occur:
Startup/shutdown:

  • 5x: Ensure NVDA restarts on crash: Old NVDA is still running.
  • 3x: symbolInSpeechUI :: Ensure symbols aren't substituted within NVDA. Unable to focus Notepad.

Known issues with pull request:

see test failures

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

@coderabbitai summary

Copilot AI review requested due to automatic review settings August 6, 2025 08:19
@seanbudd seanbudd requested a review from a team as a code owner August 6, 2025 08:20
@seanbudd seanbudd requested a review from SaschaCowley August 6, 2025 08:20
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes two intermittent test failures in NVDA's system tests by addressing timing issues in the test infrastructure. The changes increase timeout tolerance and improve robustness of the notepad startup process.

  • Increases timeout for waiting for NVDA exit after crash from 3 to 10 seconds
  • Adds fallback mechanism for notepad focus when initial focus attempt fails

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
tests/system/robot/startupShutdownNVDA.py Increases timeout for NVDA crash exit detection from 3 to 10 seconds
tests/system/libraries/NotepadLib.py Adds try-catch block with fallback task switching when notepad fails to get focus initially

@SaschaCowley SaschaCowley merged commit bd87580 into master Aug 8, 2025
22 checks passed
@SaschaCowley SaschaCowley deleted the fix-tests branch August 8, 2025 02:13
@SaschaCowley SaschaCowley added this to the 2025.3 milestone Aug 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants