Skip to content

Intermittent test failures #13401

@feerrenrut

Description

@feerrenrut

This is an issue that primarily affects developers, the standard issue template doesn't apply well.
Unfortunately there are not reliable steps to reproduce the issue, however here is an example of the test Robot.chromeTests.pr11606 failing.
This is build: pr13391-24822,2c54fafa of master 2c54fafa merged with gestureForDetails b8f79d0c

Fails with error (note the newline):

speech Actual != Expected: out of link != out of link
space

NVDA log for this test, note the timestamps for out of link and space are both at 11:17:05.336

IO - inputCore.InputManager.executeGesture (11:17:05.086) - RF Test Spy Thread (4272):
Input: kb(desktop):rightArrow
DEBUG - editableText.EditableText._hasCaretMoved (11:17:05.321) - MainThread (5616):
Caret move detected using bookmarks. Elapsed: 10 ms
IO - speech.speech.speak (11:17:05.336) - MainThread (5616):
Speaking [LangChangeCommand ('en'), 'out of link']
IO - speech.speech.speak (11:17:05.336) - MainThread (5616):
Speaking [LangChangeCommand ('en'), 'space', EndUtteranceCommand()]
IO - braille.BrailleBuffer.update (11:17:06.273) - MainThread (5616):
Braille regions text: ['NVDA Browser Test Case (783285141) doc ', 'mln editable sect ', 'lst •  lnk A  lnk B ']
DEBUG - external:globalPlugins.speechSpyGlobalPlugin.NVDASpyLib.dump_speech_to_log (11:17:06.429) - RF Test Spy Thread (4272):
dump_speech_to_log.

Further down: all NVDA speech collected by the system test spy is dumped to the log. It does not contain the space speech:

All speech:
[
[''],
[LangChangeCommand ('en'), 'Taskbar  ', IndexCommand(1)],
[LangChangeCommand ('en'), 'NVDA Browser Test Case (783285141)   Google Chrome  ', IndexCommand(2)],
[LangChangeCommand ('en'), 'NVDA Browser Test Case (783285141)  ', 'document  ', IndexCommand(3)],
[LangChangeCommand ('en'), 'Before Test Case Marker  ', IndexCommand(4)],
[LangChangeCommand ('en'), 'Test page load complete  ', IndexCommand(5)],
[LangChangeCommand ('en'), 'Test page load complete  ', IndexCommand(6)],
[LangChangeCommand ('en'), 'Focus mode  ', IndexCommand(7)],
[LangChangeCommand ('en'), 'section  ', 'multi line  ', 'editable  ', 'list  ', 'bullet  ', 'link  ', 'A  ', '  ', 'link  ', 'B  ', IndexCommand(8)],
[LangChangeCommand ('en'), 'out of link  ', IndexCommand(9)]
]

The robot framework test log timestamps show that both the out of link and space speech were received between emulateKeyPress and wait_for_speech_to_finish:

Timestamp Action
11:17:05.086 emulateKeyPress ('rightArrow',)
11:17:06.289 wait_for_speech_to_finish {'speechStartedIndex': 9}
11:17:06.289 get_speech_at_index_until_now (9,)
11:17:06.289 assert speech string matches (ignore case: False): 'out of link
space'
11:17:06.289 Argument types are:
<type 'unicode'>
<type 'unicode'>
11:17:06.289 repr of (speech) actual vs expected (ignore_case=False):
'out of link'
vs
'out of link\nspace'
11:17:06.305 speech Actual != Expected: out of link != out of link
space
11:17:06.305 Traceback (most recent call last):
File "C:\projects\nvda\tests\system\robot\chromeTests.py", line 594, in test_pr11606
"space"
File "C:\projects\nvda\tests\system\libraries\AssertsLib.py", line 28, in strings_match
ignore_case=ignore_case
File "C:\projects\nvda.venv\lib\site-packages\robot\libraries\BuiltIn.py", line 823, in should_be_equal_as_strings
self._should_be_equal(first, second, msg, values, formatter)
File "C:\projects\nvda.venv\lib\site-packages\robot\libraries\BuiltIn.py", line 642, in _should_be_equal
assert_equal(first, second, msg, include_values, formatter)
File "C:\projects\nvda.venv\lib\site-packages\robot\utils\asserts.py", line 182, in assert_equal
_report_inequality(first, second, '!=', msg, values, formatter)
File "C:\projects\nvda.venv\lib\site-packages\robot\utils\asserts.py", line 231, in _report_inequality
raise AssertionError(msg)

The screenshot does not highlight any potential cause.

Metadata

Metadata

Assignees

No one assigned

    Labels

    audience/nvda-devPR or issue is relevant to NVDA / Add-on developersp5https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#prioritytriagedHas been triaged, issue is waiting for implementation.

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions