Skip to content

UIA NVDAObject's controllerFor property: catch KeyError seen on Windows 7 due to bad UIA client library marshalling and a limitation in comtypes.#14275

Merged
michaelDCurran merged 2 commits into
masterfrom
i14270
Oct 20, 2022

Conversation

@michaelDCurran

@michaelDCurran michaelDCurran commented Oct 19, 2022

Copy link
Copy Markdown
Member

Link to issue number:

Fixes #14270

Summary of the issue:

when searching in the Windows 7 start menu or navigating in Windows 7 Explorer, the NVDA error sound is heard and the following traceback is printed to the log:

ERROR - eventHandler.executeEvent (19:40:54.124) - MainThread (4480):
error executing event: UIA_elementSelected on <NVDAObjects.Dynamic_UIItemListItemUIA object at 0x06F9EE50> with extra args of {}
Traceback (most recent call last):
  File "eventHandler.pyc", line 300, in executeEvent
  File "eventHandler.pyc", line 101, in __init__
  File "eventHandler.pyc", line 110, in next
  File "NVDAObjects\UIA\__init__.pyc", line 2058, in event_UIA_elementSelected
  File "NVDAObjects\__init__.pyc", line 1189, in event_selection
  File "baseObject.pyc", line 42, in __get__
  File "baseObject.pyc", line 146, in _getPropertyViaCache
  File "NVDAObjects\UIA\__init__.pyc", line 2041, in _get_controllerFor
  File "NVDAObjects\UIA\__init__.pyc", line 987, in _getUIACacheablePropertyValue
  File "monkeyPatches\comtypesMonkeyPatches.pyc", line 32, in __call__
  File "comtypes\automation.pyc", line 516, in __ctypes_from_outparam__
  File "monkeyPatches\comtypesMonkeyPatches.pyc", line 146, in newVARIANT_value_fget
  File "comtypes\automation.pyc", line 467, in _get_value
KeyError: 13

the KeyError is raised by comtypes as it does not know how to unpack a variant of type VT_Unknown | VT_Array.

However, the reason this variant is returned in the first place is because the UIA client library directly returns the provider's VT_Unknown | VT_Array variant on Windows 7, Which is useless for a client.
On Newer Windows versions, the client library correctly marshals this to IUIAutomationElementArray per the UI Automation documentation.

Description of user facing changes

No error sound is heard and traceback is only printed to the log at debugWarning.

Description of development approach

Catch KeyError when fetching the UIA_ControllerFor property.

An alternative could have been to limit the fetching of the controllerfor property to versions above Windows 7, but as official windows 7 UI automation documentation states that controllerFor is supported, it is unclear then as to exactly what versions are affected.

Testing strategy:

Known issues with pull request:

None known.

Change log entries:

None needed.
New features
Changes
Bug fixes
For Developers

Code Review Checklist:

  • Pull Request description:
    • description is up to date
    • change log entries
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • API is compatible with existing add-ons.
  • Documentation:
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • Security precautions taken.

@lukaszgo1

Copy link
Copy Markdown
Contributor

the KeyError is raised by comtypes as it does not know how to unpack a variant of type VT_Unknown | VT_Array.

Are you planning to report this against comtypes?

@michaelDCurran

michaelDCurran commented Oct 20, 2022 via email

Copy link
Copy Markdown
Member Author

@michaelDCurran michaelDCurran marked this pull request as ready for review October 20, 2022 13:12
@michaelDCurran michaelDCurran requested a review from a team as a code owner October 20, 2022 13:12
@michaelDCurran michaelDCurran merged commit 487d41e into master Oct 20, 2022
@michaelDCurran michaelDCurran deleted the i14270 branch October 20, 2022 23:50
@nvaccessAuto nvaccessAuto added this to the 2023.1 milestone Oct 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error sounds when navigating in UIA controls under Windows 7 after merge of PR 14222

4 participants