Hi,
This issue serves as an aggregation and summary issue for UIA notification issues brought up on GitHub:
Introduction
In Windows 10 Version 1709 (Fall Creators Update), UIA notification event was introduced to let screen readers and other access technologies announce messages from apps. UIA notification event was first supported in NVDA 2018.1 with improements since then. However, a number of issues (open and closed ) have surface that may require redesigning the event handler portion of UIA handler, most notably UIA notification event handler.
Is your feature request related to a problem? Please describe.
NVDA's UIA notification event handler requires that UIA elements report native window handles. However, some notifications, including some crucial ones, are raised by elements without native window handle value set. Some of the example notifications and corresponding GitHub issues include:
Previous efforts to resolve this issue have included runtime Id (#16871 and #16872), GUI thread info (#16888), and consulting the UIA object. Of these, the third option seemed promising until it was determined that without a native window handle, UIA objects cannot even come to life to handle the notification event. Thus, an alternative involving app modules is proposed below.
Describe the solution you'd like
As part of UIA notification event handler, ask app modules for the app where the event originated if it is okay to handle elements without native window handle value set. If yes, a suitable window handle should be used, either the desktop (shell) object window handle or the foreground object window handle. This decision should be logged as a UIA debug log to aid developers (NVDA Core and add-ons) about letting app modules handle this scenario.
Consulting app modules has some advantages, namely:
- Given any process ID (except for a few), an app module can be created and that spans both core and add-ons.
- An app module can inform NVDA if it wants UIA notifications for elements lacking native window handle to be announced.
- We have a precedent with UIA property change event.
The obvious downside is that developers may not know where the notification event might be coming from if the element lacks native window handle value - one can use AccEvent or UIA debug log in NVDA but both require some knowledge of UI Automation and/or NVDA (the case for the latter).
Another issue is that an app (and elements can send messages that are useful and sometimes unnecessary, a good case being messages from File Explorer (explorer.exe) - a useful message is window restored/maximized message whereas a redundant or unnecessary message is keyboard layout switch notification (supporting the latter requires revisiting keyboard input layout announcement implementation). This leads to a classic dilemma: who (or which class method/attribute) should be asked - the app module version of UIA notification event handler or a dedicated method that returns a true-false flag.
Even if this proposal is adopted, it then leads to an important question: which window handle would be best:
- Desktop (shell) window handle: all window traversal routines lead to the shell window (api.getDesktopObject().windowHandle).
- Current foreground window: as a top-level window (or perhaps not), it is a convenient window to use when assigning window handle values.
- Top-level window for the app where the UIA element came from: this is the obvious choice yet may require enumerating top-level windows until the correct window is found while dealing with issues such as multiple top-level windows for a process, invisible top-level window, to name a few.
Describe alternatives you've considered
Either leave the notification event handler implementation as is, ask Microsoft (and potentially other developers) about fixing this issue, or come up with a way to retrieve the native window handle (perhaps using undocumented UIA routines).
Additional context
See the linked issues for background information and past efforts. This proposal is the foundation for resolving the linked issues.
Thanks.
Hi,
This issue serves as an aggregation and summary issue for UIA notification issues brought up on GitHub:
Introduction
In Windows 10 Version 1709 (Fall Creators Update), UIA notification event was introduced to let screen readers and other access technologies announce messages from apps. UIA notification event was first supported in NVDA 2018.1 with improements since then. However, a number of issues (open and closed ) have surface that may require redesigning the event handler portion of UIA handler, most notably UIA notification event handler.
Is your feature request related to a problem? Please describe.
NVDA's UIA notification event handler requires that UIA elements report native window handles. However, some notifications, including some crucial ones, are raised by elements without native window handle value set. Some of the example notifications and corresponding GitHub issues include:
Previous efforts to resolve this issue have included runtime Id (#16871 and #16872), GUI thread info (#16888), and consulting the UIA object. Of these, the third option seemed promising until it was determined that without a native window handle, UIA objects cannot even come to life to handle the notification event. Thus, an alternative involving app modules is proposed below.
Describe the solution you'd like
As part of UIA notification event handler, ask app modules for the app where the event originated if it is okay to handle elements without native window handle value set. If yes, a suitable window handle should be used, either the desktop (shell) object window handle or the foreground object window handle. This decision should be logged as a UIA debug log to aid developers (NVDA Core and add-ons) about letting app modules handle this scenario.
Consulting app modules has some advantages, namely:
The obvious downside is that developers may not know where the notification event might be coming from if the element lacks native window handle value - one can use AccEvent or UIA debug log in NVDA but both require some knowledge of UI Automation and/or NVDA (the case for the latter).
Another issue is that an app (and elements can send messages that are useful and sometimes unnecessary, a good case being messages from File Explorer (explorer.exe) - a useful message is window restored/maximized message whereas a redundant or unnecessary message is keyboard layout switch notification (supporting the latter requires revisiting keyboard input layout announcement implementation). This leads to a classic dilemma: who (or which class method/attribute) should be asked - the app module version of UIA notification event handler or a dedicated method that returns a true-false flag.
Even if this proposal is adopted, it then leads to an important question: which window handle would be best:
Describe alternatives you've considered
Either leave the notification event handler implementation as is, ask Microsoft (and potentially other developers) about fixing this issue, or come up with a way to retrieve the native window handle (perhaps using undocumented UIA routines).
Additional context
See the linked issues for background information and past efforts. This proposal is the foundation for resolving the linked issues.
Thanks.