Background:

Windows 11 22H2 (2022 Update/build 22621) introduces Voice Access, a tool to let people dictate text and perform commands using their voices. While Voice Access is active, an overlay window is displayed with controls such as dictated text, microphone state toggle button, and a help button.

Voice Access uses UIA notification event to notify assistive technologies about changes such as microphone state and dictated text. At the moment NVDA does not announce these notifications because it cannot locate the object in question as seen in the log fragment below. Interestingly, NVDA can report notifications if Narrator is also active.

Steps to reproduce:

Prerequisities: Windows 11 22H2 or later on a computer with a microphone.

1. If not done so, open Voice Access and download the speech model when prompted.
2. With Voice Access set up and NVDA running, move system focus to Voice Access window, go to microphone button and press Enter to toggle it.
3. Alternatively, say "microphone sleep" or "microphone active" (or similar commands to toggle the microphone).

Actual behavior:

The following assumes Voice Access window is focused:

 ### Expected behavior:
NVDA announces Voice Access notifications while focused on the Voice Access window without Narrator running. The issue of NVDA not announcing UIA notifications from places other than Voice Access can be dealt with as it has to do with focused app check in default UIA notification event handler in NVDA.

NVDA logs, crash dumps and other attachments:

Context: microphone was toggled from Voice Access by pressing Enter.

Input: kb(laptop):enter
DEBUG - UIAHandler.UIAHandler.IUIAutomationNotificationEventHandler_HandleNotificationEvent (19:02:28.029) - Dummy-4 (13356):
handleNotificationEvent called with notificationKind ActionCompleted, notificationProcessing MostRecent, displayString None, activityID Windows.Shell.VoiceInput.StoppedListening, for element Loading... text with automationID SpeechTextBlock, className TextBlock and frameworkID XAML
DEBUG - NVDAObjects.UIA.UIA.init (19:02:28.029) - Dummy-4 (13356):
No windowHandle for UIA NvDAObject. Searching UIA element ancestry for nearest windowHandle
DEBUG - UIAHandler.UIAHandler.getNearestWindowHandle (19:02:28.029) - Dummy-4 (13356):
Locating nearest ancestor windowHandle for element Loading... text with automationID SpeechTextBlock, className TextBlock and frameworkID XAML
DEBUGWARNING - UIAHandler.UIAHandler.getNearestWindowHandle (19:02:28.030) - Dummy-4 (13356):
error walking up to an element with a valid windowHandle
Traceback (most recent call last):
File "UIAHandler_init_.pyc", line 1338, in getNearestWindowHandle
ctypes.COMError: (-2147467259, 'Unspecified error', (None, None, None, 0, None))
DEBUGWARNING - NVDAObjects.call (19:02:28.030) - Dummy-4 (13356):
Invalid NVDAObject: no windowHandle
Traceback (most recent call last):
File "NVDAObjects_init.pyc", line 96, in call
File "NVDAObjects\UIA_init_.pyc", line 1553, in init
NVDAObjects.InvalidNVDAObject: no windowHandle
DEBUG - UIAHandler.UIAHandler.IUIAutomationNotificationEventHandler_HandleNotificationEvent (19:02:28.030) - Dummy-4 (13356):
HandleNotificationEvent: Ignoring because no object: NotificationProcessing=3 displayString=None activityId=Windows.Shell.VoiceInput.StoppedListening
DEBUG - UIAHandler.UIAHandler.IUIAutomationNotificationEventHandler_HandleNotificationEvent (19:02:28.030) - Dummy-4 (13356):
handleNotificationEvent called with notificationKind ActionCompleted, notificationProcessing ImportantAll, displayString Microphone sleep, activityID Windows.Shell.VoiceAccess, for element Microphone sleep button with automationID MicrophoneButtonControl, className Button and frameworkID XAML
DEBUG - NVDAObjects.UIA.UIA.init (19:02:28.030) - Dummy-4 (13356):
No windowHandle for UIA NvDAObject. Searching UIA element ancestry for nearest windowHandle
DEBUG - UIAHandler.UIAHandler.getNearestWindowHandle (19:02:28.030) - Dummy-4 (13356):
Locating nearest ancestor windowHandle for element Microphone sleep button with automationID MicrophoneButtonControl, className Button and frameworkID XAML
DEBUGWARNING - UIAHandler.UIAHandler.getNearestWindowHandle (19:02:28.032) - Dummy-4 (13356):
error walking up to an element with a valid windowHandle
Traceback (most recent call last):
File "UIAHandler_init_.pyc", line 1338, in getNearestWindowHandle
ctypes.COMError: (-2147467259, 'Unspecified error', (None, None, None, 0, None))
DEBUGWARNING - NVDAObjects.call (19:02:28.032) - Dummy-4 (13356):
Invalid NVDAObject: no windowHandle
Traceback (most recent call last):
File "NVDAObjects_init.pyc", line 96, in call
File "NVDAObjects\UIA_init_.pyc", line 1553, in init
NVDAObjects.InvalidNVDAObject: no windowHandle
DEBUG - UIAHandler.UIAHandler.IUIAutomationNotificationEventHandler_HandleNotificationEvent (19:02:28.032) - Dummy-4 (13356):
HandleNotificationEvent: Ignoring because no object: NotificationProcessing=0 displayString=Microphone sleep activityId=Windows.Shell.VoiceAccess

System configuration

NVDA installed/portable/running from source:

Installed

NVDA version:

alpha-32918,3aafc821

Windows version:

Windows 11 24H2 preview (build 26100.1150)

Name and version of other software in use when reproducing the issue:

Voice Access 24H2 (10.0.26100)

Other information about your system:

Used as a development workstation, subscribed to Windows Insider/release preview channel

Other questions

Does the issue still occur after restarting your computer?

Yes

Have you tried any other versions of NVDA? If so, please report their behaviors.

Reproducible in 2024.1, 2024.2, 2024.3 beta

If NVDA add-ons are disabled, is your problem still occurring?

Yes

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

Not applicable

Additional context:

I suspect something odd is happening when fetching UIA element window handle while trying to handle notification event form Voice Access (NVDA needs window handle when constructing objects). At least resolving this will allow NVDA to announce things while focused on Voice Access. Getting NVDA to announce Voice Access status from everywhere can be simple as creating an app module to announce notification display string and no more (because focused app is not Voice Access, thus NVDA will ignore notification event by default).

The biggest impact is that NVDA users will be able to use Voice Access more fully, including getting notificatoins about microphone toggle, error alerts, among others. I provided primitive support for Voice Access (at least letting NVDA announce microphone toggle from the keyboard) via an add-on, but resolving the root issue (UIA notification event handling) would be a more effective solution.

Thanks.