Windows 11 modern keyboard: respond to UIA notification event (voice typing and suggested actions)#16014
Merged
seanbudd merged 6 commits intoJan 8, 2024
Conversation
…in Windows 11. Windows 11 modern keyboard features such as voice typing and suggested actions (Windows 11 22H2 and later) use UIA notification event to announce alerts. However, these elements are not focusable, therefore they will be dropped when handled by UIA object base implementation. Therefore, announce modern keyboard alerts and do not let the base UIA NVDA object handle it.
…string when announcing top suggested action (Windows 11 22H2). Modern keyboard uses a really odd text to announce suggested actions (Windows 11 22H2 and later). However this element is in fact a top suggested action. Therefore treat object name as display string and announce it instead (also, keyboard interaction is impossible for suggested actions).
…ts (for annotating events)
seanbudd
reviewed
Jan 8, 2024
Comment on lines
+311
to
+312
| self, obj: NVDAObject, nextHandler: Callable[[], None], | ||
| displayString: str | None = None, activityId: str | None = None, **kwargs |
Member
There was a problem hiding this comment.
please split this into new lines for readibility
…eparate lines. Note from Sean Budd (NV Access): one line per parameter for readability
seanbudd
approved these changes
Jan 8, 2024
Nael-Sayegh
pushed a commit
to Nael-Sayegh/nvda
that referenced
this pull request
Feb 15, 2024
…typing and suggested actions) (nvaccess#16014) Closes nvaccess#16009 Summary of the issue: NVDA does not respond to UIA notification event coming from modern keyboard for announcing voice typing and suggested actions alerts. This is because these parts of modern keyboard are not focusable. Description of user facing changes In Windows 11, NVDA will announce voice typing and suggested actions (22H2 and later) from everywhere including top suggested action when data such as phone numbers is copied to clipboard. Description of development approach Added UIA notification event handler to emoji panel app module. Becasue the base (UIA NVDA object) implementation of notification event checks focused element, notifications form modern keyboard will be missed simply because voice typing and suggested actions are not keyboard focusable. In case of suggested actions, display string is odd, but the top suggestion is the name of the element that raises notification event. Commits: * Modern keyboard (emoji panel and friends): update copyright year * Modern keyboard: introduce UIA notification event to announce alerts in Windows 11. Windows 11 modern keyboard features such as voice typing and suggested actions (Windows 11 22H2 and later) use UIA notification event to announce alerts. However, these elements are not focusable, therefore they will be dropped when handled by UIA object base implementation. Therefore, announce modern keyboard alerts and do not let the base UIA NVDA object handle it. * Modern keyboard/UIA notification: reclassify object name for display string when announcing top suggested action (Windows 11 22H2). Modern keyboard uses a really odd text to announce suggested actions (Windows 11 22H2 and later). However this element is in fact a top suggested action. Therefore treat object name as display string and announce it instead (also, keyboard interaction is impossible for suggested actions). * Modern keyboard: add NVDAObjects.NVDAObject and typing.Callable imports (for annotating events) * Cangelog: add Windows 11 modern keyboard/UIA notification event handler entry * Modern keyboard/UIA notification: split event handler parameters to separate lines. Note from Sean Budd (NV Access): one line per parameter for readability
SaschaCowley
pushed a commit
to SaschaCowley/nvda
that referenced
this pull request
Feb 27, 2024
…typing and suggested actions) (nvaccess#16014) Closes nvaccess#16009 Summary of the issue: NVDA does not respond to UIA notification event coming from modern keyboard for announcing voice typing and suggested actions alerts. This is because these parts of modern keyboard are not focusable. Description of user facing changes In Windows 11, NVDA will announce voice typing and suggested actions (22H2 and later) from everywhere including top suggested action when data such as phone numbers is copied to clipboard. Description of development approach Added UIA notification event handler to emoji panel app module. Becasue the base (UIA NVDA object) implementation of notification event checks focused element, notifications form modern keyboard will be missed simply because voice typing and suggested actions are not keyboard focusable. In case of suggested actions, display string is odd, but the top suggestion is the name of the element that raises notification event. Commits: * Modern keyboard (emoji panel and friends): update copyright year * Modern keyboard: introduce UIA notification event to announce alerts in Windows 11. Windows 11 modern keyboard features such as voice typing and suggested actions (Windows 11 22H2 and later) use UIA notification event to announce alerts. However, these elements are not focusable, therefore they will be dropped when handled by UIA object base implementation. Therefore, announce modern keyboard alerts and do not let the base UIA NVDA object handle it. * Modern keyboard/UIA notification: reclassify object name for display string when announcing top suggested action (Windows 11 22H2). Modern keyboard uses a really odd text to announce suggested actions (Windows 11 22H2 and later). However this element is in fact a top suggested action. Therefore treat object name as display string and announce it instead (also, keyboard interaction is impossible for suggested actions). * Modern keyboard: add NVDAObjects.NVDAObject and typing.Callable imports (for annotating events) * Cangelog: add Windows 11 modern keyboard/UIA notification event handler entry * Modern keyboard/UIA notification: split event handler parameters to separate lines. Note from Sean Budd (NV Access): one line per parameter for readability
Adriani90
pushed a commit
to Adriani90/nvda
that referenced
this pull request
Mar 13, 2024
…typing and suggested actions) (nvaccess#16014) Closes nvaccess#16009 Summary of the issue: NVDA does not respond to UIA notification event coming from modern keyboard for announcing voice typing and suggested actions alerts. This is because these parts of modern keyboard are not focusable. Description of user facing changes In Windows 11, NVDA will announce voice typing and suggested actions (22H2 and later) from everywhere including top suggested action when data such as phone numbers is copied to clipboard. Description of development approach Added UIA notification event handler to emoji panel app module. Becasue the base (UIA NVDA object) implementation of notification event checks focused element, notifications form modern keyboard will be missed simply because voice typing and suggested actions are not keyboard focusable. In case of suggested actions, display string is odd, but the top suggestion is the name of the element that raises notification event. Commits: * Modern keyboard (emoji panel and friends): update copyright year * Modern keyboard: introduce UIA notification event to announce alerts in Windows 11. Windows 11 modern keyboard features such as voice typing and suggested actions (Windows 11 22H2 and later) use UIA notification event to announce alerts. However, these elements are not focusable, therefore they will be dropped when handled by UIA object base implementation. Therefore, announce modern keyboard alerts and do not let the base UIA NVDA object handle it. * Modern keyboard/UIA notification: reclassify object name for display string when announcing top suggested action (Windows 11 22H2). Modern keyboard uses a really odd text to announce suggested actions (Windows 11 22H2 and later). However this element is in fact a top suggested action. Therefore treat object name as display string and announce it instead (also, keyboard interaction is impossible for suggested actions). * Modern keyboard: add NVDAObjects.NVDAObject and typing.Callable imports (for annotating events) * Cangelog: add Windows 11 modern keyboard/UIA notification event handler entry * Modern keyboard/UIA notification: split event handler parameters to separate lines. Note from Sean Budd (NV Access): one line per parameter for readability
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to issue number:
Closes #16009
Summary of the issue:
NVDA does not respond to UIA notification event coming from modern keyboard for announcing voice typing and suggested actions alerts. This is because these parts of modern keyboard are not focusable.
Description of user facing changes
In Windows 11, NVDA will announce voice typing and suggested actions (22H2 and later) from everywhere including top suggested action when data such as phone numbers is copied to clipboard.
Description of development approach
Added UIA notification event handler to emoji panel app module. Becasue the base (UIA NVDA object) implementation of notification event checks focused element, notifications form modern keyboard will be missed simply because voice typing and suggested actions are not keyboard focusable. In case of suggested actions, display string is odd, but the top suggestion is the name of the element that raises notification event.
Testing strategy:
Manual testing:
Prerequisites: Windows 11 22H2 or later, voice typing and suggested actions are on:
Known issues with pull request:
In Windows 11 21H2 (original release), voice typing error might be announced twice - testing shows that this is unique to 21H2 and was fixed in 22H2. Also, if UIA event registration is set to "global", NVDA may announce voice typing alerts several times as both property change (name) and notification could be fired. However, notification event processing is better since it can work across global or selecive UIA event registration.
Code Review Checklist: