Composable Shell touch keyboard: support emoji panel and other features in Windows 10 Fall Creators Update and later. re #7273#7378
Conversation
|
Hi, Note that this app module also supports dictation, so we get that for free. Thanks. |
|
Hi, Hmmm, I don't know why, but when I was assigned, the review status was set to "review required" when in fact this was reviewed before. Thanks. |
|
Hi, Yet again, Microsoft has changed emoji panel in 17704, although in a subtle way: when browsing emoji, in addition to element selected event, name change event is fired by an ancestor control which displays selected emoji character and how many items there are in a category. Unfortunately, this element is the one that says things such as "emoji panel is closed", announces categories when browsing them in Redstone 5 and such. Also, name change is fired by emoji grid items when people/skin tone is selected. Thanks. |
…ll Creators Update. re nvaccess#7273. In Windows 10 Fall Creators Update, people using United States English keyboard can use the keyboard to search for and enter emojis (Windows+period or Windows+semicolon). At the moment the app module has been added, with emoji candidates raising UIA element selected event. Things to be aware of: * When the emoji panel first opens, the first emoji isn't announced in some cases. * When searching for emojis, intermediate search results are not announced.
…vaccess#7273. In Fall Creators Update (build 16299), when emoji categories change, the new category fires name change event.
…s#7273. In Windows 10 17000 and later (particularly after 17063 and later), when emoji panel first opens, window opened event is fired. This than allows NVDA to catch this and treat it as an item selected event, thereby allowing the first category to be announced.
… Re nvaccess#7273. In build 17666 and later, emoji panel was redesigned. Instead of refreshing emoji categories when Tab is pressed, one must now select a category before emoji refreshes. This is especially the case for People category where skin tone must be selected afterwards. For backwards compatibility, continue to support original panel design in Version 1709 and 1803.
… cloud clipboard suggestions. Re nvaccess#8189. Modern keyboard is not just used for emoji panel entry: in version 1803, hardware keyboard input suggestions are lited there. In Redstone 5, cloud clipboard entries are shown through this panel. Thus add support for both.
… touch keyboard's name change event. Re nvaccess#7273. In Version 1803 (April 2018 Update), 'Microsoft Candidate UI' is announced whenever candidates appear. As this is anoying, suppress this. Also, reported by a user: under some circumstances, touch keyboard keys fire name change event, which results in NVDA announcing characters. Thus suppress this as well.
…ries. Re nvaccess#7273. Several changes as a result of deploying build 17700 series: * Clipboard: NVDA will no longer announce label for clipboard candidates list. * Emoji panel: NVDA will not respond to verbose name change events fired whenever items are selected.
|
It would be great if you could rewrite your pr description to follow the template. Also, can you provide some clear steps to reproduce in order to test the features? I think I understand how emoji input works, but I'm not sure how to test dictation. |
|
Hi, To test dictation, you need a microphone and:
NVDA will then announce dictated text via name change event. Thanks. |
|
Sometimes I am seeing the following traceback when starting to type a search string into the emoji panel: I think some extra checks for UIA are required. |
…tent generic client is the parent of the 'selected' item. Re nvaccess#7273. Reviewed by Mick Curran (NV Access): sometimes, a traceback that ends with 'AttributeError' on line 37 is shown, caused by the fact that sometimes IAccessible content generic client window becomes the parent of the 'selected' item (tree traversal issue). Thus catch this early by making sure the 'parent' is indeed a UIA control.
Hi,
Revision: revised in July 2018 to base it on the template.
Link to issue number:
Fixes #7273
Summary of the issue:
Support modern keyboard features, including emoji panel, hardware input suggestions, dictation, and cloud clipboard.
Description of how this pull request fixes the issue:
Adds an app module for Composable Shell keyboard. This overlay app provides the following features:
Testing performed:
Searching for emojis, dictating text, cloud clipboard paste, entering text with hardware input suggestion turned on in various Windows 10 releases. Note that because it is an app module, it was tested with Windows 10 App Essentials add-on first.
Known issues with pull request:
None, although features are subject to change.
Change log entry:
Section: new features:
Added support for modern input features in recent Windows 10 releases. These include emoji panel (Fall Creators Update), dictation (Fall Creators Update), hardware keyboard input suggestions (April 2018 Update), and cloud clipboard (RS5).
Technical details:
Thanks.