Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in May 2019 Update#9205
Conversation
|
|
||
| def isGoodUIAWindow(self,hwnd): | ||
| # #9204: In build 18305 and later, it is the shell that raises window open event for emoji panel. | ||
| if winVersion.winVersion.build > 17763 and winUser.getClassName(hwnd) == "ApplicationFrameWindow": |
There was a problem hiding this comment.
Just to make sure, why do you have to mark this (i.e. ApplicationFrameWindow) as a good UIA window? Doesn't it have a native UIA implementation? Given that an UIA event is fired on it, I'd say it does have one?
| def event_UIA_window_windowOpen(self, obj, nextHandler): | ||
| # Send UIA window open event to input app window. | ||
| inputPanelWindow = obj.firstChild | ||
| if isinstance(obj, UIA) and obj.UIAElement.cachedClassName == "ApplicationFrameWindow" and inputPanelWindow and inputPanelWindow.appModule.appName == "windowsinternal_composableshell_experiences_textinput_inputapp": |
There was a problem hiding this comment.
Could you split this into multiple lines? Also, I'd prefer checking the validity of obj before (and not after) getting the first child of it and checking prerequisites on that.
| inputPanelWindow = obj.firstChild | ||
| if isinstance(obj, UIA) and obj.UIAElement.cachedClassName == "ApplicationFrameWindow" and inputPanelWindow and inputPanelWindow.appModule.appName == "windowsinternal_composableshell_experiences_textinput_inputapp": | ||
| inputPanelWindow.appModule.event_UIA_window_windowOpen(inputPanelWindow, nextHandler) | ||
| nextHandler() |
There was a problem hiding this comment.
You're providing nextHandler to inputPanelWindow.appModule.event_UIA_window_windowOpen and than calling it again here. I'm afraid this can cause weird behaviour. I think I'd prefer using eventHandler.executeEvent on the object, in which case the appModule is triggered automatically.
|
Hi,
Thanks. |
| if isinstance(obj, UIA) and obj.UIAElement.cachedClassName == "ApplicationFrameWindow": | ||
| inputPanelWindow = obj.firstChild | ||
| if inputPanelWindow and inputPanelWindow.appModule.appName == "windowsinternal_composableshell_experiences_textinput_inputapp": | ||
| eventHandler.executeEvent("UIA_window_windowOpen", inputPanelWindow) |
There was a problem hiding this comment.
Note that you no longer call nextHandler here. I guess that doesn't matter as it is very unlikely that there'll be a next handler in this particular case. Personally, I'd still add it though
|
Hi, perhaps because of a previous comment about event execution. Thanks.
From: Leonard de Ruijter <notifications@github.com>
Sent: Monday, January 28, 2019 12:20 PM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Author <author@noreply.github.com>
Subject: Re: [nvaccess/nvda] Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in April 2019 Update (#9205)
@LeonarddeR commented on this pull request.
_____
In source/appModules/explorer.py <#9205 (comment)> :
@@ -299,3 +300,16 @@ def event_gainFocus(self, obj, nextHandler):
return
nextHandler()
+
+ def isGoodUIAWindow(self,hwnd):
+ # #9204: In build 18305 and later, it is the shell that raises window open event for emoji panel.
+ if winVersion.winVersion.build > 17763 and winUser.getClassName(hwnd) == "ApplicationFrameWindow":
+ return True
+ return False
+
+ def event_UIA_window_windowOpen(self, obj, nextHandler):
+ # Send UIA window open event to input app window.
+ if isinstance(obj, UIA) and obj.UIAElement.cachedClassName == "ApplicationFrameWindow":
+ inputPanelWindow = obj.firstChild
+ if inputPanelWindow and inputPanelWindow.appModule.appName == "windowsinternal_composableshell_experiences_textinput_inputapp":
+ eventHandler.executeEvent("UIA_window_windowOpen", inputPanelWindow)
Note that you no longer call nextHandler here. I guess that doesn't matter as it is very unlikely that there'll be a next handler in this particular case. Personally, I'd still add it though
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#9205 (review)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkIdjnO9Sug06cpEn--gXiXmXxqWFks5vH1tZgaJpZM4aUm81> .
|
|
@josephsl: Is this ready for another round of review? |
|
Hi, yes, I think it was sufficiently tested via my add-on to warrant another round. Thanks for reminding me about this.
From: Leonard de Ruijter <notifications@github.com>
Sent: Wednesday, April 24, 2019 10:15 PM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in May 2019 Update (#9205)
@josephsl <https://github.com/josephsl> : Is this ready for another round of review?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#9205 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEC5EMQQ3ORBBZS4LVTPSE44FANCNFSM4GSSN42Q> .
|
LeonarddeR
left a comment
There was a problem hiding this comment.
While this looks ok from a code perspective, it does not improve anything for me when running from source and opening the emoji panel.
STR:
- Open notepad
- Open de emoji panel
Expected: NVDA announces that the emoji panel is opened
Actual: NVDA remains silent. I can walk trhough the panel with the arrow keys, and after that, pressing escape announces that the panel is closed.
|
Hi, most likely because Python 2.7 will run under Windows 8 compatibility mode – source code copy cannot embed manifest info while running. Speaking of this pull request, I’ll need to do something about an important bug I found regarding object hierarchy differences for cloud clipboard. Thanks.
From: Leonard de Ruijter <notifications@github.com>
Sent: Friday, May 10, 2019 3:02 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in May 2019 Update (#9205)
@LeonarddeR commented on this pull request.
While this looks ok from a code perspective, it does not improve anything for me when running from source and opening the emoji panel.
STR:
1. Open notepad
2. Open de emoji panel
Expected: NVDA announces that the emoji panel is opened
Actual: NVDA remains silent. I can walk trhough the panel with the arrow keys, and after that, pressing escape announces that the panel is closed.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#9205 (review)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEH3WO3J2C4SVESGVITPUVBZHANCNFSM4GSSN42Q> .
|
|
Hi, I assume that explorer.py/isGoodUIAWindow is the only thing left to investigate (due to that being marked as "changes requested")? Regarding the bug I found: caused by lack of clipboard tips text when items show up under some circumstances (caught it and told NVDA to descend into different elements based on this scenario). Thanks. |
|
Note that the opening of emoji and cloud clipboard panels is not announced for me in this configuration:
Is there something language-dependent that is being tested which needs to match for different languages? |
|
Hi, not language specific from what I can tell – try restarting NVDA and see if that makes a difference (I
M thinking it may have to do with UIA, again). Thanks.
From: Marco Zehe <notifications@github.com>
Sent: Wednesday, May 22, 2019 3:32 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in May 2019 Update (#9205)
Note that the opening of emoji and cloud clipboard panels is not announced for me in this configuration:
* NVDA 2019.1.1
* WinTenApps add-on version 2019.5, release.
* Windows 10 19h1 (build 18362.116), German UI.
Is there something language-dependent that is being tested which needs to match for different languages?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#9205?email_source=notifications&email_token=AB4AXEA5TKDJABSTCEADCSTPWUOLDA5CNFSM4GSSN422YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV6T5ZQ#issuecomment-494747366> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEGUX3W6VJ7XKN7JIQDPWUOLDANCNFSM4GSSN42Q> .
|
|
No, doesn't make a difference. The window is called "Microsoft Text Input Application", and with cloud clipboard, it speaks "Clipboard" (or the German equivalent of it), but with WindowsKey+., NVDA remains silent. With Windows+V, focus seems to actually go to the static text that says "clipboard", with Windows+., it technically remains on the textbox, and only after I press arrow keys does the list gain focus. |
|
Hi, in this case, I need a debug log please (restart with debug logging enabled, which will then cause Windows 10 App Essentials to log events). Thanks.
From: Marco Zehe <notifications@github.com>
Sent: Wednesday, May 22, 2019 8:05 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in May 2019 Update (#9205)
No, doesn't make a difference. The window is called "Microsoft Text Input Application", and with cloud clipboard, it speaks "Clipboard" (or the German equivalent of it), but with WindowsKey+., NVDA remains silent. With Windows+V, focus seems to actually go to the static text that says "clipboard", with Windows+., it technically remains on the textbox, and only after I press arrow keys does the list gain focus.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#9205?email_source=notifications&email_token=AB4AXEFJD7RUZQUKE3KFY5LPWVOIBA5CNFSM4GSSN422YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV7LHRI#issuecomment-494842821> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEGYTBOBDXX46VT6JADPWVOIBANCNFSM4GSSN42Q> .
|
|
Here you go: IO - inputCore.InputManager.executeGesture (17:19:56.506): The first escape merely turned off focus mode, I had to press it a second time to get rid of the panel. |
|
Hi, I don’t see the info I’m looking for – if Windows 10 App Essentials is enabled and NVDA is restarted with debug logging on, there will be a series of statements that starts with “W10:”. You say the add-on is installed? Chances are that it is turned off for some reason. Thanks.
From: Marco Zehe <notifications@github.com>
Sent: Wednesday, May 22, 2019 8:26 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in May 2019 Update (#9205)
Here you go:
IO - inputCore.InputManager.executeGesture (17:19:56.506):
Input: kb(laptop):windows+.
DEBUG - NVDAObjects.NVDAObject._get_placeholder (17:19:56.957):
Potential unimplemented child class: <NVDAObjects.Dynamic_EditableTextWithoutAutoSelectDetectionUIA object at 0x04F26E90>
IO - braille.BrailleBuffer.update (17:19:56.959):
Braille regions text: [u'\ue094']
IO - braille.BrailleHandler.update (17:19:56.959):
Braille window dots: 6 347 1346 15 346 246 1456 6
IO - speech.speak (17:19:56.963):
Speaking [LangChangeCommand ('de_DE'), u'\ue094']
IO - braille.BrailleBuffer.update (17:19:56.963):
Braille regions text: [u'\ue094']
IO - braille.BrailleHandler.update (17:19:56.963):
Braille window dots: 6 347 1346 15 346 246 1456 6
IO - braille.BrailleHandler.update (17:20:00.957):
Braille window dots: 6 347 1346 15 346 246 1456 6
IO - inputCore.InputManager.executeGesture (17:20:02.989):
Input: kb(laptop):escape
IO - braille.BrailleBuffer.update (17:20:03.030):
Braille regions text: [u'erf Ung\xfcltig mz ef Leave a comment ef ende']
IO - braille.BrailleHandler.update (17:20:03.030):
Braille window dots: 14 135 134 134 15 1345 2345 - - - 15 124 -
IO - braille.BrailleHandler.update (17:20:03.032):
Braille window dots: 14 135 134 134 15 1345 2345 - - - 15 124 -
IO - inputCore.InputManager.executeGesture (17:20:05.155):
Input: kb(laptop):escape
DEBUGWARNING - Python warning (17:20:05.186):
C:\Program Files (x86)\NVDA\library.zip\keyboardHandler.py:524: DeprecationWarning: Yield() is deprecated
The first escape merely turned off focus mode, I had to press it a second time to get rid of the panel.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#9205?email_source=notifications&email_token=AB4AXEDGROXZR6D2GN3MGXDPWVQ2HA5CNFSM4GSSN422YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV7NQFI#issuecomment-494852117> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEBQYUQNMM2WSCEXD4LPWVQ2HANCNFSM4GSSN42Q> .
|
|
Hm, it is definitely showing up as active in the Manage Add-Ons dialog. And the level is set to debug, as you can see above from the other two messages. This is all the info that was in the log from the moment I pressed WindowsKey+. to the second press of Escape, or rather, the moment I pressed NVDA+F1 to open log viewer. |
|
The add-on is definitely active. From the startup sequence of NVDA: Found add-on wintenApps - 19.05. Requires API: (2018, 4, 0). Last-tested API: (2019, 2, 0) And further below: DEBUG - addonHandler.Addon.addToPackagePath (17:33:09.348): I also ruled out other add-ons that could potentially interfere. |
|
Hi, okay, got it. For this one, it isn’t just enough to configure NVDA to use debug logging level – you must restart NVDA with debug logging turned on (NVDA+Q, then choose the last option). Only then the add-on will display the info we need (sorry for a lot of circles on this). Thanks.
From: Marco Zehe <notifications@github.com>
Sent: Wednesday, May 22, 2019 8:32 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in May 2019 Update (#9205)
Hm, it is definitely showing up as active in the Manage Add-Ons dialog. And the level is set to debug, as you can see above from the other two messages. This is all the info that was in the log from the moment I pressed WindowsKey+. to the second press of Escape, or rather, the moment I pressed NVDA+F1 to open log viewer.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#9205?email_source=notifications&email_token=AB4AXEA6FQWKL2IADKZDCNLPWVRPVA5CNFSM4GSSN422YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV7OCJY#issuecomment-494854439> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEA6RXNAF5J344OS6ITPWVRPVANCNFSM4GSSN42Q> .
|
|
Hi, ah, this is much better – I’m taking my previous statement back. Does restarting your computer resolve this? Sometimes restarting or logging out and back in helps this kind of situations. Thanks.
From: Marco Zehe <notifications@github.com>
Sent: Wednesday, May 22, 2019 8:39 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in May 2019 Update (#9205)
The add-on is definitely active. From the startup sequence of NVDA:
Found add-on wintenApps - 19.05. Requires API: (2018, 4, 0). Last-tested API: (2019, 2, 0)
DEBUG - core.main (17:33:08.936):
Initializing appModule Handler
DEBUG - addonHandler.Addon.addToPackagePath (17:33:08.937):
Addon wintenApps added to appModules package path
And further below:
DEBUG - addonHandler.Addon.addToPackagePath (17:33:09.348):
Addon wintenApps added to globalPlugins package path
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.init (17:33:09.354):
W10: adding additional events
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.init (17:33:09.358):
W10: added event ID 20036, assigned to UIA_activeTextPositionChanged
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.init (17:33:09.361):
W10: added event ID 20026, assigned to UIA_dragStart
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.init (17:33:09.365):
W10: added event ID 20027, assigned to UIA_dragCancel
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.init (17:33:09.369):
W10: added event ID 20028, assigned to UIA_dragComplete
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.init (17:33:09.369):
W10: adding class name Popup to known dialog class names
I also ruled out other add-ons that could potentially interfere.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#9205?email_source=notifications&email_token=AB4AXEGRQNPIQWMWNKJXVV3PWVSKHA5CNFSM4GSSN422YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV7OYWI#issuecomment-494857305> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEEOYPP5FGJIDIW4UDLPWVSKHANCNFSM4GSSN42Q> .
|
|
OK, here's the info. And no, restarting the computer or logging out and back in does not solve this. IO - inputCore.InputManager.executeGesture (17:41:56.526): |
|
Hi, at least events are working. I’ve seen this before, which indicates that NVDA isn’t moving to the right control. Can you try with latest alpha along with latest development snapshot version of this add-on? If the snapshot fixes this, then it is likely that the latest commit for this pull request might fix this. Thanks.
From: Marco Zehe <notifications@github.com>
Sent: Wednesday, May 22, 2019 8:46 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in May 2019 Update (#9205)
OK, here's the info. And no, restarting the computer or logging out and back in does not solve this.
IO - inputCore.InputManager.executeGesture (17:41:56.526):
Input: kb(laptop):windows+.
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.uiaDebugLogging (17:41:57.316):
W10: UIA object: <NVDAObjects.UIA.UIA object at 0x04FBE350>, name: , event: windowOpen, app module: <'explorer' (appName u'explorer', process ID 4908) at address 4fbe9b0>, automation Id: , class name: ApplicationFrameWindow
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.uiaDebugLogging (17:41:57.362):
W10: UIA object: <NVDAObjects.UIA.UIA object at 0x04FA5E50>, name: Microsoft Text Input Application, event: windowOpen, app module: <'windowsinternal_composableshell_experiences_textinput_inputapp' (appName u'windowsinternal_composableshell_experiences_textinput_inputapp', process ID 15484) at address 4f679d0>, automation Id: , class name: Windows.UI.Core.CoreWindow
DEBUG - NVDAObjects.NVDAObject._get_placeholder (17:41:57.394):
Potential unimplemented child class: <NVDAObjects.Dynamic_EditableTextWithoutAutoSelectDetectionUIA object at 0x04F75210>
IO - braille.BrailleBuffer.update (17:41:57.395):
Braille regions text: [u'\ue094']
IO - braille.BrailleHandler.update (17:41:57.397):
Braille window dots: 6 347 1346 15 346 246 1456 6
IO - speech.speak (17:41:57.400):
Speaking [LangChangeCommand ('de_DE'), u'\ue094']
IO - braille.BrailleBuffer.update (17:41:57.401):
Braille regions text: [u'\ue094']
IO - braille.BrailleHandler.update (17:41:57.401):
Braille window dots: 6 347 1346 15 346 246 1456 6
IO - inputCore.InputManager.executeGesture (17:42:00.697):
Input: kb(laptop):rightArrow
DEBUGWARNING - Python warning (17:42:00.727):
C:\Program Files (x86)\NVDA\library.zip\api.py:274: DeprecationWarning: Yield() is deprecated
DEBUG - editableText.EditableText._hasCaretMoved (17:42:00.898):
Caret didn't move before timeout. Elapsed: 100 ms
IO - speech.speak (17:42:00.905):
Speaking [LangChangeCommand ('de_DE'), u'Leer']
IO - braille.BrailleBuffer.update (17:42:00.947):
Braille regions text: [u'Comment body erf Ung\xfcltig mz ef Leave a comment ', ' ']
IO - braille.BrailleHandler.update (17:42:00.948):
Braille window dots: -
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.uiaDebugLogging (17:42:00.953):
W10: UIA object: <NVDAObjects.UIA.ListItem object at 0x04F67F70>, name: 😺, event: elementSelected, app module: <'windowsinternal_composableshell_experiences_textinput_inputapp' (appName u'windowsinternal_composableshell_experiences_textinput_inputapp', process ID 15484) at address 4f679d0>, automation Id: , class name: GridViewItem
DEBUG - NVDAObjects.NVDAObject._get_placeholder (17:42:01.013):
Potential unimplemented child class: <NVDAObjects.UIA.ListItem object at 0x04F67F70>
IO - braille.BrailleBuffer.update (17:42:01.025):
Braille regions text: [u'\U0001f63a 2 von 54 E2']
IO - braille.BrailleHandler.update (17:42:01.026):
Braille window dots: 6 347 1346 145 1256 146 145 6 6 347 1346 145 15 146 1 6
IO - speech.speak (17:42:01.035):
Speaking [LangChangeCommand ('de_DE'), u'Ebene 2 \U0001f63a 2 von 54']
IO - braille.BrailleBuffer.update (17:42:01.042):
Braille regions text: [u'\U0001f63a 2 von 54 E2']
IO - braille.BrailleHandler.update (17:42:01.042):
Braille window dots: 6 347 1346 145 1256 146 145 6 6 347 1346 145 15 146 1 6
IO - braille.BrailleBuffer.update (17:42:01.046):
Braille regions text: [u'\U0001f63a 2 von 54 E2']
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.uiaDebugLogging (17:42:01.073):
W10: UIA object: <NVDAObjects.UIA.UIA object at 0x04F67EB0>, name: 😺 ausgewählt. 54 Elemente sind enthalten., event: nameChange, app module: <'windowsinternal_composableshell_experiences_textinput_inputapp' (appName u'windowsinternal_composableshell_experiences_textinput_inputapp', process ID 15484) at address 4f679d0>, automation Id: TEMPLATE_PART_ExpressionGroupedFullView, class name:
IO - braille.BrailleHandler.update (17:42:05.032):
Braille window dots: 6 347 1346 145 1256 146 145 6 6 347 1346 145 15 146 1 6
IO - inputCore.InputManager.executeGesture (17:42:06.532):
Input: kb(laptop):escape
IO - braille.BrailleBuffer.update (17:42:06.578):
Braille regions text: [u'erf Ung\xfcltig mz ef Leave a comment ef ende']
IO - braille.BrailleHandler.update (17:42:06.578):
Braille window dots: 14 135 134 134 15 1345 2345 - - - 15 124 -
IO - braille.BrailleHandler.update (17:42:06.581):
Braille window dots: 14 135 134 134 15 1345 2345 - - - 15 124 -
IO - inputCore.InputManager.executeGesture (17:42:07.431):
Input: kb(laptop):escape
DEBUG - external:globalPlugins.wintenObjs.GlobalPlugin.uiaDebugLogging (17:42:07.523):
W10: UIA object: <NVDAObjects.UIA.UIA object at 0x04FA5E50>, name: Emoji-Bereich ist geschlossen., event: nameChange, app module: <'windowsinternal_composableshell_experiences_textinput_inputapp' (appName u'windowsinternal_composableshell_experiences_textinput_inputapp', process ID 15484) at address 4f679d0>, automation Id: TEMPLATE_PART_ExpressionGroupedFullView, class name:
IO - speech.speak (17:42:07.535):
Speaking [LangChangeCommand ('de_DE'), u'Emoji-Bereich ist geschlossen.']
IO - braille.BrailleBuffer.update (17:42:07.536):
Braille regions text: [u'Emoji-Bereich ist geschlossen.']
IO - braille.BrailleHandler.update (17:42:07.536):
Braille window dots: 157 134 135 245 24 36 127 15 1235 15 24 14 125 -
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#9205?email_source=notifications&email_token=AB4AXEGIXCVNL4MDRZRP67TPWVTFHA5CNFSM4GSSN422YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV7PPDA#issuecomment-494860172> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEGILVBKPUXMFREPKNDPWVTFHANCNFSM4GSSN42Q> .
|
|
No luck, opening still doesn't get announced, and closing only if I change focus to an emoji first. Note that pressing RightArrow once lands me on the second list item, not the first, so initial focus doesn't get set on the first list item as one would expect, but to some other element which has this strange mute Unicode character which shows up in my logs. I confirmed that I am definitely on the latest snapshot of NVDA alpha and the 20190520-dev build of WinTenApps. |
|
Hi, yep, that’s my conclusion, too. It appears object navigation emulation isn’t working correctly for your system (it works on my systems, including virtual ones). I will provide a special try build of my add-on that will log which objects NVDA is looking at when emoji panel opens. Thanks.
From: Marco Zehe <notifications@github.com>
Sent: Wednesday, May 22, 2019 9:25 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Windows 10 emoji panel and cloud clipboard: allow NVDA to make announcements when opening it in May 2019 Update (#9205)
No luck, opening still doesn't get announced, and closing only if I change focus to an emoji first. Note that pressing RightArrow once lands me on the second list item, not the first, so initial focus doesn't get set on the first list item as one would expect, but to some other element which has this strange mute Unicode character which shows up in my logs. I confirmed that I am definitely on the latest snapshot of NVDA alpha and the 20190520-dev build of WinTenApps.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#9205?email_source=notifications&email_token=AB4AXEEMJQF6VEXDNTLWWKDPWVXWVA5CNFSM4GSSN422YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV7THNQ#issuecomment-494875574> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEGPLFZ56S7LJO7BWB3PWVXWVANCNFSM4GSSN42Q> .
|
|
Hi, Turns out my hypothesis was correct: traversal failure. I have corrected this in the latest commit, and will get to changes request later today. Thanks. |
|
Hi, To @michaelDCurran, do you have any thoughts regarding what to do next for this one - use UIA for ApplicationFrameWindow or investigate MSAA implementation? If the former, then I'll update this PR - specifying 1903 build number; if the latter, I'd happy to put this on hold for a little bit. Thanks. |
|
Hi, Technical: in Version 1903 (build 18362), a "close" button has been added to the redesigned emoji panel. When the panel opens (or for that matter, any part of modern keyboard opens), focus event is fired by the Close button. Coupled with the fact that the emoji grid items are not seen as native element explains why UIA selection event, although caught by UIA handler thread, isn't passed down to app modules. Although telling NVDA that the new button should fire focus event and editing the Explorer app module may help, it introduces an inconsistency (note that this requires latest Windows 10 App Essentials add-on):
Procedure used: edited Python 3 staging source code (stashed afterwards):
Based on above findings, unless an MSAA-based workaround is available, I propose using good UIA window (should make it future-proof, too, as this affects 1903, 19H2, and 20H1). Note: I'm deferring this PR to after Python 3 transition because it is very trivial to do unless I need to do more work on locating MSAA workaround. Another reason is that, as of now, 19H2 and 20H1 exhibits the same behavior as 1903, thus all of them can be handled via this PR, simplifying the what's new entry to state "Windows 10 May 2019 Update and later" instead of just May 2019 Update, subject to changes in 20H1 in the future. Thanks. |
5e18026 to
636501a
Compare
636501a to
5a08ba5
Compare
|
Hi, The "subject to change" did happen: in 20H1, executable for modern keyboard has been renamed to Text Input Host, which means we need to add an alias app module once this PR goes through. Because 20H1 isn't due until early next year, I can wait until 202.1 development cycle before introducing the alias app module. Thanks. |
…r emoji panel/cloud clipboard in Windows 10 April 2019 Update. Re nvaccess#9204. In Windows 10 Version 1903 (April 2019 Update), it is File Explorer/Application Frame Window UIA object that fires window open event on behalf of emoji panel and cloud clipboard. Thus: * Identify ApplicationFrameWindow in builds above 17763 as good UIA window. * If window open event is fired by the good uIA window, check if the first child is part of modern keyboard (emoji panel/cloud clipboard) and ask modern keyboard app module (from previous Windows 10 feature updates) to deal with it as though a UIA window open event was fired on its own top-level window.
…. Re nvaccess#9204. Reviewed by Leonard de Ruijter (Babbage): rework the window open event handler as follows: * UIA object instance and class name will be checked first. * eventHandler.executeEvent will be fired on input panel object (the conditional to check the existence of input panel window has also been simplified).
…ptext. In some cases, clipboard tip text isn't shown, confusing NVDA as to which UIA element to descend into. Thus set clipboard items index to: * -2: tips are shown. * -1: no tips.
…l control added to emoji panel. Reported by Marco Zehe (Mozilla): on some systems running Ve4rsion 1903 (build 18362), emoji panel includes an additional control. This causes traversal routine to locate emoji grouping list, not the selected emoji when the panel opens. Thus detect this case and move to the previous list control.
…n.isWin10 function. Re nvaccess#9204. As part of UIA console work, winVersion.isWin10 function has been added, which will return if the current build is greater than (or equal to) a given Windows 10 version (YYMM). Thus use this function for clarity and edit comment accordingly.
Lint fixes include splitting long lines and adding a space between arguments for explorer.py::isGoodUIAWindow function.
In window open event, because obj.firstChild gets used often, have a dedicated (and a more descriptive) variable to refer to the input panel screen. Along with this change, split long ternary statements into conditionals.
c1035ab to
8eb235f
Compare
…ion 20H1. re nvaccess#10093. In 20H1, CJK IME has gone through an overhaul - now powered by modern keyboard. Thus add support for modern CJK IME candidates window.
…r older modern keyboard executable.
… 10 Version 20H1. re nvaccess#10093." This reverts commit 1aee198. A separate pull request will be done to add support for modern CJK IME.
|
Hi, Although modern CJK IME is powered by this change, it is a bit complex to add to this PR, thus a separate PR will be introduced but after this one goes through first. Thanks. |
|
ping @LeonarddeR if you are willing to look over this one again. |
|
Closing the PR for now, to be replaced by two or three small PR's this weekend. Thanks. |
Hi,
A continuation of #9112 but optimized for 19H1/May 2019 Update and later:
Link to issue number:
Fixes #9204
Fixes #10093
Summary of the issue:
When emoji panel and/or cloud clipboard opens in Windows 10 Version 1903 (May 2019 Update) and later, NVDA won't announce selected emoji and/or clipboard status, caused by File Explorer firing UIA window open event instead of modern keyboard doing it. This also affects CJK IME support in 20H1.
Description of how this pull request fixes the issue:
Adds the following in appModules/explorer.py:
Adds the following in modern keyboard app module:
In addition, because modern keyboard executable has been renamed in 20H1 (textinputhost.exe), an alias app module has been added.
Testing performed:
Tested on Windows 10 Version 1809 and 1903 (and later releases) from source code as well as via Windows 10 App Essentials add-on.
Known issues with pull request:
This work is subject to change if Microsoft restores UIA window open event for modern keyboard to previous behavior (the modern keyboard firing this event instead of File Explorer).
Change log entry:
Bug fixes:
NVDA will announce first selected emoji and clipboard history item when modern keyboard opens in Windows 10 May 2019 Update and later. (#9204)
NVDA will announce first selected candidate for modern Japanese IME introduced in Windows 10 20H1. (#10093)
Thanks.