Emoji panel and clipboard history: locate correct items if controls are different due to recent changes by josephsl · Pull Request #10378 · nvaccess/nvda

josephsl · 2019-10-13T03:15:13Z

Link to issue number:

Summary of the issue:

In May 2019 Update, wrong item is announced when emoji panel and clipboard history opens due to UIA tree changes.

Description of how this pull request fixes the issue:

Depending on UIA automation ID, select the right child from emoji panel/clipboard history panel so the correct item can be announced when it opens.

Testing performed:

Tested via Windows 10 App Essentials add-on.

Known issues with pull request:

None

Change log entry:

None

LeonarddeR · 2019-10-14T06:59:46Z

 		# This event is properly raised in build 17134.
-		if winVersion.winVersion.build <= 17134 and childAutomationID in ("TEMPLATE_PART_ExpressiveInputFullViewFuntionBarItemControl", "TEMPLATE_PART_ExpressiveInputFullViewFuntionBarCloseButton"):
+		if (
+			winVersion.winVersion.build <= 17134


Could we use winVersion.isWin10 instead?

LeonarddeR · 2019-10-14T07:00:26Z

+		if (
+			winVersion.winVersion.build <= 17134
+			and inputPanelAutomationID in (
+				"TEMPLATE_PART_ExpressiveInputFullViewFuntionBarItemControl",


Could you store these in a constant?

LeonarddeR · 2019-10-14T07:01:50Z

+		):
 			try:
-				self.event_UIA_elementSelected(obj.firstChild.firstChild.firstChild, nextHandler)
+				self.event_UIA_elementSelected(inputPanel.firstChild.firstChild, nextHandler)


I'm curious what @michaelDCurran thinks here, but I'd prefer something like eventHandler.executeEvent instead. IN the current code, it is difficult to find out how the flow of events is.

LeonarddeR · 2019-10-14T07:02:57Z

+				emojisList = emojisList.previous
 			try:
-				self.event_UIA_elementSelected(obj.firstChild.children[-2].firstChild.firstChild, nextHandler)
+				self.event_UIA_elementSelected(emojisList.firstChild.firstChild, nextHandler)


Same as above

LeonarddeR · 2019-10-14T07:03:28Z

+			clipboardHistory = obj.children[-2]
+			if clipboardHistory.UIAElement.cachedAutomationID == inputPanelAutomationID:
+				clipboardHistory = clipboardHistory.next
+			self.event_UIA_elementSelected(clipboardHistory, nextHandler)


Same as above

josephsl · 2019-10-14T14:05:07Z

Hi, I’ll experiment with executeEvent in the add-on today, and as for using isWin10, that’s something I’ve considered last weekend (and will do so). As for emoji panel constants in Version 1709 and 1803, yes, I’ll do so in a list. Thanks.

josephsl · 2019-12-17T03:38:01Z

Hi,

Deferred to after 2019.3 release. Thanks.

josephsl · 2020-02-16T10:22:35Z

Hi,

Now that 2019.3 stable version is out, let's get this PR going again. Note that this PR might be influenced by a UIA IME branch being worked on by @michaelDCurran.

Thanks.

…s. Re nvaccess#10377. For ease of readability, rename childAutomationID to inputPanelAutomationID, and assign obj.firstChild to inputPanel to better communicate event traversals.

…and clipboard history can be announced. Re nvaccess#10377. In recent versions of emoji panel and clipboard history (particularly since may 2019 Update), something other than selected item is announced - for example, clipboard tip instead of most recently copied item, or emoji group rather than selected emoji (the latter leading to NVDA staying silent). Thus make sure to keep in mind by checking for UIA automation ID's for both old and new versions. moving to a different element if the automation ID is different from old releases.

… with indecies. Re nvaccess#10377.

…vents directly. Re nvaccess#10377. Reviewed by Leonard de Ruijter: try using eventHandler.executeEvent when firing UIA item selected event for various modern keyboard features.

…e tuple. Re nvaccess#10377.

josephsl · 2020-05-06T05:52:57Z

Hi,

I'm looking into an alternative implementation that may future-proof this and could serve as a test bed for File Explorer status bar announcement issue (don't remember exactly which ticket it was). basically I'm thinking about locating a specific UIA element based on automation ID rather than traversing through the UIA tree. If it works (to be tested via Windows 10 App Essentials add-on), I'll open a successor PR to resolve this unless it is fine to continue working on this branch.

Thanks.

josephsl · 2020-05-06T08:26:43Z

Hi,

Actually, scratch the alternative - traversal is still required when reading clipboard announcement when clipboard history is empty. So traversal is the way to go for this one.

Thanks.

josephsl · 2020-05-27T23:01:00Z

Potentially blocked by #11214. Once that PR is included in master, I suggest merging this PR, as the follow-up described in the mentioned PR will apply to this work also.

Update: #11214 does not block this PR, as they address two different things.

Thanks.

feerrenrut · 2020-07-01T14:02:30Z

@josephsl let me know if I misunderstood please.

This PR is blocked requiring #11214 in order to be able traverse the emoji and clipboard history. Labeling as blocked.

josephsl · 2020-07-01T15:33:26Z

Hi,

Technically, #11214 addresses name change event in emoji panel. This PR addresses UIA element traversal, which is different from that PR (note: the above comment was corrected).

Thanks.

josephsl · 2020-08-03T05:53:59Z

Hi,

Since #11447 was merged, this pull request has become a bit complicated, as modern keyboard support must be revised extensively thanks to detection of UIA events for elements with no Automation Id property. Therefore a future PR will succeed this one - closing for now (I plan to integrate this PR into the succeeding PR).

Thanks.

LeonarddeR suggested changes Oct 14, 2019

View reviewed changes

josephsl added 8 commits February 16, 2020 02:23

appModules/modern keyboard: update copyright headers.

97ed554

Modern keyboard: rename input panel object and automation ID variable…

1d2cfa9

…s. Re nvaccess#10377. For ease of readability, rename childAutomationID to inputPanelAutomationID, and assign obj.firstChild to inputPanel to better communicate event traversals.

Modenr keyboard: lint.

47435b6

Emoji panel and clipboard history: locate controls instead of working…

2ebb2b3

… with indecies. Re nvaccess#10377.

Emoji panel: use winVersion.isWin10 function to determine build ranges.

58d4c08

Modern keyboard: call eventHandler.executeEvent rather than calling e…

cec7252

…vents directly. Re nvaccess#10377. Reviewed by Leonard de Ruijter: try using eventHandler.executeEvent when firing UIA item selected event for various modern keyboard features.

Emoji panel: separate 1709 and 1803 automation ID's to its own privat…

4c463eb

…e tuple. Re nvaccess#10377.

josephsl force-pushed the i10377emojiPanelTraversal branch from 42609f5 to 4c463eb Compare February 16, 2020 10:23

feerrenrut added the maintenance label Apr 8, 2020

dpy013 mentioned this pull request Apr 23, 2020

Touch interaction: allow users to disable touch support completely #10557

Merged

josephsl mentioned this pull request May 27, 2020

Selective UIA event handling #11214

Merged

feerrenrut added the blocked label Jul 1, 2020

feerrenrut removed the blocked label Jul 2, 2020

josephsl closed this Aug 3, 2020

josephsl mentioned this pull request Aug 8, 2020

Modern keyboard 2020: modernization and detection of state changes #11474

Closed

josephsl deleted the i10377emojiPanelTraversal branch August 10, 2020 04:09

josephsl mentioned this pull request Aug 11, 2020

Emoji panel and clipboard history: locate actual items when traversing the elements for element selected event #11485

Merged

Uh oh!

Conversation

josephsl commented Oct 13, 2019

Link to issue number:

Summary of the issue:

Description of how this pull request fixes the issue:

Testing performed:

Known issues with pull request:

Change log entry:

Uh oh!

LeonarddeR Oct 14, 2019

Choose a reason for hiding this comment

Uh oh!

LeonarddeR Oct 14, 2019

Choose a reason for hiding this comment

Uh oh!

LeonarddeR Oct 14, 2019

Choose a reason for hiding this comment

Uh oh!

LeonarddeR Oct 14, 2019

Choose a reason for hiding this comment

Uh oh!

LeonarddeR Oct 14, 2019

Choose a reason for hiding this comment

Uh oh!

josephsl commented Oct 14, 2019 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josephsl commented Dec 17, 2019

Uh oh!

josephsl commented Feb 16, 2020

Uh oh!

josephsl commented May 6, 2020

Uh oh!

josephsl commented May 6, 2020

Uh oh!

josephsl commented May 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

feerrenrut commented Jul 1, 2020

Uh oh!

josephsl commented Jul 1, 2020

Uh oh!

josephsl commented Aug 3, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

josephsl commented Oct 14, 2019 via email •

edited by feerrenrut

Loading

josephsl commented May 27, 2020 •

edited

Loading