Skip to content

Report virtual desktop switches in Windows 10#8259

Merged
michaelDCurran merged 10 commits into
masterfrom
i5641
May 22, 2019
Merged

Report virtual desktop switches in Windows 10#8259
michaelDCurran merged 10 commits into
masterfrom
i5641

Conversation

@michaelDCurran

Copy link
Copy Markdown
Member

Link to issue number:

Fixes #5641

Summary of the issue:

In Windows 10, it is possible to have different virual desktops, so as to switch between large groups of applications.
To switch between desktops, press control+windows+left/rightArrow.
NVDA however does not report when the desktop changes.

Description of how this pull request fixes the issue:

Partly taken from @josephsl's Windows 10 App Essentials, but improved to better handle the ordering of announcements.
Specifically:

  • Added eventHandler.handlePossibleDesktopNameChange function. This function checks to see if the Desktop's name has changed since the last time it was called. If so, then it reports the new desktop name. it fetches the desktop name off the root UIAutomation element. Also, this function does nothing on Windows versions lower than Windows 10.
  • eventHandler.doPreGainFocus now calls handlePossibleDesktopNameChange, but only after any possible foreground event, so as to ensure that speech is not interrupted. In short this means that the desktop name will be announced before the new focus and its ancestry, but after speech is canceled due to a foreground change.
  • handlePossibleDesktopNameChange is also called for nameChange events on the Desktop itself. However, NVDA delays the execution of handlePossibleDesktopNamechange in this case by 250 ms, to ensure that it executes after any other possible focus changes. Of course if a focus change did detect the desktop name change, then the delayed call from the name change event will simply do nothing. The reason for the call in the name change event is to handle the situations where the desktop changes but there is no focus event, such as when focused on the taskbar.

Testing performed:

Ran NVDA on Windows 10. Switched between multiple virtual desktops. Confirmed that the new desktop name was reported. This test was done both with focus in an application, and on the taskbar.

Known issues with pull request:

None.

Change log entry:

New features:

  • The name of the current virtual desktop on windows 10 is now reported when switching virtual desktops.

@michaelDCurran michaelDCurran requested a review from josephsl May 11, 2018 00:42

@josephsl josephsl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

This breaks if a virtual desktop contains File Explorer windows and this is seen as the focused item.

STR:

  1. Create multiple desktops.
  2. On One of them, open File Explorer.

Expected: desktop position is announced before File Explorer focus announcement is made.
Actual: Desktop announcement is cut off by focus announcement on File Explorer windows.

This was tested on Version 1803 (RS4/build 17134).

Thanks.

@michaelDCurran

michaelDCurran commented May 11, 2018 via email

Copy link
Copy Markdown
Member Author

@josephsl

Copy link
Copy Markdown
Contributor

Hi,

Cutting off: I briefly hear the first syllables of the desktop name before it is cut off by the focus announcement for File Explorer's folders list.

Thanks.

@michaelDCurran

michaelDCurran commented May 11, 2018 via email

Copy link
Copy Markdown
Member Author

@michaelDCurran

michaelDCurran commented May 11, 2018 via email

Copy link
Copy Markdown
Member Author

@josephsl

josephsl commented May 11, 2018 via email

Copy link
Copy Markdown
Contributor

@PratikP1

PratikP1 commented May 14, 2018 via email

Copy link
Copy Markdown

@josephsl

josephsl commented May 14, 2018 via email

Copy link
Copy Markdown
Contributor

@PratikP1

Copy link
Copy Markdown

Sorry for not noticing the pull request description.

@michaelDCurran

Copy link
Copy Markdown
Member Author

@PratikP1 If you could share the narrator bug you file, either publically or with me privately that would be much appreciated. I'd like to follow up with Microsoft internally. It looks like the call to actually get the desktop name, whether that be via MSAA or UIA hangs for 200 ms and then just returns "Desktop", if called as soon as the foreground change occurs. Thus, then NVDA and Narrator both just announce "Desktop" rather than say "Desktop 1".

@PratikP1

Copy link
Copy Markdown

@michaelDCurran Here is the feedback link: https://aka.ms/AA1ajhj
I included your analysis of the 200MS delay in the description.

@michaelDCurran

Copy link
Copy Markdown
Member Author

This seems to be working well for me now in 17692. @josephsl can you confirm this as well?

@michaelDCurran

Copy link
Copy Markdown
Member Author

If so I think we should limit this code to 17692 and above as it is broken in lower builds (including the last stable Windows 10 update).

@josephsl

josephsl commented Jun 19, 2018 via email

Copy link
Copy Markdown
Contributor

@josephsl

josephsl commented Jun 19, 2018 via email

Copy link
Copy Markdown
Contributor

@josephsl

Copy link
Copy Markdown
Contributor

Hi,

Testing with Korean as well: looks like we might be able to do a workaround: if the name of the object is "Desktop" (for which we have translations for), then we could safely assume that we need to delay announcing new desktop title (I'm sure there are better solutiosn out there).

As this also affects servers, I think it might be best to mention Windows Server 2019 Desktop Experience in the what's new document.

Thanks.

@michaelDCurran

Copy link
Copy Markdown
Member Author

In that case, if it is still broken in 17692, then we will wait further until it is addressed properly by Microsoft.

@josephsl

Copy link
Copy Markdown
Contributor

Hi,

Looks like it is working as designed (so far) in build 17713.

Thanks.

@michaelDCurran

michaelDCurran commented Jul 19, 2018 via email

Copy link
Copy Markdown
Member Author

@PratikP1

PratikP1 commented Jul 19, 2018 via email

Copy link
Copy Markdown

@josephsl

Copy link
Copy Markdown
Contributor

Hi,

Looks like we need to:

  1. Resolve conflicts.
  2. Make sure it works on build 17700 series (it doesn't even announce desktop switches in 17744 on my computer).

Thanks.

The desktop is checked on focus changes, and also the desktop's own name changes in case the focus does not move (such as when on the task bar).
@josephsl

Copy link
Copy Markdown
Contributor

Hi,

Never mind about desktop switch not announced in build 17744 - I was running from source. Thanks.

@josephsl

Copy link
Copy Markdown
Contributor

Hi,

Update: looks like it is better in build 17744 - with WSL open on a virtual desktop, NVDA does announce which desktop one is switching to.

@PratikP1, is this the case for you in build 17744 (slow)/17746 (fast)/18219 (skip ahead)? Does more recent builds remedy issues you've reported?

Thanks.

@PratikP1

Copy link
Copy Markdown

Unfortunately, the issues I reported remain the same in 17746.100. To summarize: With 4 virtual desktops running, desktop 1 is the most problematic. On occasion, "desktop 1" is announced. More often than not, however, only "desktop" is announced when navigating to the first desktop. All other desktops are announced. I still believe that the desktop announcement should take precedents over focused window. It should be number of desktop > focused window rather than focused window > number of desktop.

@michaelDCurran

Copy link
Copy Markdown
Member Author

This is working very well for me in Windows build 8334.0. It is always accurate, and I don't even see the strange delay I used to get. Can you confirm @josephsl that this also works for you in this build or later? We'll have to limit to this build as a minimum though before merging if we do.

@josephsl

josephsl commented Feb 20, 2019 via email

Copy link
Copy Markdown
Contributor

@josephsl

Copy link
Copy Markdown
Contributor

Hi,.

Results:

  • 18329: delay observed.
  • 18334: no delay
  • 18342: no delay

So far, the code matches Mick's predictions and observations. Thanks.

@dpy013

dpy013 commented May 17, 2019

Copy link
Copy Markdown
Contributor

Hi,

Two things:

  1. April 2019 Update -> May 2019 Update.
  2. NVDA might not report "Desktop 1" under the following scenario: user opens many desktops, then moves to a previous desktop, closes virtual desktops, focus moves to first virtual desktop. From there, whenever desktops are closed, NVDA won't report "Desktop 1". Personally, I'm okay with this approach, but I imagine someone might ask us in the future to revisit the approach proposed - comparing new versus last stored desktop title.

As for the first one, would you like to do it or let me take care of it once 2019.2 beta ships?

Thanks.
hi josephsl
My suggestion is to solve this problem once and for all.
thank

@josephsl

josephsl commented May 17, 2019 via email

Copy link
Copy Markdown
Contributor

@dpy013

dpy013 commented May 19, 2019

Copy link
Copy Markdown
Contributor

Hi, which one – first or the second? If the second, the current approach was done in order to avoid repetitions. Thanks. From: eric notifications@github.com Sent: Friday, May 17, 2019 7:05 AM To: nvaccess/nvda nvda@noreply.github.com Cc: Joseph Lee joseph.lee22590@gmail.com; Mention mention@noreply.github.com Subject: Re: [nvaccess/nvda] Report virtual desktop switches in Windows 10 (#8259) Hi, Two things: 1. April 2019 Update -> May 2019 Update. 2. NVDA might not report "Desktop 1" under the following scenario: user opens many desktops, then moves to a previous desktop, closes virtual desktops, focus moves to first virtual desktop. From there, whenever desktops are closed, NVDA won't report "Desktop 1". Personally, I'm okay with this approach, but I imagine someone might ask us in the future to revisit the approach proposed - comparing new versus last stored desktop title. As for the first one, would you like to do it or let me take care of it once 2019.2 beta ships? Thanks. hi josephsl My suggestion is to solve this problem once and for all. thank — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8259?email_source=notifications&email_token=AB4AXEEL35H5VG5HQQ32MQ3PV23RHA5CNFSM4E7K7QC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVU3CFI#issuecomment-493465877> , or mute the thread https://github.com/notifications/unsubscribe-auth/AB4AXEHTNGL5JOQAJOHEMHTPV23RHANCNFSM4E7K7QCQ .

hi josephsl
Is the first
thank

@josephsl

josephsl commented May 19, 2019 via email

Copy link
Copy Markdown
Contributor

@josephsl josephsl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

Resolved the conflict, and comment has been updated.

Thanks.

@dpy013

dpy013 commented May 21, 2019

Copy link
Copy Markdown
Contributor

thank josephsl

@josephsl

josephsl commented May 21, 2019 via email

Copy link
Copy Markdown
Contributor

@michaelDCurran michaelDCurran merged commit 0d29265 into master May 22, 2019
@nvaccessAuto nvaccessAuto added this to the 2019.2 milestone May 22, 2019
@beqabeqa473

Copy link
Copy Markdown
Contributor

hello.

this change broke nvda in some situations.
specifically, openning nvda menu freezes, also messageboxes which are openning also are spoken after a second or 2.

@michaelDCurran

Copy link
Copy Markdown
Member Author

I've now seen multiple people state that this pr causes freezes in the NVDA menu. I can't reproduce this myself, but most likely we'll have to revert this pr.

@michaelDCurran

Copy link
Copy Markdown
Member Author

this change broke nvda in some situations.
specifically, openning nvda menu freezes, also messageboxes which are openning also are spoken after a second or 2.
@beqabeqa473 If you have not already, can you please try this with all add-ons disabled?

michaelDCurran added a commit that referenced this pull request May 27, 2019
@michaelDCurran

Copy link
Copy Markdown
Member Author

Here is a try build of master with this pr reverted: https://ci.appveyor.com/api/buildjobs/7n0cgffqatnr3qch/artifacts/output%2Fnvda_snapshot_try-revert_i5641-17363%2C6490bbf3.exe
@beqabeqa473 Does this issue go away when running this try build?

@beqabeqa473

beqabeqa473 commented May 27, 2019 via email

Copy link
Copy Markdown
Contributor

@beqabeqa473

beqabeqa473 commented May 28, 2019 via email

Copy link
Copy Markdown
Contributor

@LeonarddeR

LeonarddeR commented May 28, 2019 via email

Copy link
Copy Markdown
Collaborator

@beqabeqa473

beqabeqa473 commented May 28, 2019 via email

Copy link
Copy Markdown
Contributor

@josephsl

josephsl commented May 28, 2019 via email

Copy link
Copy Markdown
Contributor

michaelDCurran added a commit that referenced this pull request May 28, 2019
codeofdusk pushed a commit to codeofdusk/nvda that referenced this pull request May 31, 2019
jcsteh added a commit to jcsteh/nvda that referenced this pull request Apr 19, 2020
Desktop switches can only be reported on Windows 10 May 2019 and above.
Previous Windows 10 builds could experience delays and incorrect desktop names.

Implementation details:

1. Added a Desktop IAccessible NVDAObject which is now used instead of the Desktop Window NVDAObject.
	This is used for api.getDesktopObject().
	It is also used to retrieve the name and to catch nameChange events (see below).
	Querying of the name via IAccessible is disabled altogether before Windows 10 may 2019, in which case it is hard-coded like it was before this PR.
2. Added eventHandler.handlePossibleDesktopNameChange function.
	This function checks to see if the Desktop's name has changed since the last time it was called.
	If so, then it reports the new desktop name.
	It fetches the desktop name using the Desktop IAccessible object.
3. eventHandler.doPreGainFocus now calls handlePossibleDesktopNameChange, but only after any possible foreground event, so as to ensure that speech is not interrupted.
	In short this means that the desktop name will be announced before the new focus and its ancestry, but after speech is canceled due to a foreground change.
4. handlePossibleDesktopNameChange is also called for nameChange events on the Desktop IAccessible object.
	However, NVDA delays the execution of handlePossibleDesktopNameChange in this case by 250 ms, to ensure that it executes after any other possible focus changes.
	Of course if a focus change did detect the desktop name change, then the delayed call from the name change event will simply do nothing.
	The reason for the call in the name change event is to handle the situations where the desktop changes but there is no focus event, such as when focused on the taskbar.

Co-authored-by: Michael Curran <mick@nvaccess.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows 10: NVDA does not report which desktop I am on when I switch amongst multiple desktops

8 participants