Start menu suggestions: announce result details#10340
Merged
michaelDCurran merged 5 commits intoOct 9, 2019
Merged
Conversation
…ductory docstring.
…es with redesigned Windows Search. Re nvaccess#10329. In 2019, Windows Search has been redesigned. Although the new UI appeared in Version 1903 (May 2019 Update), it was made available to older Windows 10 releases with a server-side update. Compared to old UI, suggestions are grouped by categories - instead of all results grouped under a single list, results list is now populated with search categories, with each category in turn hosting a list of results. This latter control isn't recognized as suggestions list, leading to NVDA not announcing item detalis as it used to. Therefore recognize the new UI and recognize actual results so NVDA can announce result details. This results in repetitions, which will be corrected with a accompanying work dealing with search field caret event changes.
Member
|
In 18995, It looks as though the suggestions in each category are housed in an unlabeled list item which is a sibling to the category listItem itself. I.e. If I type "word" There is a "find results in apps" listItem, and its next sibling is an unlabeled listItem, which contains "Microsoft Word Mobile" "Microsoft Word 2007" etc. |
Contributor
Author
|
Hi. Exactly the same, even in 18362 and 18363. The “cool” thing is that this unlabeled item is seen as suggestion list item when in fact results are inside them, akin to a treeview structure. Thanks.
From: Michael Curran <notifications@github.com>
Sent: Monday, October 7, 2019 3:21 PM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Author <author@noreply.github.com>
Subject: Re: [nvaccess/nvda] Start menu suggestions: announce result details (#10340)
In 18995, It looks as though the suggestions in each category are housed in an unlabeled list item which is a sibling to the category listItem itself. I.e. If I type "word" There is a "find results in apps" listItem, and its next sibling is an unlabeled listItem, which contains "Microsoft Word Mobile" "Microsoft Word 2007" etc.
Neither these suggestions nor the unlabelled listItem have a useful automationID or className.
Is this also what you are seeing?
In short, this pr does not seem to make anydifference for me on this build of Windows.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#10340?email_source=notifications&email_token=AB4AXEEA33LFIVHDPE56TR3QNOY4DA5CNFSM4I55W4BKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAR7U6Y#issuecomment-539228795> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEHAVNR7XZPOZW4KIQTQNOY4DANCNFSM4I55W4BA> .
|
LeonarddeR
reviewed
Oct 8, 2019
| clsList.insert(0, StartMenuSearchField) | ||
| # #10329: Since 2019, some suggestion items are grouped inside another suggestions list item. | ||
| # Because of this, result details will not be announced like in the past. | ||
| elif obj.role == controlTypes.ROLE_LISTITEM and isinstance(obj.parent, SuggestionListItem): |
Collaborator
There was a problem hiding this comment.
Wouldn't it be possible to add this extra check to the findOverlayClasses function in NVDAObjects.UIA? I have the feelling that current code scatters the places where SuggestionListItem is added as an overlay.
Contributor
Author
|
Hi, Ah, I see what's up: executable has been renamed, that's why you are not seeing changes. I'll file a PR that'll support this PR and a host of Windows Search PR's (I should have done that before doing this PR series)... Thanks. |
Contributor
Author
|
Hi, in a way, that would be ideal, but I think sometimes it is helpful to let app modules recognize objects like this if it is a custom implementation or does not follow specs correctly. I learned it the hard way this year when dealing with search field in Edge – we were catching all sorts of search fields when in fact Edge should have been excluded in the first place. If you believe UIA.findOverlayClasses is the way to go (and if others agree), I’m willing to make that change tonight. Thanks.
From: Leonard de Ruijter <notifications@github.com>
Sent: Monday, October 7, 2019 11:40 PM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Author <author@noreply.github.com>
Subject: Re: [nvaccess/nvda] Start menu suggestions: announce result details (#10340)
@LeonarddeR commented on this pull request.
_____
In source/appModules/searchui.py <#10340 (comment)> :
@@ -25,3 +29,7 @@ def chooseNVDAObjectOverlayClasses(self,obj,clsList):
elif isinstance(obj, UIA):
if obj.UIAElement.cachedAutomationID == "SearchTextBox":
clsList.insert(0, StartMenuSearchField)
+ # #10329: Since 2019, some suggestion items are grouped inside another suggestions list item.
+ # Because of this, result details will not be announced like in the past.
+ elif obj.role == controlTypes.ROLE_LISTITEM and isinstance(obj.parent, SuggestionListItem):
Wouldn't it be possible to add this extra check to the findOverlayClasses function in NVDAObjects.UIA? I have the feelling that current code scatters the places where SuggestionListItem is added as an overlay.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#10340?email_source=notifications&email_token=AB4AXEG6EDUIDVNGV76B4MLQNQTKFA5CNFSM4I55W4BKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCHFZ4HY#pullrequestreview-298556959> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXECKMOCJJMYSLNFHD3DQNQTKFANCNFSM4I55W4BA> .
|
Collaborator
|
I don't insist on having it moved to UIA.findOverlayClasses. However, I
think it must be documented somehow what suggestion items are covered by
UIA.findOverlayClasses and what suggestion items are not.
|
Contributor
Author
|
Hi, I think it can be done as part of SuggestionsListItem class docstring, as well as a comment when introducing this class in find overlay classes method. Thanks for pointing this out (shall be done tonight).
From: Leonard de Ruijter <notifications@github.com>
Sent: Monday, October 7, 2019 11:47 PM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Author <author@noreply.github.com>
Subject: Re: [nvaccess/nvda] Start menu suggestions: announce result details (#10340)
I don't insist on having it moved to UIA.findOverlayClasses. However, I
think it must be documented somehow what suggestion items are covered by
UIA.findOverlayClasses and what suggestion items are not.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#10340?email_source=notifications&email_token=AB4AXEB2UFT3ORFN4R2SMRTQNQUH5A5CNFSM4I55W4BKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEATBG2Q#issuecomment-539366250> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEDBLONVEEWRK7NIQ4DQNQUH5ANCNFSM4I55W4BA> .
|
…due to UI redesign. Commented by Leonard de Ruijter (Babbage): because not all suggestions list items are not part of results list (some are inside another item such as results from Windows Search in 2019 due to UI changes), document this fact for clarity.
Contributor
Author
|
Hi, To @michaelDCurran - can you try testing the PR with latest commit which adds searchapp app module for 20H1? Thanks. |
Member
|
@josephsl Yes, this is now working great on 18995 :) |
michaelDCurran
approved these changes
Oct 9, 2019
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi,
First of a series of pull requests on updating Start menu support routines:
Link to issue number:
Fixes #10329
Summary of the issue:
Details for search results are not announced like in the past.
Description of how this pull request fixes the issue:
Due to restructured UIA tree for Start menu (applicable in Version 1607/Anniversary Update with updated Windows Search design), search results are housed inside search categories, with categories themselves reported as suggestions list items. Therefore recognize result entries as seen in newer Windows Search experience.
Testing performed:
Tested via Windows 10 App Essentials on Version 1607 and later (no need to look into 1507 and 1511 because they retain the original Windows 10 Start menu experience).
Known issues with pull request:
Results will be announced twice due to caret movement script (see #10336 and a follow-up PR for a fix).
Change log entry:
Bug fixes:
In Start menu for Windows 10 Anniversary Update and later, NVDA will announce details of search results. (#10232