Skip to content

Windows 10/11 Calculator: use UIA find descendants when fetching results while handling UIA notification event#14745

Merged
seanbudd merged 5 commits into
nvaccess:masterfrom
josephsl:i14679calculatorUIADescendants
Mar 28, 2023
Merged

Windows 10/11 Calculator: use UIA find descendants when fetching results while handling UIA notification event#14745
seanbudd merged 5 commits into
nvaccess:masterfrom
josephsl:i14679calculatorUIADescendants

Conversation

@josephsl

Copy link
Copy Markdown
Contributor

Link to issue number:

Closes #14679

Summary of the issue:

Portable NVDA does not announce results when Windows 10/11 Calculator is in compact overlay mode.

Description of user facing changes

Portable copy of NVDA will announce calculator expressions and results when Windows Calculator is in compact overlay mode (Alt+up arrow from standard calculator mode).

Description of development approach

Instead of using tree traversal, use UIA find descendants approach (employed in File Explorer) to obtain results element when Calculator is in normal and compact overlay mode. This also removes the need to check Calculator version as Windows 11 Calculator has made its way to Windows 10.

Testing strategy:

Manual testing:

  1. After creating a portable copy of this PR or alpha builds with this PR applied, open Windows 10/11 Calculator.
  2. Press Alt+H, then select standard calculator.
  3. From standard calculator, enter some expressions. NVDA will announce expressions.
  4. Press Alt+up arrow to switch to compact overlay mode, then type expressions.

Prior to this PR: portable NVDA would not announce anything or play error tones.

Expected: Portable NVDA will announce expressions and results when Calculator is in compact overlay mode.

The code was also tested with Windows App Essentials add-on.

Known issues with pull request:

If COM error occurs, UIA notification event handler will simply exit, but no issues so far.

Change log entries:

Bug fixes:

In Windows 10 and 11 Calculator, portable copy of NVDA will no longer do nothing or play error tones when entering expressions in standard calculator in compact overlay mode. (#14679)

Code Review Checklist:

  • Pull Request description:
    • description is up to date
    • change log entries
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • API is compatible with existing add-ons.
  • Documentation:
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • Security precautions taken.

…access#14679.

If a portable copy of NVDA is running and compact overlay mode is active from Calculator, NVDA will not announce results as users type expressions, caused by index error stemming from tree traversal failure. As an alternative, use UIA descendants to locate results element, allowing portable NVDA to announce results when Calculator is in compact overlay mde. The find descendants code ultimately comes from the one employed in File Explorer when fetching status bar items.
…fetching results element. Re nvaccess#14679.

NVDA had to descend one more level when fetching results in Windows 11 Calculator. Now that the moer modern Calculator is available on Windows 10 along with use of UIA descendants to fetch results element, there is no need to check app version.
@josephsl josephsl requested a review from a team as a code owner March 25, 2023 22:04
@josephsl josephsl requested a review from seanbudd March 25, 2023 22:04
@seanbudd

Copy link
Copy Markdown
Member

What happens here with older version of Windows 10?

@josephsl

Copy link
Copy Markdown
Contributor Author

Hi,

Older Windows 10 Calculator releases do not fire UIA notification event, so we can ignore it. Newer Windows Calculator releases were initially made available to Windows 11 but was made available to Windows 10 Version 2004 and later (as far as consumers and businesses are concerned, the oldest supported version is 20H2; LTSC can be ignored becasue Windows Calculator is unavailable on that channel).

By the way, Calculator compact overlay mode is also called always on top mode.

Thanks.

@seanbudd seanbudd merged commit 9958841 into nvaccess:master Mar 28, 2023
@nvaccessAuto nvaccessAuto added this to the 2023.2 milestone Mar 28, 2023
@josephsl josephsl deleted the i14679calculatorUIADescendants branch March 31, 2023 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows 10/11 Calculator: "list index out of range" when entering expressions in portable NVDA with Calculator compact overlay on

3 participants