Skip to content

Windows 10 Calculator: resolve expression entry repetitions#9429

Merged
feerrenrut merged 24 commits into
nvaccess:masterfrom
josephsl:win10calculator
Nov 30, 2020
Merged

Windows 10 Calculator: resolve expression entry repetitions#9429
feerrenrut merged 24 commits into
nvaccess:masterfrom
josephsl:win10calculator

Conversation

@josephsl

@josephsl josephsl commented Apr 1, 2019

Copy link
Copy Markdown
Contributor

Link to issue number:

Fixes #9428

Summary of the issue:

Calculator raises UIA notification event (live region change event in older releases and in some screens) to communicate progress and results of calculations. However, this results in repetitions if typed character announcement is turned on. Thus, as part of this new app module, suppress repetitive announcements if possible.

Description of how this pull request fixes the issue:

This new app module performs the following:

  1. Detects conditions for live region change, name change, and UIA notification events that will suppress NVDA from announcing expressions.
  2. Adds a script for Enter (regular and numpad) and Escape keys that will allow NVDA to announce current result.

Testing performed:

Tested with Windows 10 App Essentials implementation (identical to this pull request).

Known issues with pull request:

None

Change log entry:

Bug fixes: In Windows 10 Calculator, NVDA will not announce progress of calculations if speak typed characters is disabled. (#9428)

Additional notes:

Because Windows 10 Calculator is open-source now, changes from inside and outside Microsoft will impact NVDA's support for this app, thus at least one person must be in charge of updating the app module for this app on a regular basis. Also, the new app module is a donation from Windows 10 App Essentials add-on, so it went through years of testing and real-life experiences.

Thanks.

@josephsl josephsl self-assigned this Apr 1, 2019
Comment thread source/appModules/calculator.py
Comment thread source/appModules/calculator.py
Comment thread source/appModules/calculator.py Outdated
Comment thread source/appModules/calculator.py Outdated
Comment thread source/appModules/calculator.py
Comment thread source/appModules/calculator.py Outdated
@josephsl

josephsl commented May 1, 2019

Copy link
Copy Markdown
Contributor Author

Hi,

Will plan to experiment with it (likely tonight provided my health permits).

Thanks for reminding me about that flag.

@josephsl

josephsl commented May 1, 2019

Copy link
Copy Markdown
Contributor Author

Hi,

Regarding script decorator issue, I'm thinking it has to do with name mangling.

Overall work: I'm willing to delay this work until Python 3 comes to NVDA. Doing so could also give me more time to experiment with things here and there.

Thanks.

@josephsl

josephsl commented May 1, 2019

Copy link
Copy Markdown
Contributor Author

Hi,

Regarding live region change event: after looking at the UIA live region flag and my own code, it might be possible, but we run into an important design limitation: when @michaelDCurran designed live region change handling for UIA objects in 2017 (with revisions afterwards), it was intended that all live regions will be announced, provided that texts are different and there is a reasonable time interval between event firings. Part of the reason for writing that function in the Calculator app module was to detect which regions should be announced, because it isn't quite possible to name all live region descriptors in one go and lump them into an overlay class. Perhaps the newly opened Calculator will change that, but that'll take months to hunt and fix. Besides, detecting live regions and defining the regions to be silenced will introduce code duplication like so:

  1. Descriptors of live regions in chooseNVDAObjectOverlayClasses.
  2. Flagging which live regions to announce in obj._get__shouldAllowUIALiveRegionChangeEvent.

Both will describe the same thing. Resolving the underlying issue that causes this code duplication will involve opening up other possibilities to silence live regions.

Thanks.

@josephsl

josephsl commented May 1, 2019

Copy link
Copy Markdown
Contributor Author

Hi,

If talking about the result, it isn't necessarily a focused object because it'll be announced as soon as enter key is pressed to finalize the expression.

Thanks.

josephsl added 2 commits July 29, 2019 07:18
…ess#9428.

Calculator raises UIA notification event (live region change event in older releases and in some screens) to communication progress and results of calculations. However, this results in repetitoins if typed character announcement is turned on. Thus, as part of this new app module, suppress repetitive announcements if possible.
… 10 App Essentials (accidentally committed, now gone).
@josephsl

Copy link
Copy Markdown
Contributor Author

Hi,

Update: after doing more tests, it appears @LeonarddeR's intuition was correct - disabling live region announcement will do. But first, I'll let people test possible changes via my add-on before editing the pull request.

Thanks.

In the old days of Calculator, live regions were used from various places, including calculator expressions, unit conversion and other places. Now that uIA notification event and some name change events will take care of this, remove live region change event support code (and also becasue NVDA itself will announce live regions if possible).
@josephsl

josephsl commented Aug 2, 2019

Copy link
Copy Markdown
Contributor Author

Hi,

Two things:

  1. Actually, name change event handling was modified - we still need to keep live regions alive for date calculation at least.
  2. Flake8 gives all sorts of problems with indents. If I fix E121 (visual appearance), it comes back with E101 (mixed spaces and indents). I think it would be best to ignore visual appearance bit.

Thanks.

@josephsl

josephsl commented Aug 2, 2019

Copy link
Copy Markdown
Contributor Author

Hi,

Actually, ET128.

lint results include splitting each part of a conditional, spaces between assignment operator, and block comments.
@josephsl

Copy link
Copy Markdown
Contributor Author

Hi,

Calculator has gone through yet another UI redesign, this time due to "always on" i.e. picture-in-picture mode support. The redesign also rearranges the UIA tree, which breaks this PR and the app module that ships with Windows 10 App Essentials. I'll prepare a fix after testing this week's Insider Preview build (if any).

Thanks.

…uld not allow NVDA to repeatedly announce calculation results or expressions. Re nvaccess#9428.

With the advent of alwasy on top i.e. compact overlay mode in Calculator, additional UIA automation ID's were added for calculation results and expressions in overlay mode. Because of ine length concerns, the additional ID's, together with the existing ones, are now housed inside a list of known ID's causing NVDA to announce expressions repeatedly. Not only this makes it easy to add more ID's in the future (either through NVDA Core or add-ons), it improves readability for name change and notification event handler methods (and keeps the lines short).
In August 2019, Calculator app (preview) added always on top mode. As the name suggests, this puts standard calculator on top of other windows. Because of this, UIA tree has been changed, so work with old and new tree structure.
@josephsl

josephsl commented Sep 8, 2019

Copy link
Copy Markdown
Contributor Author

To @LeonarddeR: ready for another round of review, if you are willing.

Thanks,

@feerrenrut feerrenrut added the app/windows-interface Interactions between NVDA and the default Windows GUI label Apr 8, 2020
@josephsl

josephsl commented May 6, 2020

Copy link
Copy Markdown
Contributor Author

Hi,

Update: since the last time we talked, graphing mode has been added to Calculator, but supporting that is beyond the scope of this PR. Giving this PR a bit of break due to backlog (otherwise ready for review).

Thanks.

@josephsl

josephsl commented Jul 2, 2020

Copy link
Copy Markdown
Contributor Author

Hi,

Update: it turns out the solution is simpler than I thought: rather than adding individual UIA notification activity ID's, we can just take care of one or two corner cases (DisplayUpdate ID should not be announced unless noted otherwise). This is thanks to the fact that Calculator app source code is hosted on GitHub (MIT license), so we now know the full list of activity ID's.

Thanks.

josephsl added 4 commits July 2, 2020 02:00
…ty ID.

Display updated activity ID should be ignored unless needed, as it causes repetitions when entering calculations while speak typed characters is on.
@feerrenrut feerrenrut requested a review from LeonarddeR July 7, 2020 14:31
@feerrenrut feerrenrut self-assigned this Jul 10, 2020
@feerrenrut

Copy link
Copy Markdown
Contributor

@josephsl There are a number of remaining review actions on this PR. Please can you check them out comment / resolve / make changes as appropriate. Then we can move onto a second review.

@josephsl

Copy link
Copy Markdown
Contributor Author

Hi,

Addressed in subsequent commits - will mark them as such. Thanks for the reminder.

@feerrenrut

Copy link
Copy Markdown
Contributor

@LeonarddeR since you reviewed this previously, and the feedback has now all been addressed, could you please review this again.

LeonarddeR
LeonarddeR previously approved these changes Nov 20, 2020

@LeonarddeR LeonarddeR left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@feerrenrut sorry for missing the request for review.

@josephsl

josephsl commented Nov 20, 2020 via email

Copy link
Copy Markdown
Contributor Author

feerrenrut
feerrenrut previously approved these changes Nov 30, 2020

@feerrenrut feerrenrut left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @josephsl

@feerrenrut feerrenrut dismissed stale reviews from LeonarddeR and themself via 67438ca November 30, 2020 07:40
@feerrenrut feerrenrut merged commit 559ce07 into nvaccess:master Nov 30, 2020
@nvaccessAuto nvaccessAuto added this to the 2020.4 milestone Nov 30, 2020
@josephsl

Copy link
Copy Markdown
Contributor Author

Hi,

Two issues found, to be resolved in a PR (coming today):

  1. Results are not announced in compact overlay (always on top) mode. This will require tweaking result announcement script to include a second UIA Automation Id. A much better solution is locating and announcing results element (next year because I would like to test this solution with users through WinTenApps add-on).
  2. In unit converter, source and target values (and units) are announced twice if focused on these fields, caused by notification and name change events being fired one after another. This can be resolved by adding two new Automation Id's to a list of Id's that should be checked for suppressing announcements.

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

app/windows-interface Interactions between NVDA and the default Windows GUI feature ux

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows 10 Calculator: repeated typed character announcement if speak typed characters is enabled

5 participants