Added ability for braille to show what NVDA is speaking by Emil-18 · Pull Request #15956 · nvaccess/nvda

Emil-18 · 2023-12-22T03:03:37Z

Link to issue number:

Summary of the issue:

Both in NVDA core, as well as some add-ons, have features that work with speech but don't work with braille

Description of user facing changes

Braille only users will be able to read NVDA speech in braille, thus get access to a lot more features then they would otherwise

Description of development approach

A new braille mode has been added, called "speech emulation". In this mode, NVDA will send all text that has been spoken since the last time speech was canceled, to the braille display. If the speech was canceled do to the user scrolling the braille display back or fourth, the speech output mode will treat it as if the speech was not canceled, so the user can read the text without erasing it accidentally. If braille.handler.rawText is grater then 100000 characters, it will not be sent, because in my experience, NVDA will freeze when that much text is sent to the braille display at once. The braille settings panel has been rearranged so that all settings not aplickable to speech emulation is grouped together. A new combo box has been added to the braille settings panel, and a new script with the gesture NVDA+alt+t have been added, that allows the user to change braille mode.

Testing strategy:

tested with a Focus braille display. To test that it ignored speech being canceled when scrolling the braille display back or fourth, I turned off "Speech interrupt for typed characters", then opened the run dialog, scrolled the braille display to the end of the text while in speech output mode with the "Interrupt speech while scrolling" setting turned on, and then typed some text. I then scrolled the braille display back and fourth, and continued typing text while doing this to ensure that it worked as it should. Everything worked as expected

Known issues with pull request:

None so far

Code Review Checklist:

Documentation:
- Change log entry
- User Documentation
- Developer / Technical Documentation
- Context sensitive help for GUI changes
Testing:
- Unit tests
- System (end to end) tests
- Manual testing
UX of all users considered:
- Speech
- Braille
- Low Vision
- Different web browsers
- Localization in other languages / culture than English
API is compatible with existing add-ons.
Security precautions taken.

…an error when speech emulation mode is selected

… clear the braille regions in speech emulation mode

…se when canceling speech when the script is one of the braille scrolling scripts

…mulation and a large amount of text is given

…dissabled when speech emulation is selected

XLTechie · 2023-12-22T09:55:27Z

Is this something that enough users will want to toggle on the fly, that it warrants assigning a key by default?

lukaszgo1 · 2023-12-22T11:48:03Z

Both in NVDA core, as well as some add-ons, have features that work with speech but don't work with braille

Would you be able to provide some examples. Especially for core features if they don't work well in Braille it is useful to have an issue to track the problem.

burmancomp · 2023-12-22T12:31:00Z

I tested and noticed that something about new setting should be added to configSpec.py to get nvda from source running.

If braille is tethered to review, changing setting from settings dialog braille category from "speech emulation" to "follow cursors" causes attribute error and displaying braille is stopped until changed to what braille is tethered to.

I did not get nvda+alt+t to work with laptop layout when I used caps lock as nvda key although it was shown in braille category of gestures but this is something what is not problem of this pr.

…via the braille settings pannel when braille is tethered to review

…emulateSpeech

Emil-18 · 2023-12-23T18:16:56Z

@lukaszgo1 As of the latest alpha, braille does not show:

The object under the mouse when mouse tracking is enabled.
Formatting information when it is turned on in document formatting settings.
How many list items it is in a list on the web when entered, in a situation where speech would report list with x items.
How many rows and colums a web/word table have when entered.
Information about objects that are higher up in the object hierarchy, such as lists or groupings, in legacy edge UIA browse mode documents when browse mode is turned on. Same for open/libre office documents.
On screen text on web pages that you can review with the review cursor commands in focus mode that isn't in edit boxes. I know that you aren't actualy supposed to review this text, but it is still information that a speech user can get and a braille only user can't.
It is impossible to use the f9 selection mode with braille when the current review position is in an object that don't has useful text according to the braille.NVDAObjectHasUsefulText function.
When ui.message is called in a fiew milliseconds after a previous call, something I have seen a lot of add-ons do, it is impossible to read what was sent to the braille display in the first call, as this text is overwritten by the text in the second.
When ui.message is called inside a thread, and a braille display is connected, and show messages is set to use timeout, it gives the following error
ERROR - stderr (19:11:34.275) - Thread-2 (test) (10760):
C++ assertion "wxThread::IsMain()" failed at ....\src\common\timerimpl.cpp(57) in wxTimerImpl::Start(): timer can only be started from the main thread
resulting in the rest of the function never executing.

Emil-18 · 2023-12-23T18:37:03Z

@XLTechie I don't know.

XLTechie · 2023-12-23T23:37:25Z

Actually, given your response to @lukaszgo1, and the probable use cases you site, I am thinking that this probably does warrant a shortcut key out of the box.

…ille mode script

AppVeyorBot · 2024-03-15T03:49:58Z

PASS: Translation comments check.
PASS: Unit tests.
FAIL: Lint check. See test results for more information.
PASS: System tests (tags: installer NVDA).
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/bb4m5sj83hebkw4k/artifacts/output/nvda_snapshot_pr15956-31453,672d8524.exe
CI timing (mins):
INIT 0.0,
INSTALL_START 1.1,
INSTALL_END 0.9,
BUILD_START 0.0,
BUILD_END 29.5,
TESTSETUP_START 0.0,
TESTSETUP_END 0.4,
TEST_START 0.0,
TEST_END 2.4,
FINISH_END 0.2

See test results for failed build of commit 672d8524ed

Emil-18 · 2024-03-16T16:46:12Z

@seanbudd I committed your changes and tested them. It works as it should now.
You wrote

I don't believe script is a tuple here? I don't believe this is the correct type hint, please see comment
#15956

But in that comment, you suggested that exact change?

AppVeyorBot · 2024-03-29T05:02:09Z

PASS: Translation comments check.
FAIL: Unit tests. See test results for more information.
FAIL: Lint check. See test results for more information.
FAIL: System tests (tags: installer NVDA). See test results for more information.
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/sb2da8r6e0oklsjf/artifacts/output/nvda_snapshot_pr15956-31531,29373466.exe
CI timing (mins):
INIT 0.0,
INSTALL_START 1.2,
INSTALL_END 1.0,
BUILD_START 0.0,
BUILD_END 32.7,
TESTSETUP_START 0.0,
TESTSETUP_END 0.3,
TEST_START 0.0,
TEST_END 0.7,
FINISH_END 0.2

See test results for failed build of commit 29373466ff

AppVeyorBot · 2024-03-29T09:22:56Z

Build (for testing PR): https://ci.appveyor.com/api/buildjobs/p7vpdl4irpq3gsr9/artifacts/output/
CI timing (mins):
INIT 0.0,
INSTALL_START 1.2,
INSTALL_END 1.0,
BUILD_START 0.0,
FINISH_END 29.4

See test results for failed build of commit ea8be83f53

AppVeyorBot · 2024-03-29T13:15:43Z

PASS: Translation comments check.
FAIL: Unit tests. See test results for more information.
FAIL: Lint check. See test results for more information.
FAIL: System tests (tags: installer NVDA). See test results for more information.
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/6dj7eep40m1adk6y/artifacts/output/nvda_snapshot_pr15956-31533,d28da2d2.exe
CI timing (mins):
INIT 0.0,
INSTALL_START 1.1,
INSTALL_END 1.0,
BUILD_START 0.0,
BUILD_END 30.7,
TESTSETUP_START 0.0,
TESTSETUP_END 0.2,
TEST_START 0.0,
TEST_END 0.6,
FINISH_END 0.2

See test results for failed build of commit d28da2d23b

AppVeyorBot · 2024-03-29T19:00:47Z

PASS: Translation comments check.
FAIL: Unit tests. See test results for more information.
FAIL: Lint check. See test results for more information.
FAIL: System tests (tags: installer NVDA). See test results for more information.
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/bxq0p82r2m0hvk5c/artifacts/output/nvda_snapshot_pr15956-31536,c12f4224.exe
CI timing (mins):
INIT 0.0,
INSTALL_START 1.1,
INSTALL_END 1.0,
BUILD_START 0.0,
BUILD_END 31.7,
TESTSETUP_START 0.0,
TESTSETUP_END 0.2,
TEST_START 0.0,
TEST_END 0.7,
FINISH_END 0.1

See test results for failed build of commit c12f4224ba

Emil-18 · 2024-03-31T20:22:33Z

@seanbudd I think I have fixed the type hint on the suppressClearBrailleRegions function now

seanbudd

this looks almost ready

seanbudd · 2024-04-15T05:48:23Z


 :param value: the speech sequence to be filtered.
 :type value: SpeechSequence
+


can you remove this junk new line added

seanbudd · 2024-04-15T05:48:57Z


 """High-level functions to speak information.
 """ 
-


please restore this new line

Qchristensen

Reads well. Agree with Sean's change.

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

CyrilleB79 · 2024-09-12T07:57:16Z

I was looking for the release where pre_speech extension point had been added. Unfortunately, this addition does not appear in the change log.

When reviewing PRs adding new extension points in the future, @seanbudd and @SaschaCowley could you double check that this addition is also documented in the changes for developers section? This would be faster to see in which release an extension point has been added, instead of doing a git blame, a git log on the found commit and open the corresponding GitHub PR number to check the milestone. Thanks.

XLTechie · 2024-09-12T08:40:51Z

Yes, I had also noticed that #15956 and #16191 weren't in the changelog.

Emil-18 and others added 15 commits December 19, 2023 23:41

made some functions compatible with speech emulation mode

2216282

made some braille commands that are related to cursor/selection give …

8c5c6e0

…an error when speech emulation mode is selected

added a select mode combo box to the braille settings pannel

e95e8ea

modified speech and cancelSpeech, to support speech emulation mode

fc27025

added a command to cycle through the available braille modes

b81738a

added a optional argument to cancelSpeech wich determens if it should…

105ad6b

… clear the braille regions in speech emulation mode

removed commented code that wasnt neaded

3da5cfd

updated the executeGesture function to set clearBrailleRegions to Fal…

7f9b563

…se when canceling speech when the script is one of the braille scrolling scripts

updated changes.t2t

a3343b6

changed cursor tethering to follow cursors

f9814b0

made speech.speak much less likely to freeze NVDA when using speech e…

cd0f254

…mulation and a large amount of text is given

fixed a bug with the toggleBrailleMode script

52b60b3

added a section for the braille mode combo box

e466266

settings that are only applicable for the follow cursors mode is now …

789bc9f

…dissabled when speech emulation is selected

Merge branch 'nvaccess:master' into emulateSpeech

ccdd5e5

This comment was marked as resolved.

Sign in to view

Emil-18 changed the title ~~Added abillity for braille to show what NVDA is speaking~~ Added ability for braille to show what NVDA is speaking Dec 22, 2023

Emil-18 added 2 commits December 23, 2023 18:27

It is now possible to change from speech emulation to follow cursors …

4551ca8

…via the braille settings pannel when braille is tethered to review

Merge branch 'emulateSpeech' of https://github.com/Emil-18/nvda into …

c8666c1

…emulateSpeech

This comment was marked as resolved.

Sign in to view

Fixed the placement of the braille mode script

6d98602

This comment was marked as resolved.

Sign in to view

Added a blank line between the key command and description of the bra…

3815709

…ille mode script

This comment was marked as resolved.

Sign in to view

Fixed linting issues

fb59a69

Emil-18 added 2 commits March 29, 2024 05:22

Updated inputCore.py

c6fcac3

Fixed the type hint on the suppressClearBrailleRegions function

0f934d0

Import typing.Callable

0eede39

fixed a syntax error

a828b4b

Emil-18 and others added 2 commits March 29, 2024 19:05

Merge branch 'nvaccess:master' into emulateSpeech

ed02cc0

updated inputCore.py

69b2af1

moved the ScriptT definition below the InputGestureClass

27408c2

Emil-18 marked this pull request as ready for review April 6, 2024 16:37

seanbudd reviewed Apr 15, 2024

View reviewed changes

Qchristensen approved these changes Apr 15, 2024

View reviewed changes

Emil-18 and others added 4 commits April 15, 2024 14:23

Update projectDocs/dev/developerGuide/developerGuide.t2t

ba77962

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

Update user_docs/en/changes.t2t

ef5327b

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

Removed an unnecessary blank line

1971ae4

added a blank line

87e2f90

seanbudd approved these changes Apr 16, 2024

View reviewed changes

seanbudd closed this Apr 16, 2024

seanbudd reopened this Apr 16, 2024

seanbudd merged commit 3d4a0a8 into nvaccess:master Apr 16, 2024

nvaccessAuto added this to the 2024.2 milestone Apr 16, 2024

CyrilleB79 mentioned this pull request Jul 23, 2024

Unable to reset settings to factory default #16896

Closed


		:param value: the speech sequence to be filtered.
		:type value: SpeechSequence

Uh oh!

Conversation

Emil-18 commented Dec 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Link to issue number:

Summary of the issue:

Description of user facing changes

Description of development approach

Testing strategy:

Known issues with pull request:

Code Review Checklist:

Uh oh!

This comment was marked as resolved.

XLTechie commented Dec 22, 2023 via email

Uh oh!

lukaszgo1 commented Dec 22, 2023

Uh oh!

burmancomp commented Dec 22, 2023

Uh oh!

This comment was marked as resolved.

Emil-18 commented Dec 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Emil-18 commented Dec 23, 2023

Uh oh!

XLTechie commented Dec 23, 2023 via email

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

AppVeyorBot commented Mar 15, 2024

Uh oh!

Emil-18 commented Mar 16, 2024

Uh oh!

AppVeyorBot commented Mar 29, 2024

Uh oh!

AppVeyorBot commented Mar 29, 2024

Uh oh!

AppVeyorBot commented Mar 29, 2024

Uh oh!

AppVeyorBot commented Mar 29, 2024

Uh oh!

Emil-18 commented Mar 31, 2024

Uh oh!

seanbudd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

seanbudd Apr 15, 2024

Choose a reason for hiding this comment

Uh oh!

seanbudd Apr 15, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Qchristensen left a comment

Choose a reason for hiding this comment

Uh oh!

CyrilleB79 commented Sep 12, 2024

Uh oh!

XLTechie commented Sep 12, 2024 via email

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

Emil-18 commented Dec 22, 2023 •

edited

Loading

Emil-18 commented Dec 23, 2023 •

edited

Loading