Skip to content

Show local ui.messages in braille when controlling a remote computer#19056

Merged
SaschaCowley merged 12 commits into
masterfrom
brailleLocalUiMessage
Oct 9, 2025
Merged

Show local ui.messages in braille when controlling a remote computer#19056
SaschaCowley merged 12 commits into
masterfrom
brailleLocalUiMessage

Conversation

@SaschaCowley

@SaschaCowley SaschaCowley commented Oct 7, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Link to issue number:

Fixes #18004

Summary of the issue:

Currently, when controlling a remote computer via NVDA Remote Access, messages emitted via ui.message on the local computer are not displayed on the refreshable braille display. This is problematic because Remote Access outputs some status messages via ui.message, and this bug means that braille-only users are unable to perceive these messages.

Description of user facing changes:

These messages are now shown in braille.

Description of developer facing changes:

None.

Description of development approach:

  • Add _pre_showBrailleMessage, _post_dismissBrailleMessage and _decide_disabledIncludesMessage extension points.
  • Allow BrailleHandler.message to show a message if enabled is False and _decide_disabledIncludesMessage.decide() is also False.
  • Set Remote Access to not receive braille when showing a ui.message, as detected by the aforementioned extension points.
  • Temporarily enable braille.handler when showing a ui.message, as detected by the aforementioned extension points.
  • Process braille display scrolling and routing locally when showing a ui.message, as these keys are used to pan through and dismiss the message.
  • Add a hostPendingNonmodifier attribute to _RemoteClient.client.RemoteClient, to track the non-modifier key of the toggle gesture.
  • When processing key input, also block the hostPendingNonmodifier from being sent.
  • When controlling the remote computer, dismiss a ui.message shown by the local computer on keyboard or braille display input.
    • This input is also sent to the remote computer, unlike routing or panning keys if a local message is being shown.

Testing strategy:

Ensure that the "Controlling remote computer" message can be read and interacted with in braille.

Trigger local ui.message while controlling the remote computer (via wx.CallLater in the NVDA Python Console), and ensure the message can be read and interacted with in braille. Did this with short and long messages (to ensure panning works).

Performed these tests with "Show messages" set to "Use timeout", "Show indefinitely" and "Disabled" (in the latter case confirming that messages weren't shown).

Known issues with pull request:

The display contents changing on the remote computer does not cause the message to be dismissed. This is because we cannot tell the difference between a contentful display change, and something like the cursor flashing.

Some users may prefer the old behaviour. We could possibly make this configurable, though I think doing so may be premature since we don't know if users will want the option.

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

@SaschaCowley

SaschaCowley commented Oct 7, 2025

Copy link
Copy Markdown
Member Author

@LeonarddeR this isn't finished yet, and is a rather ugly hack, but I seem to have it mostly working. Would you mind letting me know what you think of this approach if you have time? Hopefully it is clear where I'm going - I will of course be documenting more once I have it working

@LeonarddeR

LeonarddeR commented Oct 7, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

I actually consider this solution to be pretty elegant. Executing the getter to re-eveluate state is indeed a bit ugly. May be _get_enabled should be split into a new method _reEvaluateEnabledState or something like that.

@SaschaCowley

SaschaCowley commented Oct 7, 2025

Copy link
Copy Markdown
Member Author

Thanks, @LeonarddeR, I'll consider that

@SaschaCowley SaschaCowley marked this pull request as ready for review October 8, 2025 06:23
@SaschaCowley SaschaCowley requested a review from a team as a code owner October 8, 2025 06:23
@SaschaCowley SaschaCowley requested review from Copilot and seanbudd and removed request for seanbudd October 8, 2025 06:23
Copilot AI reviewed Oct 8, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enables braille users to read local ui.message content when controlling a remote computer via NVDA Remote Access. Previously, these messages were only available to speech users, creating an accessibility gap for braille-only users.

  • Adds extension points to manage braille message display coordination between local and remote contexts
  • Temporarily switches braille control back to the local machine when displaying ui.message content
  • Implements proper handling of braille navigation keys (scrolling/routing) for local messages while blocking other input

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
user_docs/en/changes.md Documents the new feature in the changelog
source/braille.py Adds extension points and modifies message handling to support remote access scenarios
source/_remoteClient/session.py Handles braille gesture routing during local message display
source/_remoteClient/localMachine.py Implements core logic for switching between local and remote braille control

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread source/braille.py
Comment thread source/_remoteClient/localMachine.py Outdated
Comment thread source/_remoteClient/localMachine.py
seanbudd
seanbudd approved these changes Oct 8, 2025
View reviewed changes

@seanbudd seanbudd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @SaschaCowley

Comment thread source/_remoteClient/localMachine.py Outdated
Comment thread source/_remoteClient/localMachine.py Outdated
@SaschaCowley @seanbudd
Apply suggestions from code review
6e48f2a 
Co-authored-by: Sean Budd <sean@nvaccess.org>
@SaschaCowley SaschaCowley marked this pull request as draft October 8, 2025 23:17
@SaschaCowley

SaschaCowley commented Oct 8, 2025

Copy link
Copy Markdown
Member Author

@LeonarddeR what are your thoughts on the known issues with this PR?

@SaschaCowley SaschaCowley marked this pull request as ready for review October 9, 2025 01:49
@SaschaCowley SaschaCowley merged commit a2e5b2a into master Oct 9, 2025
29 checks passed
@SaschaCowley SaschaCowley deleted the brailleLocalUiMessage branch October 9, 2025 03:32
@github-actions github-actions Bot added this to the 2026.1 milestone Oct 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@seanbudd seanbudd seanbudd approved these changes

@LeonarddeR LeonarddeR Awaiting requested review from LeonarddeR

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

2026.1

Development

Successfully merging this pull request may close these issues.

ui.message fails to output braille when controlling a remote machine

4 participants

@SaschaCowley @LeonarddeR @seanbudd