Skip to content

Make automatic braille display detection more pythonic#14147

Merged
seanbudd merged 11 commits into
nvaccess:masterfrom
LeonarddeR:threadPool
Nov 9, 2022
Merged

Make automatic braille display detection more pythonic#14147
seanbudd merged 11 commits into
nvaccess:masterfrom
LeonarddeR:threadPool

Conversation

@LeonarddeR

@LeonarddeR LeonarddeR commented Sep 14, 2022

Copy link
Copy Markdown
Collaborator

Link to issue number:

Related to #14130

Summary of the issue:

Parts of the braille display detection code are hard to read and rely on Win32 APCs to be queued. A major drawback about APCS is that, as soon as theyre queued, they can not be canceled any more.

Description of user facing changes

None, though I hope things will be more stable after this change

Description of development approach

In an attempt to make the code more pythonic as well as changing the code in such a way that it no longer depends on the braille background thread, I changed the following:

  • The detection work now has a dedicated thread, using a thread pool under the hood. This is a thread pool with only one thread. This might sound stupid, but the major advantage of a ThreadPoolExecutor is that it allows to execute and queue functions on the thread
  • The detection function is no longer a loop. It was a loop to avoid queuing an apc when we knew one was already running, but as the ThreadPoolExecutor has an internal queue and we can cancel futures now, this decreases compexity.

Testing strategy:

This requires extensive testing in alpha for a while. Note that @dkager reported improvements, see #14147 (comment)

Known issues with pull request:

None known

Change log entries:

For Developers

  • Reduced the complexity of the background braille display auto detection logic.

Code Review Checklist:

  • Pull Request description:
    • description is up to date
    • change log entries
  • [] Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • API is compatible with existing add-ons.
  • Documentation:
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • Security precautions taken.

@AppVeyorBot

Copy link
Copy Markdown

See test results for failed build of commit 57b7268556

@AppVeyorBot

Copy link
Copy Markdown

See test results for failed build of commit cfefdb5e9b

@AppVeyorBot

Copy link
Copy Markdown

See test results for failed build of commit 2f2ac00002

@AppVeyorBot

Copy link
Copy Markdown

See test results for failed build of commit 8a08c1db77

@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

This was tested by @dkager. He reported that crashes on his system that happened on Alpha still occur with this branch, so it looks like there is no real benefit in this pr other than that it decouples auto detection from braille._BgTrhead. I leave it up to @feerrenrut or someone else from NV Access to decide about this pr.

@lukaszgo1

Copy link
Copy Markdown
Contributor

@LeonarddeR Have you considered creating an issue describing these crashes? I understand that you don't have a solution, but not reporting at all means that unless either you or @dkager would create a fix the problem will remain unsolved.

@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

@dkager Would you be able to do this? As far as I understood you correctly, the crashes are very rare and hard to reproduce reliably.

@dkager

dkager commented Oct 8, 2022

Copy link
Copy Markdown
Contributor

While the code from @LeonarddeR does not fix everything, I did notice that it was much rarer for NVDA to fall over altogether. With 2022.3 I estimate there to be a 20-30% chance of NVDA crashing after waking the Brailliant BI 40X from sleep while on bluetooth. I haven't found any log entries pointing to the cause, though.

@dkager

dkager commented Oct 8, 2022

Copy link
Copy Markdown
Contributor

Update: I got this straight after booting the PC with NVDA auto-starting. Obviously, Leonard's code fixes this.

INFO - braille.BrailleHandler.setDisplayByName (11:15:37.686) - braille._BgThread (9276):
Loaded braille display driver noBraille, current display has 0 cells.
INFO - braille.BrailleHandler.setDisplayByName (11:15:39.143) - braille._BgThread (9276):
Loaded braille display driver noBraille, current display has 0 cells.
INFO - braille.BrailleHandler.setDisplayByName (11:15:39.159) - braille._BgThread (9276):
Loaded braille display driver noBraille, current display has 0 cells.
INFO - brailleDisplayDrivers.brailliantB.BrailleDisplayDriver.__init__ (11:15:40.259) - braille._BgThread (9276):
Found display with 40 cells connected via hid (\\?\hid#{00001124-0000-1000-8000-00805f9b34fb}_vid&00021d6b_pid&0246&col01#8&f2c30de&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030})
INFO - braille.BrailleHandler.setDisplayByName (11:15:40.264) - braille._BgThread (9276):
Loaded braille display driver brailliantB, current display has 40 cells.
ERROR - stderr (11:15:40.269) - braille._BgThread (9276):
Exception in thread braille._BgThread:
Traceback (most recent call last):
  File "threading.pyc", line 926, in _bootstrap_inner
  File "threading.pyc", line 870, in run
  File "braille.pyc", line 2368, in func
OSError: exception: access violation writing 0xDBB3EFC4

@dkager

dkager commented Oct 8, 2022

Copy link
Copy Markdown
Contributor

And interestingly, when disabling HID braille support:

ERROR - braille.executor (12:15:09.681) - braille._BgThread (7440):
Error displaying cells. Disabling display
Traceback (most recent call last):
  File "braille.pyc", line 2344, in executor
  File "brailleDisplayDrivers\brailliantB.pyc", line 240, in display
  File "hwIo\hid.pyc", line 301, in setOutputReport
OSError: [WinError 1167] The device is not connected.
ERROR - braille.BrailleHandler.handleDisplayUnavailable (12:15:09.681) - braille._BgThread (7440):
Braille display unavailable. Disabling
Traceback (most recent call last):
  File "braille.pyc", line 2344, in executor
  File "brailleDisplayDrivers\brailliantB.pyc", line 240, in display
  File "hwIo\hid.pyc", line 301, in setOutputReport
OSError: [WinError 1167] The device is not connected.
DEBUG - autoSettingsUtils.autoSettings.AutoSettings._registerConfigSaveAction (12:15:09.681) - braille._BgThread (7440):
registering pre_configSave action: <class 'brailleDisplayDrivers.noBraille.BrailleDisplayDriver'>
DEBUG - braille.BrailleHandler.setDisplayByName (12:15:09.681) - braille._BgThread (7440):
Switching braille display from brailliantB to noBraille
ERROR - braille.BrailleDisplayDriver.terminate (12:15:09.681) - braille._BgThread (7440):
Display driver <brailleDisplayDrivers.brailliantB.BrailleDisplayDriver object at 0x08CA3870> failed to display while terminating.
Traceback (most recent call last):
  File "braille.pyc", line 2344, in executor
  File "brailleDisplayDrivers\brailliantB.pyc", line 240, in display
  File "hwIo\hid.pyc", line 301, in setOutputReport
OSError: [WinError 1167] The device is not connected.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "braille.pyc", line 2504, in terminate
  File "brailleDisplayDrivers\brailliantB.pyc", line 240, in display
  File "hwIo\hid.pyc", line 301, in setOutputReport
OSError: [WinError 1167] The device is not connected.
DEBUG - autoSettingsUtils.autoSettings.AutoSettings._loadSpecificSettings (12:15:09.692) - braille._BgThread (7440):
loading braille noBraille
INFO - braille.BrailleHandler.setDisplayByName (12:15:09.692) - braille._BgThread (7440):
Loaded braille display driver noBraille, current display has 0 cells.
DEBUG - autoSettingsUtils.autoSettings.AutoSettings._registerConfigSaveAction (12:15:09.702) - braille._BgThread (7440):
registering pre_configSave action: <class 'brailleDisplayDrivers.brailliantB.BrailleDisplayDriver'>
DEBUG - braille.BrailleHandler.setDisplayByName (12:15:11.161) - braille._BgThread (7440):
Reinitializing noBraille braille display
DEBUG - autoSettingsUtils.autoSettings.AutoSettings._registerConfigSaveAction (12:15:11.161) - braille._BgThread (7440):
registering pre_configSave action: <class 'brailleDisplayDrivers.noBraille.BrailleDisplayDriver'>
DEBUG - autoSettingsUtils.autoSettings.AutoSettings._loadSpecificSettings (12:15:11.166) - braille._BgThread (7440):
loading braille noBraille
INFO - braille.BrailleHandler.setDisplayByName (12:15:11.166) - braille._BgThread (7440):
Loaded braille display driver noBraille, current display has 0 cells.
DEBUG - autoSettingsUtils.autoSettings.AutoSettings._registerConfigSaveAction (12:15:11.166) - braille._BgThread (7440):
registering pre_configSave action: <class 'brailleDisplayDrivers.brailliantB.BrailleDisplayDriver'>
DEBUGWARNING - brailleDisplayDrivers.brailliantB.BrailleDisplayDriver.__init__ (12:15:11.166) - braille._BgThread (7440):
Traceback (most recent call last):
  File "brailleDisplayDrivers\brailliantB.pyc", line 101, in __init__
  File "hwIo\hid.pyc", line 150, in __init__
PermissionError: [WinError 5] Access is denied.
DEBUG - braille.BrailleHandler.setDisplayByName (12:15:11.166) - braille._BgThread (7440):
Reinitializing noBraille braille display
DEBUG - autoSettingsUtils.autoSettings.AutoSettings._registerConfigSaveAction (12:15:11.166) - braille._BgThread (7440):
registering pre_configSave action: <class 'brailleDisplayDrivers.noBraille.BrailleDisplayDriver'>
DEBUG - autoSettingsUtils.autoSettings.AutoSettings._loadSpecificSettings (12:15:11.171) - braille._BgThread (7440):
loading braille noBraille
INFO - braille.BrailleHandler.setDisplayByName (12:15:11.172) - braille._BgThread (7440):
Loaded braille display driver noBraille, current display has 0 cells.
DEBUG - autoSettingsUtils.autoSettings.AutoSettings._registerConfigSaveAction (12:15:11.172) - braille._BgThread (7440):
registering pre_configSave action: <class 'brailleDisplayDrivers.brailliantB.BrailleDisplayDriver'>
INFO - brailleDisplayDrivers.brailliantB.BrailleDisplayDriver.__init__ (12:15:12.252) - braille._BgThread (7440):
Found display with 40 cells connected via hid (\\?\hid#{00001124-0000-1000-8000-00805f9b34fb}_vid&00021d6b_pid&0246&col01#8&f2c30de&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030})
DEBUG - braille.BrailleHandler.setDisplayByName (12:15:12.252) - braille._BgThread (7440):
Switching braille display from noBraille to brailliantB
DEBUG - autoSettingsUtils.autoSettings.AutoSettings._loadSpecificSettings (12:15:12.262) - braille._BgThread (7440):
loading braille brailliantB
INFO - braille.BrailleHandler.setDisplayByName (12:15:12.262) - braille._BgThread (7440):
Loaded braille display driver brailliantB, current display has 40 cells.

@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

These permissions aren't that worrying. The access violation is much more concerning, though.
Ok, let's give this a spin.

@LeonarddeR LeonarddeR marked this pull request as ready for review October 8, 2022 11:00
@LeonarddeR LeonarddeR requested a review from a team as a code owner October 8, 2022 11:00
@LeonarddeR LeonarddeR requested a review from seanbudd October 8, 2022 11:00
@dkager

dkager commented Oct 8, 2022

Copy link
Copy Markdown
Contributor

An access violation in the Windows sleep function especially. :)

@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

An access violation in the Windows sleep function especially. :)

I guess it is crashing because the sleep waits for a handle that has been destroyed. That's why I want to get rid of this APC stuff altogether, it requires a lot of specific Win32 knowledge while in contrast, the Pythonic implementation this pr is much more readable IMO.

@seanbudd seanbudd requested a review from feerrenrut October 9, 2022 23:59

@seanbudd seanbudd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a possibility of added unit tests?

Added @feerrenrut as a reviewer to confirm the new style/approach is appropriate.

Comment thread source/bdDetect.py Outdated
@seanbudd seanbudd added merge-early Merge Early in a developer cycle blocked/needs-code-review labels Oct 10, 2022
Comment thread source/bdDetect.py Outdated
@dkager

dkager commented Oct 12, 2022

Copy link
Copy Markdown
Contributor

Perhaps refactoring should be a separate step beyond implementing a fix for the very real issue of the display detection crashing. I imagine that making braille more deterministic could cause some interesting new issues at first.

@seanbudd seanbudd marked this pull request as draft October 14, 2022 04:38
@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

@seanbudd Not sure why you're marking this as draft? It is ready for review and currently under review.

@seanbudd

Copy link
Copy Markdown
Member

@seanbudd Not sure why you're marking this as draft? It is ready for review and currently under review.

This comment is unresolved: #14147 (comment)

@LeonarddeR LeonarddeR marked this pull request as ready for review October 29, 2022 13:06
@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

I think this is ready again if you agree this case is too complex for unit tests.

seanbudd
seanbudd previously approved these changes Nov 2, 2022
Comment thread source/braille.py Outdated
@seanbudd seanbudd dismissed their stale review November 2, 2022 00:23

mistaken

@seanbudd

seanbudd commented Nov 2, 2022

Copy link
Copy Markdown
Member

For Developers

Reduced the complexity of the background braille display auto detection logic

Other than API breaking change I raised, I don't believe this PR affects developers.
Would a braille display driver developer notice any changes other than better performance from this PR?

@seanbudd seanbudd marked this pull request as draft November 8, 2022 23:23
@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

Other than API breaking change I raised, I don't believe this PR affects developers. Would a braille display driver developer notice any changes other than better performance from this PR?

Not likely. There will be a difference in that an automatically detected braille display driver will now be initialized on a different thread (i.e. the threadpool thread, not braille._BgThread). However:

  1. This shouldn't be problematic as a braille display that is selected explicitly from the braille display selection dialog is initialized on the main thread.
  2. These drivers are supposed to be threadsafe

@seanbudd seanbudd marked this pull request as ready for review November 9, 2022 23:50
@seanbudd

seanbudd commented Nov 9, 2022

Copy link
Copy Markdown
Member

As we don't expect any changes for developers, i.e. this is changes to internal code only, I am going to avoid adding a \change log entry.

@seanbudd seanbudd merged commit 2a96316 into nvaccess:master Nov 9, 2022
@nvaccessAuto nvaccessAuto added this to the 2023.1 milestone Nov 9, 2022
feerrenrut pushed a commit that referenced this pull request Nov 22, 2022
Closes #14130
Related to #14147

Summary:
#14130 notes that the background thread for i/o is too tightly coupled too the braille module.
This required the hwIo module to queue an APC to the braille background thread.

User facing changes:
None, apart from more safety checks that might impose stability improvements.

Changes:
1. Moved the background thread from the braille module.
   - Instead make it a class on the hwIo module.
   - It is also no longer a singleton and therefore add-ons can create their own i/o thread.
2. The APC queue function now takes a callable or lambda.
   - Rather than all cases where APCs have to be queued to the background thread being responsible for wrapping themselves in a winKernel.PAPCFUNC.
   - The provided callable is converted to an APC when queued.
   - There is a cache on the thread class to keep references to wrapped APCs until they are finished.
    - The wrapper also ensures that the APC doesn't execute when the thread is about to exit.
3. hwIo.base.IoBase now contains a new `_initialRead` method that initiates reading on the background thread.
   - This method can be overridden on subclasses to outsource the reading to another thread.
@LeonarddeR LeonarddeR deleted the threadPool branch August 23, 2025 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-early Merge Early in a developer cycle

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants