Abstract vision framework with included support for focus, navigator object and browse mode caret highlighting#9064
Conversation
|
Support from @nishimotz is very welcome I think. As part of the japanese NVDA Team, he is the creator of the add-on focus highlight |
|
I am testing the AppVeyor build (nvda_snapshot_pr9064-16441,7aaee3c0.exe) |
| except NotImplementedError: | ||
| rects = None | ||
| if rects: | ||
| index = 0 if self.obj.isTextSelectionAnchoredAtStart else -1 |
There was a problem hiding this comment.
I got errors as follows regarding this line:
ERROR - queueHandler.flushQueue (13:47:02.601):
Error in func _loadBufferDone from eventQueue
Traceback (most recent call last):
File "queueHandler.pyc", line 53, in flushQueue
File "virtualBuffers\__init__.pyc", line 464, in _loadBufferDone
File "browseMode.pyc", line 1152, in event_treeInterceptor_gainFocus
File "browseMode.pyc", line 1465, in event_gainFocus
File "vision\__init__.pyc", line 601, in handleGainFocus
File "vision\NVDAHighlighter.pyc", line 63, in updateContextRect
File "vision\__init__.pyc", line 270, in updateContextRect
File "vision\__init__.pyc", line 190, in getContextRect
File "vision\__init__.pyc", line 205, in _getRectFromTextInfo
NameError: global name 'self' is not defined
…a gainFocus, since the updated object might have a caret in it and we also want to account for this
|
nvda_snapshot_pr9064-16574,1ca723ff.exe fails initialization as follows: I cannot import this via Python console as well: |
|
Thanks @nishimotz! This should have been fixed in the most recent code. |
…n the chain. We don't want random global plugins to override vision enhancement provider gesture people might rely on. Also update some copyright headers
…merge conflict and adding vision enhancement providers to scratchpad.
…rt due to an error caused by locationHelper conversion
|
Except there is a failing unit test, "No module named 'wx'" |
|
Ah, I'm sorry for that. This all has to do with the linter complaining about module imports have to be at the top of the file.
|
|
@feerrenrut I'm afraid I had to use some noqa in the unit module to get this work. This is because we have to change the path before importing modules, and this is something Flake8 doesn't agree with. The sourceEnv noqa is intentional, as this module is only imported to manipulate the path. |
| # Let us import modules from the NVDA source. | ||
| sys.path.insert(1, SOURCE_DIR) | ||
| import sourceEnv | ||
| import sourceEnv # noqa: F401 |
There was a problem hiding this comment.
I'm fine with using noqa to suppress this case, however please include a comment about why and a hint about what the lint error is. It's not much fun for a future developer to have to look these up and understand why it is like this.
There was a problem hiding this comment.
What would be proper syntax for this? I assume something like this is not going to work
| import sourceEnv # noqa: F401 | |
| import sourceEnv # noqa: F401 module imported but unused |
There was a problem hiding this comment.
Yep something like that works, though it gets messy if you have multiple errors to ignore.
I would do:
# F401: module imported but unused. We need sourceEnv for unit tests because of ?
import sourceEnv # noqa: F401
There was a problem hiding this comment.
I think I've done something along those lines.
|
Mm, I was ignoring linter warning F402, which had to be E402. Interesting enough, Ignoring F402 also ignored E402 unexpectedly, that's why I didn't see this before. |
|
PR introduces Flake8 errors 😲 See test results for Failed build of commit 6fbc40b7de |
| import appModuleHandler | ||
| appModuleHandler.initialize() | ||
| # Anything which notifies of cursor updates requires braille to be initialized. | ||
| # Vision needs a wx app to be available. |
There was a problem hiding this comment.
From a unit testing perspective it would be better if we can avoid this. Which part of the vision framework relies on a wx app being available? Can we mock this part instead?
feerrenrut
left a comment
There was a problem hiding this comment.
Generally this looks good. But I am a little worried about the dependencies of vision, if possible I would prefer it rely on dependency injection if possible. Can we start by identifying what fails if wx is missing?
|
wx is used to make sure that the gui is initialized (CallAfter) before initializing providers. This, because providers might rely on the gui. The highlit"ghter, for example, relies on gui.mainFrame.handle to be the parent of its window.
By default, there is no provider active. So it might make sense to check for configured providers before using wx.CallAfter. If we ever decide to write a pseudo provider, this requires another revisit.
We can also mock wx.CallAfter, e.g. 'lambda func, *args, **kwargs: func(*args, **kwargs)'. Other code might benefit from this too.
|
* CustomWindow.className should be abstract, which is now possible in Python 3. * Ensure that when calling del on a window, handle the case where handle is not yet set on the window * Add additional logging to windowUtils Window destruction. * Use the typing module
* In Python 2, we couldn't join the highlighter thread, as it stopped the quit message from arriving. This now seems to work again. * Reinitialization is no longer broken. We should create the transparent brush at window class creation time and not at class construction time.
… navigator object and browse mode caret highlighting (PR nvaccess#9064)
…Console A few new entries were missing as of PR nvaccess#9064 and PR nvaccess#9790
Pr provided on behalf of @BabbageCom
Link to issue number
Implements #971, but without a gui
Background
For years, NVDA has been an open source screen reader for the blind and partially sighted people who rely on braille and speech output. There are no built-in facilities within NVDA to manipulate the contents of the screen. There are several outstandign issues asking for such features, such as:
There is also a long standing desire to have a basic screen magnification facility within NVDA.
This pr intends to lay the base of a vision framework that can be used to implement such functionality in the core of NVDA. Though there is no GUI in the current pull request, the framework is fully working. The GUI will be added in a follow up pr.
A new vision module
Most magic is supposed to happen within a new vision module. The vision module consists of two distinct concepts:
Vision enhancement providers: providers of functionality that change the contents of the screen. A vision enhancement provider can be a magnifier, highlighter, or whatever else that changes the screen contents.
This pr contains a highlighter based on a transparent overlay window on which lines are drawn around the several objects on screen.
The Screen Curtain add-on, which is a prototype add-on written by me on behalf of @BabbageCom and ported to an add-on by @josephsl, is supposed to be added to NVDA in subsequent pr.
Vision enhancement providers will also be able to benefit from the functionality to set driver/provider specific options in the gui (Allow setting braille display specific settings in the GUI #8214).
The vision handler, which receives information about focus, foreground, navigator object, review and mouse changes from the core of NVDA. It keeps track of the location of objects on screen and communicates this information to one or more active vision enhancement providers. This information is communicated to vision enhancement providers using extension points. The handler keeps track of active enhancement providers and can activate and deactivate them. Therefore, it is very much similar to the braille handler in its approach.
Follow up strategy
@feerrenrut has requested for this framework to be filed into subsequent pull requests. After this pr has been fully reviewed and merged, I intent to file at least two additional pr's:
Known issues
The current highlighter colors might be difficult for some people to see. In the future, I'd like this to be configurable.
Changelog entries: