UI Automation in Windows Console: add STABILIZE_DELAY, only enable UIA when available, limit blank lines in review, make speaking of passwords configurable, improvements to "speak typed words", and initial word movement support#9646
Closed
codeofdusk wants to merge 12 commits into
Conversation
…legacy consoles). This helps NVDA process large amounts of text.
Note: there are issues when reaching the bottom of the review (cursor gets stuck).
This should mostly restore caret movement support.
This approach works on my system, but needs wider testing.
…rators for UIA console options in the GUI.
…ffer when enter or tab is pressed. Note: this will need to be investigated once nvaccess#8110 is merged.
…xtInfo._getCurrentOffset. The reverse option has been removed.
Member
|
Could you please replace this pr with smaller specific prs please? Perhaps one that handles blank lines / move by words (as that is all changes to a TextInfo). Then one for UIA when available (I.e. moving winConsole to IAccessible etc), and finally one for speak typed characters/words. It makes it a lot easier to describe the specific issue in the pr, and easier to review. |
This was referenced May 31, 2019
Collaborator
|
Since you're now replacing this with smaller ones,I'm closing this one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to issue number:
Builds on #9614.
Summary of the issue:
Currently, in consoles with UI Automation enabled:
_isTypingflag is not cleared on the console, so new text is not announced immediately.Description of how this pull request fixes the issue:
Word movement and blank line filtering have been implemented in an overridden
movemethod onconsoleUIATextInfo. Since the console's UIA implementation does not provide word movement, we must use an offset-based word movement algorithm which uses the Uniscribe API to find word boundaries. The algorithm used for this implementation is based on that inNVDAObjects.offsets._getWordOffsets. Blank lines are filtered by comparing thetextInfo's_rangeObjto the last visible text range (returned bywinConsoleUIA.UIATextPattern.GetVisibleRanges).When "speak passwords in UIA consoles" is disabled and one of "speak typed characters" or "speak typed words" is enabled,
typedCharacterevents are trapped and queued until atextChangeevent is received. InwinConsoleUIA.event_textChange, typed characters are removed from the queue andtypedCharacterevents are fired. InwinConsoleUIA.script_clear_isTyping, this queue is flushed. Since password prompts do not update the screen while the user is typing, notextChangeevents are received, so notypedCharacterevents are fired and therefore no characters are announced.The legacy console code has been moved to
NVDAObjects.IAccessible.winConsoleLegacy, and the check for consoles has been moved fromNVDAObjects.window.findOverlayClassestoNVDAObjects.IAccessible.findOverlayClasses. Since NVDA prefers UIA over MSAA, UIA is used when enabled and available and legacy is used otherwise.The
speech.curWordCharsbuffer is cleared when atypedCharacterof enter or tab is received. This prevents NVDA from speaking partially-typed words (during tab completion) or the last word of a command (when enter is pressed). This approach will need to be modified once #8110 is merged, as it significantly changes handling of typed words.The interrupt characters control+c, control+d, and control+break have been bound to the
script_clear_isTypingscript onNVDAObjects.UIA.winConsoleUIA.Testing performed:
Tested password entry (by authenticating with
ssh), text review, and all combinations of the "use legacy console" and UIA checkboxes on Windows 10 versions 1803 and 1903.Known issues with pull request:
typedCharacterevents are only dispatched oncetextChangeis received. This may introduce a small performance penalty.winConsoleUIA._isTypingis cleared), reporting of new text and speak typed words may not function correctly.Change log entry:
None.