Skip to content

Automatically announce selection changes in Delphi Grids and other inaccessible controls#8414

Closed
LeonarddeR wants to merge 11 commits into
nvaccess:masterfrom
BabbageCom:behaviors
Closed

Automatically announce selection changes in Delphi Grids and other inaccessible controls#8414
LeonarddeR wants to merge 11 commits into
nvaccess:masterfrom
BabbageCom:behaviors

Conversation

@LeonarddeR

Copy link
Copy Markdown
Collaborator

Link to issue number:

Closes #7418

Summary of the issue:

In many legacy applications that have a bad accessibility implementation, we can still make these applications quite accessible using the display model. Even when an application does not fire focus events or doesn't implement accessibility API support, the display model can be used to find out what part of a window is relevant to report. An example of this is a Delphi Grid, where all information in the grid is only accessible by means of the display model, and in which case we can track the focus based on DrawFocusRect API calls. Even if DrawFocusRect isn't called, we can use the display model text info to find out what part of the window is considered selected and should therefore be reported.

Description of how this pull request fixes the issue:

This pr is split up into several area's which are more or less related to each other.

Changed behaviors structure

  1. The monitoring specific stuff in behaviors.LiveText is now abstracted into behaviors.TextMonitor. LiveText inherrits from TextMonitor
  2. Based on this change, there is now a DisplayModelTextMonitor class. DisplayModelLiveText now inherrits from LiveText and DisplayModelTextMonitor

changes to documentBase and editableText

  1. The selection support from editableText.EditableText has been abstracted into documentBase.SelectableTextContainer. editableText.EditableText now inherrits from SelectableTextContainer. The major reason why I did this, is that the fact that selectable text in a container doesn't necessarily mean that the window contains a caret.
  2. While most functionality from SelectableTextContainer is coming from editableText.EditableText, I've made an additional change which allows unselected text not to be spoken. IN this case, the selected text is spoken with speech.SpeakTextInfo, which avoids unnecessarily calculation of what text has to be spoken. It also doesn't speak the redundant "selected" word every time a new selection is spoken.

New behaviors

  1. As behaviors.TextMonitor is now abstract, we can build more types of monitors upon its functionality. the SelectionChangeMonitor combines the TextMonitor with the new SelectableTextContainer to automatically announce selection changes when there is a change in the text.
  2. Based on this, there is window.DisplayModelSelectionChangeMonitor that announces the selection when the display model detects a change in the text.
  3. There is also window.DisplayModelDrawFocusRectProcessor, which is meant to be used on windows which have no focusable children, yet communicate focus changes in the window with the displayModel_drawFocusRectNotify event. This is applicable to delphi grids, for example, but also to several inaccessible list controls.

Enhanced display model selection offsets detection

Display model text info selection offsets detection was based on the background and foreground selection colors reported by Windows. This can give excelent results, but is somewhat limited as it doesn't allow detection of selection when the selection is communicated to the visual user in a different way, either by different colors or font attributes. Therefore, I borrowed the idea of property conditions from UIAUtils to create one or more conditions that should be met in order to have selection to be reported. To evaluate a condition, you can simply throw one or more condition dicts at the evaluateCondition method of a textInfos.Field instance.

To give some real life examples. I've seen several situations in the wild where selection had to be detected based on two different combinations of foreground and background color. There are also cases where the selection can be detected based on only the foreground color.

User visible changes

  1. NVDA will now speak selection updates in Delphi grids
  2. A change that we'll have to debate on, NVDA will now add the DisplayModelSelectionChangeMonitor overlay class for ContentGenericClient objects when the selection can be detected within these object's windows. This is intended to improve the base line level of accessibility within ContentGenericClient windows.

Testing performed:

  1. Tested focus rectangle behavior in Delphi Grids
  2. Tested the selection based focus container in several work place specific applications

Known issues with pull request:

Similar to terminal announcement behavior, the Selection based monitor is a bit limited in how it detects new selections and announces them. For example, when you select a list item that is continuously updating its value (e.g. when the selected item contains a counter) the whole item is repeated when the counter updates.

Change log entry:

@LeonarddeR LeonarddeR changed the title Behaviors Automatically announce selection changes in Delphi Grids and other inaccessible controls Jun 18, 2018
@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

This pr is pretty big, it might be better to revisit this after the Python 3 transition and file it in some smaller bits, e.g.

  1. Enhanced dictionary based selection conditions for display model text info
  2. Abstraction of selection tracking vs. caret tracking and selection monitoring
  3. Specific code for pretty generic inaccessible controls, such as lists and grids for which the only way of fetching the selection is by means of the display model

@Adriani90

Copy link
Copy Markdown
Collaborator

I am labeling this as abandoned until someone takes this over.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Abandoned requested reports or updates are missing since more than 1 year, author or users are not available. BabbageWork Pull requests filed on behalf of Babbage B.V.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Speak selection changes in Delphi grids

2 participants