Skip to content

Implement new displayChunk unit for display model#10165

Merged
michaelDCurran merged 4 commits into
nvaccess:masterfrom
BabbageCom:displaymodelReadingChunk
Sep 9, 2019
Merged

Implement new displayChunk unit for display model#10165
michaelDCurran merged 4 commits into
nvaccess:masterfrom
BabbageCom:displaymodelReadingChunk

Conversation

@LeonarddeR

@LeonarddeR LeonarddeR commented Sep 4, 2019

Copy link
Copy Markdown
Collaborator

Link to issue number:

None

Summary of the issue:

In a display model, pieces of text are saved as separate chunks. This information is currently not available from NVDA APIs itself.
However, when trying to label text fields or combo boxes in an inaccessible application using a display model, this information is still pretty important, as a label is usually stored in one chunk.

Example

  1. Make a DisplayModelTextInfo for the desktop using displayModel.DisplayModelTextInfo(fg, "all")
  2. Using the getTextInChunks method on the textInfo, iterate through the text.
  3. You will notice that it is not possible to find out where the boundaries are between the several desktop icons. For example, using word offsets cuts the this pc icon into two words, whereas using line offsets gives you an entire row of "desktop icons.

Description of how this pull request fixes the issue:

I implemented support to get the distinc display model chunks from the display model. This is mapped to a new unit constant, displayModel.UNIT_DISPLAYCHUNK.

Testing performed:

  1. Tested on the desktop that desktop icons are treated as single chunks, also when their names contain spaces. Note that icon names spanning multiple lines (having a hard line break) are still treated as two chunks.
  2. Tested in a delphi application that text field labels containing spaces are treated as single chunks.

Known issues with pull request:

  1. In many cases, white space is treated as a single chunk, for example the white space between desktop icons. However, when getting text chunks per line, the line separator is also stored in a separate
  2. See also limitations mentioned by @michaelDCurran in Implement new displayChunk unit for display model #10165 (comment)

Change log entry:

  • Changes for developers
    • Introduced displayModel.UNIT_DISPLAYCHUNK as a textInfos unit constant specific to DisplayModelTextInfo.
      • This new constant allows walking over the text in a DisplayModelTextInfo in a way that more closely resembles how the text chunks are saved in the underlying model.

@michaelDCurran

Copy link
Copy Markdown
Member

A couple of points:

  • Although the majority of basic labels will be written as one chunk, there is no guarantee. If the label has multiple formats or fonts within its text, or the text goes over multiple lines, it will most likely be written in multiple chunks.
  • Although this does not apply to labels, it is worth mentioning that for some other displayModel text, if it is highlightable or selectable, and the selection/highlight changes, this will break up the text into smaller chunks.

I think I would like to see a new constant added. I feel that this usage of readingChunk is not really matching what readingChunk was originally for. I think calling it something like displayChunk is fine. I can't see how you could realy use it logically without knowing that you were dealing with displayModels anyway.

@LeonarddeR

Copy link
Copy Markdown
Collaborator Author
* Although the majority of basic labels will be written as one chunk, there is no guarantee. If the label has multiple formats or fonts within its text, or the text goes over multiple lines, it will most likely be written in multiple chunks.

Correct. I think cases where font weight, color or size differs for a single label are very limited. There is the multiple lines label though, indeed.

* Although this does not apply to labels, it is worth mentioning that for some other displayModel text, if it is highlightable or selectable, and the selection/highlight changes, this will break up the text into smaller chunks.

Thanks for pointing this out. So, I assume you're saying that if a piece of text is written in one large single chunk, and a part of that text is highlighted at some point, that that causes a split of the chunks, and that splitting continues if the highlight moves again, resulting in increasingly smaller chunks?

I think I would like to see a new constant added. I feel that this usage of readingChunk is not really matching what readingChunk was originally for. I think calling it something like displayChunk is fine. I can't see how you could realy use it logically without knowing that you were dealing with displayModels anyway.

This makes much sense. IN fact, I had a previous implementation where I used a separate constnat. I will add such a constant again to the current pr.

@LeonarddeR LeonarddeR changed the title Implement readingChunk unit support for display model Implement new displayChunk unit for display model Sep 9, 2019
@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

@michaelDCurran I updated the title and description of the pr accordingly.

@michaelDCurran michaelDCurran merged commit a9bce46 into nvaccess:master Sep 9, 2019
@nvaccessAuto nvaccessAuto added this to the 2019.3 milestone Sep 9, 2019
michaelDCurran added a commit that referenced this pull request Sep 9, 2019
Also fix some indenting in changes for Developers section.
@LeonarddeR LeonarddeR deleted the displaymodelReadingChunk branch September 24, 2019 05:11
@LeonarddeR LeonarddeR added the BabbageWork Pull requests filed on behalf of Babbage B.V. label Oct 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

BabbageWork Pull requests filed on behalf of Babbage B.V. component/display-model

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants