Skip to content

MozillaCompoundTextInfo: Add special handling for the insertion point at the end of a line.#16745

Merged
seanbudd merged 3 commits into
nvaccess:masterfrom
jcsteh:firefoxEol
Jun 27, 2024
Merged

MozillaCompoundTextInfo: Add special handling for the insertion point at the end of a line.#16745
seanbudd merged 3 commits into
nvaccess:masterfrom
jcsteh:firefoxEol

Conversation

@jcsteh

@jcsteh jcsteh commented Jun 26, 2024

Copy link
Copy Markdown
Contributor

Link to issue number:

Fixes #3156. Strictly speaking, that issue covers other problems as well, but this addresses the issue for which it was originally opened. Other issues should be (re)opened for any remaining problems.

Summary of the issue:

When editing text, there is an insertion point at the end of a line. This is where you are positioned when you press the end key. Firefox also allows you to reach this position with the left/right arrow keys. When text wraps across multiple lines, lines other than the last have this insertion point, but it does not have its own IAccessible2 offset, since it doesn't actually exist in the text. Instead, the caret offset reported is the same offset as the start of the next line.

This is problematic for NVDA because when the user is at this position, NVDA reports the character at the start of the next line. Worse, it reports the next word and line instead of the current word and line.

Description of user facing changes

In Mozilla Firefox, NVDA now correctly reports the current character, word and line when the cursor is at the insertion point at the end of a line.

Description of development approach

IAccessible2 provides IA2_TEXT_OFFSET_CARET to deal with this. While somewhat awkward, the idea is that you pass this special offset to IAccessibleText functions so that they know you care about the caret and can handle it specially if the caret is at the end of line insertion point. For example, asking for the character at the caret might report (3, 3, None), where 3 is the caret offset. In contrast, if the caret was at the start of the next line, it might report (3, 4, 'c'). Similarly, asking for the line at the caret when at the end of the line might report (0, 3, 'ab ') instead of (3, 6, 'cd ') at the start of the next.

Unfortunately, making NVDA use IA2_TEXT_OFFSET_CARET when fetching units is tricky. IA2TextTextInfo would need to keep track of the fact that it was constructed for the caret and pass IA2_TEXT_OFFSET_CARET if so. But this means that when the caret moves, the TextInfo moves too, which breaks detaching the review cursor from the caret, etc.

Instead, a flag _isEndOfLineInsertionPoint has been added to MozillaCompoundTextInfo. This is set by querying the character at IA2_TEXT_OFFSET_CARET when constructing with POSITION_CARET. This allows us to implement special behaviour when this is True: we expand to no character, we expand to the current word/line instead of the next word/line, etc. If the TextInfo is copied, this flag is copied too. If the TextInfo is mutated in any way, the flag is set to False. This allows us to represent this position (even across copies) without being attached to the current position of the caret.

Some of this could theoretically be done in IA2TextTextInfo, but that would have meant plumbing (and coupling) this special casing through both classes. I'm not aware of any other implementation which supports IA2_TEXT_OFFSET_CARET that doesn't use MozillaCompoundTextInfo.

Note that although Chromium supports IA2_TEXT_OFFSET_CARET, it doesn't special case the insertion point at the end of a line. Thus, although this works fine in Chromium, there is no benefit there, though Chromium could choose to implement this to gain that benefit.

Testing strategy:

Test case:
data:text/html,<textarea cols="2">ab cd</textarea>

In Firefox:

  1. Open the test case and focus the textarea.
  2. Press end.
    • Incorrect result before this PR: "c"
    • Correct result after this PR: "blank"
  3. Press read current line.
    • Incorrect result before this PR: "cd"
    • Correct result after this PR: "ab"
  4. Press right arrow.
    • Observe: "c"
  5. Press read current line.
    • Observe: "cd"

In Google Chrome:

  1. Open the test case and focus the textarea.
  2. Press end.
    • Incorrect result (before and after this PR): "c"
  3. Press read current line.
    • Correct result (before and after this PR): "ab"
    • Note that Chromium does this by returning the current line for the real offset if the caret is positioned at the insertion point at the end of a line. However, this causes other problems; e.g. you are unable to move the review cursor to the next line if the caret is at this position.

Known issues with pull request:

None.

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

Summary by CodeRabbit

  • New Features

    • NVDA in Mozilla Firefox now accurately reports the current character, word, and line when the cursor is at the end of a wrapped line.
  • Documentation

    • Updated user documentation to reflect the new enhancement in text reporting for Mozilla Firefox.

@jcsteh jcsteh requested a review from a team as a code owner June 26, 2024 05:24
@jcsteh jcsteh requested a review from seanbudd June 26, 2024 05:24
@coderabbitai

coderabbitai Bot commented Jun 26, 2024

Copy link
Copy Markdown
Contributor

Walkthrough

The recent changes in ia2TextMozilla.py introduce a new attribute, _isEndOfLineInsertionPoint, within the MozillaCompoundTextInfo class. This attribute helps accurately identify when the caret is at the end of a wrapped line in Firefox, thereby addressing text navigation issues like duplicated or skipped lines and words. Adjustments were made to several methods to incorporate this new attribute, ensuring more precise text reporting in multiline edit fields.

Changes

Files Change Summaries
ia2TextMozilla.py Introduced _isEndOfLineInsertionPoint attribute in MozillaCompoundTextInfo. Adjusted methods (__init__, expand, move, compareEndPoints, _findUnitEndpoints, setEndPoint) to handle the caret at the end of a line.
user_docs/en/changes.md Documented the enhancement for accurate reporting in Firefox when the cursor is at the end of a line.

Assessment against linked issues

Objective Addressed Explanation
Resolve Firefox edit fields navigation issue leading to duplicated lines and words (#3156).
Ensure accurate reporting of current character, word, and line at the end of a line in Firefox (user documentation reference).

These changes enhance text navigation in Firefox by accurately identifying the caret's position at the end of lines, leading to better reporting and consistency, addressing the issues mentioned in #3156.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Comment on lines +439 to +447
if (
self._isEndOfLineInsertionPoint and unit != textInfos.UNIT_CHARACTER
and obj is self._startObj and expandTi._startOffset > 0
):
# #3156: This is the insertion point at the end of a line. If we use
# this offset, we'll get the unit at the start of the next line.
# Subtract 1 so that we get the unit on the current line.
expandTi._startOffset -= 1
expandTi._endOffset -= 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optimize the handling of end-of-line insertion points in _findUnitEndpoints.

The logic to adjust the offsets when _isEndOfLineInsertionPoint is True is sound, but this block of code might benefit from a refactor for clarity and potential performance improvements. Consider encapsulating this logic into a method to reduce complexity in _findUnitEndpoints and improve readability.

- if (
-     self._isEndOfLineInsertionPoint and unit != textInfos.UNIT_CHARACTER
-     and obj is self._startObj and expandTi._startOffset > 0
- ):
-     expandTi._startOffset -= 1
-     expandTi._endOffset -= 1
+ self.adjustOffsetsForEndOfLine(expandTi, unit, obj)

And then define a new method:

def adjustOffsetsForEndOfLine(self, expandTi, unit, obj):
    if (
        self._isEndOfLineInsertionPoint and unit != textInfos.UNIT_CHARACTER
        and obj is self._startObj and expandTi._startOffset > 0
    ):
        expandTi._startOffset -= 1
        expandTi._endOffset -= 1

Comment thread source/NVDAObjects/IAccessible/ia2TextMozilla.py Outdated

@seanbudd seanbudd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me other than the coderabbit comments

@seanbudd seanbudd marked this pull request as draft June 26, 2024 23:16
@jcsteh jcsteh marked this pull request as ready for review June 27, 2024 01:30
Comment thread source/NVDAObjects/IAccessible/ia2TextMozilla.py Outdated
@seanbudd seanbudd merged commit 0336466 into nvaccess:master Jun 27, 2024
@jcsteh jcsteh deleted the firefoxEol branch June 27, 2024 12:47
seanbudd pushed a commit that referenced this pull request Jun 28, 2024
…evious character is a line feed. (#16763)

Fixup of #16745.

Summary of the issue:
In my work on #16745, I neglected to consider an empty line after a line feed. In that case, adjusting for the line end caused NVDA to read from the previous line instead of the empty one.

Description of user facing changes
In Firefox, NVDA no longer speaks the previous line when the caret is on the last line and the last line is empty. However, this does not need a change log entry because this bug never shipped in release.

Description of development approach
Don't set the _isInsertionPointAtEndOfLine flag to True if the previous character is a line feed. This prevents the line end adjustment.

Because this additional check makes the code a little more complex, I refactored this code into a helper method.
seanbudd pushed a commit that referenced this pull request Jul 29, 2024
…d of an object. (#16914)

Fixup of #16745.

Summary of the issue:
In my work on #16745, I apparently neglected to consider the end of an inline object such as a link. In that case, adjusting for the line end caused NVDA to report blank when moving to the character after the link instead of reporting the actual character.

Description of user facing changes
When editing text in Firefox, NVDA now reports the correct character instead of blank when pressing right arrow to move out of a link. This doesn't need a change log entry if we can get this into beta because the bug will not have shipped in release.

Description of development approach
Return False in _isCaretAtEndOfLine if we're at the end of an object. While the end of an object could indeed be the end of a line, we don't need the special adjustment in this case and the adjustment causes problems if it isn't the end of a line.

This also means we can remove the change in #16763 because that was only ever a problem for a line feed at the end of an object anyway. This new fix covers both cases
seanbudd pushed a commit that referenced this pull request Aug 27, 2024
…rge files (#17051)

Fixes #17039

Summary of the issue:
With the introduction of pr #16745 which added better reporting of the caret at when at the end of a line, VS Code became quite slow when arrowing up and down through particular large files.
IAccessibleText::textAtOffset with IA2_OFFSET_CARET is very costly in VS Code, and Chromium does not actually implement this correctly yet anyway.

Description of user facing changes
NVDA is less sluggish when arrowing up and down through large files in VS Code.

Description of development approach
VS code appModule: provide a new VsCodeEditorTextInfo which disables end-of-line caret detection by returning False in _IsCaretAtEndOfLine.
seanbudd pushed a commit that referenced this pull request Aug 28, 2024
… yet implement what we need, and it is costly for no gain. (#17067)

More broadly fixes #17039

Summary of the issue:
In PR #16745, code was added to detect the caret at end of lines in Mozilla-compatible edit areas, so as to not report the next line when on the final insertion point of the line before.
Although this works great in Firefox and other Gecko-based apps, it does not work at all in Chromium due to Chromium not correctly
implementing IAccessibleText::textAtOffset with IA2_OFFSET_CARET.
More importantly, this is a very costly check in VS code, which has a noticeable performance hit when arrowing up and down large files (issue #17039), for no gain.
PR #17051 successfully fixed the performance hit in VS code by disabling caret at end of line detection just for VS Code. But it was pointed out that there are several VS Code variants including VS Code.dev which require this also.

Description of user facing changes
NVDA is no longer as sluggish when arrowing up and down large fles in VS code and otherChromium-based applications.

Description of development approach
Remoe the VSCodeditor and VSCodeEditorTextInfo classes from the VS ode appModule introduced in pr VS Code is no longer as sluggish when arrowing up and down through large files #17051.
Add Editor and EditorTextInfo classes to NVDAObjects.IAccessible.chromium which are used in Chromium where ever NVDAObject.IAccessible.ia2Web.Editor is used. These classes disable caret at end of line detection for any chromium edit area.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

firefox edit fields: duplicate llines and words

2 participants