Skip to content

Ensure outlook AutoCompleteListItems are highlighted when selected#18484

Merged
seanbudd merged 5 commits into
nvaccess:masterfrom
BabbageCom:outlook-autocomplete-highlighter
Jul 29, 2025
Merged

Ensure outlook AutoCompleteListItems are highlighted when selected#18484
seanbudd merged 5 commits into
nvaccess:masterfrom
BabbageCom:outlook-autocomplete-highlighter

Conversation

@Nerlant

@Nerlant Nerlant commented Jul 16, 2025

Copy link
Copy Markdown
Contributor

Link to issue number:

Closes #18483

Summary of the issue:

The NVDA Highlighter does not highlight Outlook recipient auto-complete entries

Description of user facing changes:

The NVDA Highlighter highlights Outlook recipient auto-complete entries correctly

Description of developer facing changes:

Description of development approach:

Testing strategy:

  • Ensure the NVDA Highlighter is turned on
  • Click the "New Email" button in Outlook
  • Navigate to the "To" recipient edit field
  • A list with contact suggestions should open
  • Navigate into the list to select one of the suggested contact entries

Known issues with pull request:

  • The highlighting does not return to the actual recipient edit field when the auto-complete list is closed

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

@coderabbitai summary

Copilot AI review requested due to automatic review settings July 16, 2025 10:21
@Nerlant Nerlant requested a review from a team as a code owner July 16, 2025 10:21
@Nerlant Nerlant requested a review from seanbudd July 16, 2025 10:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes an issue where the NVDA Highlighter was not highlighting Outlook recipient auto-complete entries when selected. The solution adds vision handler integration and navigator object updates to ensure proper highlighting behavior when users navigate through auto-complete suggestions.

  • Adds vision handler integration to handle focus events for auto-complete list items
  • Updates navigator object to maintain proper focus tracking for highlighting

Comment thread source/appModules/outlook.py
Comment thread source/appModules/outlook.py
@seanbudd

Copy link
Copy Markdown
Member

The highlighting does not return to the actual recipient edit field when the auto-complete list is closed

This sounds like a potentially blocking issue. Why isn't this the case?

@Nerlant

Nerlant commented Jul 17, 2025

Copy link
Copy Markdown
Contributor Author

The highlighting does not return to the actual recipient edit field when the auto-complete list is closed

This sounds like a potentially blocking issue. Why isn't this the case?

The auto-complete list and its items are not focusable and never receive system focus. The only events the NVDAObject of the AutoCompleteListItem receives, are the event_stateChange and event_selection events when navigating through the list.
There seems to be no event fired on the list items when the list is closed, either through selecting an item, or closing the list by pressing escape.

By setting the NVDA-internal focus and nagivator object to these list items, the Highlighter highlights them correctly. That is what I am doing so far.

The edit field themselves receive event_valueChange events when a item from the list is confirmed with enter and inserted into the edit field, or when typing in the edit field. By providing a custom class for these edit fields, it is possible to transfer focus back to them when their value changes.
This works for all cases, except when the list is closed by dismissing it (pressing escape). In that case the focus would stay on the (hidden) list, until the edit field value changes or the focus is set to a different element.

If you can think of a way to direct focus back to the edit field when the list is dismissed, I would be more than happy to try to implement it.

@LeonarddeR

Copy link
Copy Markdown
Collaborator

... except when the list is closed by dismissing it (pressing escape). In that case the focus would stay on the (hidden) list, until the edit field value changes or the focus is set to a different element.

How are you validating what events are fired? With MSAA/Events debugging categories?
I would expect the hide event when the auto complete list disappears.

@Nerlant

Nerlant commented Jul 18, 2025

Copy link
Copy Markdown
Contributor Author

How are you validating what events are fired? With MSAA/Events debugging categories? I would expect the hide event when the auto complete list disappears.

Currently I am overriding all event_* methods defined by the NVDAObject class in the AutoCompleteListItem class to log which events are fired. This does not have a hide event method though.

I know that the winEvent EVENT_OBJECT_HIDE exists. Is there a way to retrieve it in classes that inherit from NVDAObject?

@LeonarddeR

LeonarddeR commented Jul 18, 2025

Copy link
Copy Markdown
Collaborator
  1. You can just define event_hide on the object class.
  2. The proper way to inspect events would be enabling debug logging and explicitly opt into the MSAA/Events categories in NVDA's advanced settings. This gives you all the events fired.

@CyrilleB79

Copy link
Copy Markdown
Contributor

It would make sense to do something similar to what happens in other auto-complete lists.

Looking at the start menu, when I type in the search field and then press arrows, the focus and nav object highlighters move between suggestions. Pressing NVDA+numpad5 reports the current suggestion, but pressing NVDA+tab reports the edit field. So the reported focus with NVDA+tab and its highlighted position are somewhat inconsistent.

On the opposite, the experience with Firefoxaddress bar seems similar to what is described in this PR.

In any case, the fact that the highlighter does not turn back to the edit field when the suggestions are dismissed is problematic IMO.

@Nerlant

Nerlant commented Jul 21, 2025

Copy link
Copy Markdown
Contributor Author

I have spent quite some time investigating the events fired on the auto-complete list and its items, as well as on the parent elements of the list. Unfortunately, I could not find any event, like hide or similar, to detect when the list is closed.

The only solution that I could think of, that works, is to detect the escape keypress itself using a script on the edit fields. When detecting the escape keypress there and directing the focus and navigator object back it it, everything seems to work as intended.

I have also looked at the general Highlighter behavior on the Windows start menu. There it seems that only the navigator object is set to the selected item while the focus object remains on the edit field. If this is the desired behavior, I can adjust my code to also only set the navigator object.

@CyrilleB79

Copy link
Copy Markdown
Contributor

The only solution that I could think of, that works, is to detect the escape keypress itself using a script on the edit fields. When detecting the escape keypress there and directing the focus and navigator object back it it, everything seems to work as intended.

At first glance, seems nice to me. Are there drawbacks to this solution? Or uncovered use cases / workflows?

I have also looked at the general Highlighter behavior on the Windows start menu. There it seems that only the navigator object is set to the selected item while the focus object remains on the edit field. If this is the desired behavior, I can adjust my code to also only set the navigator object.

Here it should be discussed and agreed to know which is the desired behaviour? Start menu behaviour or Firefox address bar? Then you can change your code according to what is chosen.

@Nerlant

Nerlant commented Jul 22, 2025

Copy link
Copy Markdown
Contributor Author

... everything seems to work as intended.

At first glance, seems nice to me. Are there drawbacks to this solution? Or uncovered use cases / workflows?

None that I can currently think of.

@SaschaCowley

Copy link
Copy Markdown
Member

... everything seems to work as intended.

At first glance, seems nice to me. Are there drawbacks to this solution? Or uncovered use cases / workflows?

None that I can currently think of.

@Nerlant a few that spring to mind:

  • What about when the user clicks outside the autocomplete list? Does that cause the autocomplete list to disappear? If so, does the highlight rectangle move appropriately?
  • What about when the user types text that initially returns matches, but when they continue typing suggestions are no-longer available? E.G. I type "Jane" and Outlook suggests "Jane Doe" and "Jane Blogs", but then I keep typing " St" and there's no-one of that description in my address book, so no autocompletions are offered.
  • What about when the user presses enter or clicks on a suggestion to accept it?

@Nerlant

Nerlant commented Jul 22, 2025

Copy link
Copy Markdown
Contributor Author
  • What about when the user clicks outside the autocomplete list? Does that cause the autocomplete list to disappear? If so, does the highlight rectangle move appropriately?

When the user clicks on another element that can have focus, the highlight rectangle moves to the element with focus.

  • What about when the user types text that initially returns matches, but when they continue typing suggestions are no-longer available? E.G. I type "Jane" and Outlook suggests "Jane Doe" and "Jane Blogs", but then I keep typing " St" and there's no-one of that description in my address book, so no autocompletions are offered.
  • What about when the user presses enter or clicks on a suggestion to accept it?

As soon as the value of the contact edit field changes, the focus is set back to it. This applies both to entering text in the edit field and to accepting suggestions.

@CyrilleB79

Copy link
Copy Markdown
Contributor
  • What about when the user clicks outside the autocomplete list? Does that cause the autocomplete list to disappear? If so, does the highlight rectangle move appropriately?

When the user clicks on another element that can have focus, the highlight rectangle moves to the element with focus.

Even if this focusable element is the field where they have typed the first characters?

In any case, even if there are such corner cases, in my opinion the benefit is higher than the little bugs introduced by the corner cases (if any).

@Nerlant

Nerlant commented Jul 22, 2025

Copy link
Copy Markdown
Contributor Author

When the user clicks on another element that can have focus, the highlight rectangle moves to the element with focus.

Even if this focusable element is the field where they have typed the first characters?

When clicking into the edit field for which the auto-complete list is currently opened, the list closes for a short moment before re-opening. The highlighter rectangle then resets to the first entry of the auto-complete list, which is automatically selected by Outlook.

Comment thread source/appModules/outlook.py Outdated
@seanbudd

Copy link
Copy Markdown
Member

Please add a changelog entry under bug fixes

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

@seanbudd seanbudd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Nerlant - just need a change log entry

@seanbudd

Copy link
Copy Markdown
Member

@Nerlant - can you please resolve merge conflicts? or grant me permission to push to your branch

@Nerlant

Nerlant commented Jul 29, 2025

Copy link
Copy Markdown
Contributor Author

@Nerlant - can you please resolve merge conflicts? or grant me permission to push to your branch

I will resolve them and request a review from you again

@seanbudd seanbudd enabled auto-merge (squash) July 29, 2025 09:01
@seanbudd seanbudd merged commit e16ae19 into nvaccess:master Jul 29, 2025
21 of 22 checks passed
@SaschaCowley SaschaCowley added this to the 2025.3 milestone Aug 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Highlighter does not highlight Outlook recipient auto-complete entries

6 participants