Skip to content

Ignore events where Java Access Bridge is unable to provide a window handle#13796

Merged
seanbudd merged 5 commits into
nvaccess:masterfrom
aphtech:jabInvalidHwnd
Jun 27, 2022
Merged

Ignore events where Java Access Bridge is unable to provide a window handle#13796
seanbudd merged 5 commits into
nvaccess:masterfrom
aphtech:jabInvalidHwnd

Conversation

@mwhapples

@mwhapples mwhapples commented Jun 14, 2022 •
edited by seanbudd
Loading

Copy link
Copy Markdown
Contributor

Link to issue number:

Closes #13039

Summary of the issue:

When launching IntelliJ IDEA you minimise all windows with Windows+M before the IntelliJ window comes into view, then you get many errors and IntelliJ will take a long time to load. In addition NVDA will log many errors.

Description of how this pull request fixes the issue:

It seems like if a Java window never gets shown then Java Access Bridge is unable to get a window handle for it. Also the caching of window handles in NVDA's Java Access Bridge is unable to help as the Java window was never in focus. As the affected window is in the background and as there is nothing more NVDA can do to solve this I have checked for the presence of a window handle in the Java Access Bridge event handling, if no window handle is present then the event is ignored.

Testing strategy:

Manual testing. Tested using the steps in the issue. Also did some other use of Java applications to observe if this fix has negative impacts anywhere else, none found.

Known issues with pull request:

Technically these events could be processed as the Java Access Bridge objects contain all accessible information, so we are throwing away events when may be we could process them. However due to NVDA Window objects insisting on a window handle we cannot proceed with processing due to the NVDA limitation.

Change log entries:

New features
Changes
Bug fixes

  • Ignore Java Access Bridge events where no window handle can be found for Java applications. This will improve performance for Java applications under specific circumstances.
    For Developers

Code Review Checklist:

  • Pull Request description:
    • description is up to date
    • change log entries
  • Testing:
    • Unit tests: Not known how to write tests for this part of the code.
    • System (end to end) tests: Not known how to write such tests for this part of the code.
    • Manual testing Done
  • API is compatible with existing add-ons.
  • Documentation:
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English

@mwhapples
Ignore events where Java Access Bridge is unable to provide a window …
aa655cb 
…handle, prevents errors when trying to create the NVDA objects
@mwhapples mwhapples requested a review from a team as a code owner June 14, 2022 14:02
@mwhapples mwhapples requested a review from seanbudd June 14, 2022 14:02
@mwhapples mwhapples mentioned this pull request Jun 14, 2022
Closed
@AppVeyorBot

AppVeyorBot commented Jun 14, 2022

Copy link
Copy Markdown

See test results for failed build of commit 493469ab01

seanbudd
seanbudd reviewed Jun 14, 2022
View reviewed changes
Comment thread source/JABHandler.py
seanbudd
seanbudd reviewed Jun 14, 2022
View reviewed changes
Comment thread source/JABHandler.py
@AppVeyorBot

AppVeyorBot commented Jun 15, 2022

Copy link
Copy Markdown

See test results for failed build of commit 5dd5e92750

@seanbudd seanbudd linked an issue Jun 16, 2022 that may be closed by this pull request
NVDA throws a lot of errors during opening IntelliJ IDEA #13039
Closed
@seanbudd seanbudd marked this pull request as draft June 17, 2022 04:47
@mwhapples mwhapples marked this pull request as ready for review June 21, 2022 13:42
@seanbudd seanbudd self-assigned this Jun 22, 2022
@mwhapples mwhapples mentioned this pull request Jun 23, 2022
Open
seanbudd
seanbudd previously approved these changes Jun 27, 2022
View reviewed changes
@seanbudd

seanbudd commented Jun 27, 2022

Copy link
Copy Markdown
Member

Thanks @mwhapples

@seanbudd

seanbudd commented Jun 27, 2022

Copy link
Copy Markdown
Member

Sorry @mwhapples I just noticed this part in the PR description:

However due to NVDA Window objects insisting on a window handle we cannot proceed with processing due to the NVDA limitation.

Can you point out where this occurs? Perhaps this is something we can avoid and fix for JAB.

@seanbudd seanbudd dismissed their stale review June 27, 2022 03:26

stale - better approach may be possible

@mwhapples

mwhapples commented Jun 27, 2022

Copy link
Copy Markdown
Contributor Author

Sorry @mwhapples I just noticed this part in the PR description:

However due to NVDA Window objects insisting on a window handle we cannot proceed with processing due to the NVDA limitation.

Can you point out where this occurs? Perhaps this is something we can avoid and fix for JAB.

For the vast majority of Java Access Bridge calls a window handle is not needed. The only ones which come to mind are the functions where one wants to check if a window is a Java application or to get the window handle for a Java application. This means that for gathering the information about the application and what has changed in an event does not need a window handle. Thus Java Access Bridge API does allow us to continue processing the event without a window handle.
The limitation from NVDA comes when constructing instances of class NVDAObjects.JAB.JAB. This inherits from NVDAObjects.window.Window, which requires a window handle. NVDAObjects.JAB.JAB if not passed a windoe handle to its constructor will attempt to get the window handle for the (java application, however under certain circumstances it may fail to get the window handle. One such case is where a (java application is started but the user minimises all windows with Windows+M before the Java window is shown.
There is a question as to what accessibility events for an application apparently without a window handle means for a screen reader, where does that application fit in the wider system? With (java Access Bridge one could get the top level ancestor of the event source and check if that has the active state to determine whether the application is the foreground window (no window handle needed). This at least lets the screen reader know if the event is from the foreground or a background application.
I don't know all the cases where the Java Access Bridge is unable to report a window handle for a Java application. If it only relates to cases where the (java application is in the background, then performing that above check is pointless and we may as well assume failed to get window handle means background.
In most cases NVDA does not do anything with events from background applications, this is why my solution to this issue was to simply ignore the event as it seemed the simplest option. However if NVDA should want to make use of events from background java applications, then we need to find a way around this limitation that we must be able to obtain the window handle when constructing a NVDAObjects.JAB.JAB instance.

@seanbudd seanbudd merged commit 8bc1bd8 into nvaccess:master Jun 27, 2022
@nvaccessAuto nvaccessAuto added this to the 2022.3 milestone Jun 27, 2022
@seanbudd

seanbudd commented Jun 29, 2022

Copy link
Copy Markdown
Member

In most cases NVDA does not do anything with events from background applications, this is why my solution to this issue was to simply ignore the event as it seemed the simplest option. However if NVDA should want to make use of events from background java applications, then we need to find a way around this limitation that we must be able to obtain the window handle when constructing a NVDAObjects.JAB.JAB instance.

I'm sure this is technically possible, but if we do not have the motivation to do so, there is no need to do the work.

@seanbudd

seanbudd commented Jun 29, 2022

Copy link
Copy Markdown
Member

Thanks @mwhapples

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@seanbudd seanbudd seanbudd approved these changes

Assignees

@seanbudd seanbudd

Labels

None yet

Projects

None yet

Milestone

2022.3

Development

Successfully merging this pull request may close these issues.

NVDA throws a lot of errors during opening IntelliJ IDEA

4 participants

@mwhapples @AppVeyorBot @seanbudd @nvaccessAuto