Skip to content

UIA: Support half checked state for checkboxes and toggle buttons#13980

Merged
seanbudd merged 9 commits into
nvaccess:masterfrom
mzanm:UIAIndeterminate
Aug 15, 2022
Merged

UIA: Support half checked state for checkboxes and toggle buttons#13980
seanbudd merged 9 commits into
nvaccess:masterfrom
mzanm:UIAIndeterminate

Conversation

@mzanm

@mzanm mzanm commented Aug 3, 2022

Copy link
Copy Markdown
Contributor

Link to issue number:

Fixes #13975

Summary of the issue:

NVDA does not support 3 state checkboxes and toggle buttons with UIA. When using WPF or Microsoft edge UIA.

Description of user facing changes

When a checkbox is partially checked, NVDA will report half checked, same as the turn features control panel applet.

Description of development approach

in NVDAObjects\UIA, if toggle state is indeterminate, add half checked state

Testing strategy:

Tested using WPF three-state checkbox and toggle button. And with using UIA chromium-baste browsers on, tested with Microsoft Edge.

Known issues with pull request:

None.

Change log entries:

Bug fixes

  • In programs using UI Automation, Partially checked checkboxes will be reported correctly.

Code Review Checklist:

  • Pull Request description:
    • description is up to date
    • change log entries
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • API is compatible with existing add-ons.
  • Documentation:
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English

@seanbudd

seanbudd commented Aug 4, 2022

Copy link
Copy Markdown
Member

This looks good to me, is there a reason it is still a draft?

@mzanm

mzanm commented Aug 4, 2022

Copy link
Copy Markdown
Contributor Author

It currently says for toggle buttons
Toggle button not pressed half checked
apart from the negative state bug, I'm asking if that's ok as is or a new state would be better to be consistent with pressed and not pressed of toggle buttons.

@feerrenrut

Copy link
Copy Markdown
Contributor

Please ensure that you test with how this affects the reporting of "indeterminate progress bars" which may use the same state, but a different role.

We don't want them to be reported as "progress bar half checked".

See implementation of "busy indicator": #13428
and issues:

@mzanm

mzanm commented Aug 4, 2022

Copy link
Copy Markdown
Contributor Author

This does not effect progress bars as the indeterminate state is fetched using the UIA toggle pattern, which is either on, off, or indeterminate. It's specific to toggle buttons and checkboxes. UIA doesn't seem to have a way to fetch if a progress bar is indeterminate.

@rperez030

Copy link
Copy Markdown

Half pressed for toggle sounds good to me. I cannot tell if it conveys the way it looks visually, but conveys the meaning.

@OzancanKaratas

Copy link
Copy Markdown
Collaborator

@Mazen428, please mark this pull request as ready for review.

@mzanm

mzanm commented Aug 12, 2022

Copy link
Copy Markdown
Contributor Author

I don't have a braille display so can't add or test the state in braille, so if someone could that'd be great

@mzanm mzanm marked this pull request as ready for review August 12, 2022 22:04
@mzanm mzanm requested a review from a team as a code owner August 12, 2022 22:04
@mzanm mzanm requested a review from seanbudd August 12, 2022 22:04
@OzancanKaratas

Copy link
Copy Markdown
Collaborator

@cary-rowen, would you test this on your braille display?

@cary-rowen

Copy link
Copy Markdown
Contributor

Answer what you care about first:
Yes, I tested with Seika Notetaker, the checkbox can render three states in braille.

The sample I tested is as follows(Enchantment table checkBox):
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/checkbox

@cary-rowen

Copy link
Copy Markdown
Contributor

Also, I don't know if it will be off topic.
Still using the sample given above, But with UIA enabled NVDA doesn't report checkbox checked status in time.

NVDA can report the checked state of a checkbox in time when UIA is disabled, but Braille can only have two states, checked or unchecked.

@mzanm

mzanm commented Aug 13, 2022

Copy link
Copy Markdown
Contributor Author

Thank you. Could you test with braille using this WPF application?
Simply unpack the archive and you can find the exe inside the bin folder.
You might need the dot net 6 runtime to run it if it fails to launch.
the app just have 3 state toggle button and a 3 state checkbox.
wpftest.zip
Alternatively, the code is included so you could run it if you have the SDK.

@rperez030

rperez030 commented Aug 14, 2022

Copy link
Copy Markdown

WPF example:

The checkbox works as expected for both speech and Braille. the indeterminate state is represented in Braille with the appropriate symbol for the halfchecked state.
The toggle button reports the three states as expected in speech, but the halfpressed state is represented in braille with the whole state instead of the symbol (half pressed).
Edge with the UIA flag set to Yes:

Checkboxes work as expected for both speech and braille. The half checked checkbox is represented in braille with the appropriate symbol for the halfchecked state.
I also observed that NVDA sometimes fails to report the state change for the checkboxes with UIA enabled, but I believe that should be handled outside this PR.

@mzanm

mzanm commented Aug 14, 2022

Copy link
Copy Markdown
Contributor Author

Ok, I think the half pressed braille state is fixed.
By NVDA not announcing UIA checkbox state, is that with WPF or just chromium UIA?

@AppVeyorBot

Copy link
Copy Markdown

See test results for failed build of commit cbede09d0e

@cary-rowen

Copy link
Copy Markdown
Contributor

By NVDA not announcing UIA checkbox state, is that with WPF or just chromium UIA?
The issue only exists in Edge's web pages.
If you want to reproduce it, you must enable NVDA's UIA support for Edge in Advanced Settings Panel.

@mzanm

mzanm commented Aug 14, 2022

Copy link
Copy Markdown
Contributor Author

I can reproduce this on 2022.2, I'm not sure how to fix it and I agree it should be fixed in another PR.
I've also noticed if you go into focus mode it reports the state changes, so it might be a UIA browse mode state change issue, so it's unrelated to what's done here.

@rperez030

Copy link
Copy Markdown

Ok, I think the half pressed braille state is fixed.

Yes it is. Thanks @Mazen428 for your work on this PR. It has also helped me in my journey to understand NVDA internals.

@seanbudd

seanbudd commented Aug 15, 2022

Copy link
Copy Markdown
Member

I don't have a braille display so can't add or test the state in braille, so if someone could that'd be great

In the future, testing using Braille Viewer should be sufficient.

if s == UIAHandler.ToggleState_Indeterminate:
if role == controlTypes.Role.TOGGLEBUTTON:
states.add(controlTypes.State.HALFPRESSED)
else:

@seanbudd seanbudd Aug 15, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this explicitly check if this is a checkbox?

Suggested change
else:
elif role == controlTypes.Role.CHECKBOX:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are checkable tree view items, so just in case, if it has a toggle pattern then the state is added

Comment thread source/controlTypes/deprecatedAliases.py Outdated
Comment thread source/controlTypes/state.py Outdated

@seanbudd seanbudd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Mazen428!

@seanbudd seanbudd merged commit 157ac49 into nvaccess:master Aug 15, 2022
@nvaccessAuto nvaccessAuto added this to the 2022.4 milestone Aug 15, 2022
@mzanm mzanm deleted the UIAIndeterminate branch August 15, 2022 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NVDA does not support the ToggleState_Indeterminate for a three-state checkbox that implements the UIATogglePattern

8 participants