Skip to content

Composable Shell: handle different aspects of modern input panel via automation ID lookup #9104

@josephsl

Description

@josephsl

Hi,

A slightly better way of handling #9103:

Background

Windows 10 Fall Creators Update (Version 1709/build 16299) introduced emoji panel (powered by Composable Shell) for browsing and inserting emojis. This facility was expanded in April 2018 Update (Version 1803/build 17134) to include hardware keyboard input suggestions and dictation, and October 2018 Update (Version 1809/build 17763) introduced cloud clipboard paste along with adding skin tone modifiers for people emoji. The current 19H1 preview (build 18305) expands emoji panel portion to include kaomoji and symbols, and emoji panel will open to the last browsed emoji category instead of most recently used group when it opens.

Until now NVDA just assumed that an object located on a specific UIA tree node was emoji list. However, it led to problems such as not announcing clipboard empty status (#9103), not announcing top emoji search suggestion in Version 1809 and later (#9105), and attribute error due to the input panel parent object not being a UIA object (#9106). This became more prominent in October 2018 Update where the UIA automation ID's are different for different features of input panel.

Describe the solution you'd like

The pull request to follow reorganizes modern input panel support by grouping handlers based on automation ID's instead of just build number. This brings several benefits:

  1. Easier isolation of different features of input panel via automation ID's.
  2. Better readability.
  3. Solutions for problems from future builds can be traced via feature sets/build combination.

One possible drawback is lengthy event handler routine for one or more events, but with descriptions of features and bug fixes, it should make it easy for people to spot things. Also, note that automation ID's can change without notice, so one needs to keep an eye on it.

Describe alternatives you've considered

The alternative is using build lookup, but given a release may introduce multiple features to input panel with different automation ID's it would be unmanageable.

Additional context

Due to advances in UIA throughout Windows 10 development cycle, things can change (automation ID's, for instance), so one must keep an eye on it.

Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions