Skip to content

Braille input: emulate keys outside text content and support combining input with modifiers#7843

Merged
michaelDCurran merged 10 commits into
nvaccess:masterfrom
BabbageCom:i7306
Apr 15, 2018
Merged

Braille input: emulate keys outside text content and support combining input with modifiers#7843
michaelDCurran merged 10 commits into
nvaccess:masterfrom
BabbageCom:i7306

Conversation

@LeonarddeR

@LeonarddeR LeonarddeR commented Dec 9, 2017

Copy link
Copy Markdown
Collaborator

Link to issue number:

Follow up of #6213/#7503
Fixes #7306.

Summary of the issue

Although the braille input system has been vastly improved due to support of contracted input, combining braille input was not yet supported. Also, even though letter quick navigation was supported in the forward direction, navigating backwards did not work as well.

Description of how this pull request fixes the issue:

This pr addresses the following:

  1. No longer always send unicode characters to the system, but in the case of non-editable text, emulate a gesture. See Single letter navigation from a braille terminal #7306 (comment) . This means that pressing dot1257 in US English 8 dot, which translates to capital H, results in shift+h being sent to the system, which will go to the previous heading.

  2. In the case of key emulation, make sure we only emulate one key stroke at once. This is different from sending unicode chars, which allowed sending whole words at once.

  3. The point above means that we had to deal with contracted braille in non-editable cases. It turned out that the liblouis partial trans mode works perfectly for this. I tested with both Unified english braille code grade 2 and English US grade 2 that at least all the alphanumeric characters work. They do not translat to their contracted equivalents, so this seems to be the perfect method to use a contracted table in uncontracted mode .

  4. Also discussed in Single letter navigation from a braille terminal #7306 is the possibility to use braille input together with modifiers. As noted by @jcsteh in Single letter navigation from a braille terminal #7306 (comment) , the implementation for this had to be quite different from Use braille display keys as modifier keys #6213, al though I have been able to integrate them as much as possible.

    A. New, unbound scripts have been added to toggle control, shift, alt, windows and NVDA. Here, toggling means that the state of these keys will be enabled or disabled within the scope of braille input. For example, when you assign control to space+dot7 and you press space+dot7, than dot 125, control+h will be emulated. This also works for contracted tables due to partial translation mode.
    B. The toggable modifier scripts integrate in the modifier system that already existed (Use braille display keys as modifier keys #6213/Basic support for braille display key modifiers #7503), For example, when key1 is bound to toggle control, key 2 to kb:shift and key3 to kb:down arrow, you can press and release key 1 to enable control, than key2+key3 to emulate control+shift+down arrow.

Testing performed:

  • Tested Unified English braille grade 2 and English US grade 2 in non-editable text cases, no contracted translation did occur and thus the implementation worked as expected.
  • Tested assigning modifier toggle scripts and combined them with braille input in both editable and non-editable cases.
  • Tested combining modifier toggle scripts with emulate scripts, as explained above.
  • Tested some braille input cases that, in contracted tables, always result in multiple characters (e.g. dot123456). NVDA reported correctly that this braille input was not supported.

Known issues with pull request:

  • It is not possible to combine emulated modifiers (e.g. kb:shift) with braille input. Having support for this would mean a rewrite of all drivers that use gesture classes that inherrid both from BrailleDisplayGesture and BrailleInputGesture.
  • The way the nVDA key is emulated whe nusing the toggle script is a bit hacky, but coming up with a cleaner way would mean a massive rewrite of the way the NVDA key is used in the keyboard handler. There is now a new function in the keyboardHandler that returns a list of keys that have been set as NVDA key. When emulating a gesture containing the NVDA key, the real key being emulated is either caps lock, extended insert or numpad insert, which internally translates back to the NVDA key. This is a hthing I'm not quite happy with, so any feedback to improve this would be extremely helpful. Note that using the modifier toggle script for the NVDA key will result in undefined behavior when you toggle the NVDA key, then press pass next key through, then press a braille input combination.

Change log entry:

  • New features

    • You can now toggle control, shift, alt, windows and NVDA from your braille keyboard and combine these modifiers with braille input (e.g. press control+s). (Single letter navigation from a braille terminal #7306)
      • You can assign these new modifier toggles using the commands found under Emulated system keyboard keys in the Input gestures dialog.
  • Changes

  • Bug fixes

@AAClause

AAClause commented Jan 15, 2018

Copy link
Copy Markdown
Contributor

I've just tested this. It works perfectly. :)

I have two suggestions:

  1. When entering a shortcut, NVDA says "NVDA pressed", "Control pressed", etc. but we have no feedback in braille. Could you add this information in braille also?
    In my addon BrailleExtender, I display this in this form: "Alt+...", "Control+...", "NVDA+Control+...".
  2. Could you define some default gestures for some braille displays?
    In BrailleExtender, I have defined these gestures for Baum, Brailliant and Focus displays:
    • n+space: NVDA;
    • Dot1+dot2+space (⠉+space): Control;
    • Dot2+dot5+space (⠒+space): Alt;
    • Dot3+dot6+space (⠤+space): Windows;
    • Dot7+dot8+space (⣀+space): Shift.
      Except the NVDA key, all modifiers can be combined in the same time (CTRL+SHIFT, CTRL+ALT, etc.).

What do you think?

@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

@Andre9642 commented on 15 Jan 2018, 04:40 CET:

  1. When entering a shortcut, NVDA says "NVDA pressed", "Control pressed", etc. but we have no feedback in braille. Could you add this information in braille also?

When does the current code say this? It should only do it when speaking commands is on, when it is off and it says the shortcut, it is actually a bug :) As I'd like to bind this with speaking command keys, I think your request is beyond the scope of this pr.

  1. Could you define some default gestures for some braille displays?

This will be a part of another issue, most likely #2945.

@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

I'd say, let's delay this until 2018.2. I don't think it's a good idea to have too many braille patches end up in 2018.1. New native drivers as well as auto tethering feels ok to me.

@derekriemer

Copy link
Copy Markdown
Collaborator

I just edited the what's new part to fix pres to press.

@michaelDCurran

Copy link
Copy Markdown
Member

Was more work going on in this PR, or should this be incubated now that 2018.1 is complete?

@LeonarddeR

Copy link
Copy Markdown
Collaborator Author

I think this is ready to be incubated in order to get tested as broadly as possible.

michaelDCurran added a commit that referenced this pull request Mar 8, 2018
michaelDCurran added a commit that referenced this pull request Mar 28, 2018
@michaelDCurran michaelDCurran merged commit 61c9f91 into nvaccess:master Apr 15, 2018
@nvaccessAuto nvaccessAuto added this to the 2018.2 milestone Apr 15, 2018
@LeonarddeR LeonarddeR added the BabbageWork Pull requests filed on behalf of Babbage B.V. label Oct 11, 2019
seanbudd pushed a commit that referenced this pull request Sep 23, 2021
…d braille tables when in text controls (#12850)

Fixes #12667

Summary of the issue:
When entering text from a braille display NVDA reports that it is unsupported even though entered combination is valid. This check has been introduced in #7843 to prevent a situation in which someone uses contracted table outside text content and enters a combination which cannot be converted to a single character (a good example is entering dots 356 using English UK Grade 2 which is a contraction for "was"). The check however was too broad and didn't take into account that for literary / computer braille tables single combination of dots can either always be converted to a single character, or if not it does not result in multiple characters i.e. number sign.

Description of how this pull request fixes the issue:
Now the check applies only when the table in use is contracted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

BabbageWork Pull requests filed on behalf of Babbage B.V. component/braille feature/browse-mode

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Single letter navigation from a braille terminal

5 participants