Skip to content

A keyboard input model #753

@pyfisch

Description

@pyfisch

TLDR: I think that Winit needs more expressive keyboards events and to follow a written specification to keep platform inconsistencies to a minimum. I propose to adapt the JS KeyboardEvent for winit and to follow the UI Events specification for keyboard input.

Winit is used for many applications that need
to handle different kinds of keyboard input.

  • Games: Physical location of keys like
    WASD for movement and actions.
    Text inpput for names and chat.
  • GUI applications: Text input and keyboard shortcuts.
  • the Servo Browser: Wants to support JS KeyboardEvent well.

Currently there are two events for text input in Winit:
KeyboardInput and ReceivedCharacter.

pub struct KeyboardInput {
    pub scancode: ScanCode,
    pub state: ElementState,
    pub virtual_keycode: Option<VirtualKeyCode>,
    pub modifiers: ModifiersState,
}

The KeyboardInput event carries information about keys pressed and released.
scancode is a platform-dependent code identifying the physical key.
virtual_keycode optionally describes the meaning of the key.
It indicates ASCII letters, some punctuation and some function keys.
modifiers tells if the Shift, Control, Alt and Logo keys are currently pressed.

The ReceivedCharacter event sends a single Unicode codepoint. The character can
be pushed to the end of a string and if this is done for all events the user
will see the text they intended to enter.

Shortcomings

This is my personal list in no particular order.

  1. List of VirtualKeyCode is seen as incomplete (Enable Less and Greater Keys on X11 #71, Support keypress events with non-ascii chars #59).
    Without a given list it is hard to decide which keys to include
    and when the list is complete.
    Also it is necessary to define each virtual key code so multiple platforms will
    map keys to the same virtual key codes.
    While it probably uncontroversial that ASCII keys should be included
    for non-ASCII single keys found on many keyboards like é, µ, or ü
    it is more difficult to decide and to create an exhaustive list.
  2. While VirtualKeyCode should capture the meaning of the key there
    are different codes for e.g. "0": Key0 and Numpad0 or LControl and RControl.
  3. The ScanCode is platform dependent. Therefore apps wanting to use keys like
    WASD for navigation will assume an QWERTY layout instead of
    using the key locations.
  4. It is unclear if a key is repeated or not. Some applications only want to
    act on the first keypress and ignore all following repeated keys. Right
    now these applications need to do extra tracking and are probably not
    correct if the keyboard focus changes while a key is held down. ( A way to disable key repeats #310)
  5. A few useful modfiers like AltGraph and NumLock are missing.
  6. There is no relation between ReceivedCharacter and KeyboardInput
    events. While this is not necessary for every application some
    (like browsers) need it and have to use ugly (and incorrect) work-arounds. (Associate received characters with key inputs #34)
  7. Dead-key handling is unspecified and IMEs (Input Method Editors) are not supported.

In general there are many issues that are platform-dependant and where it is
unclear what the correct behavior is or it is not documented.
Both alacritty and Servo just to name two applications have multiple
issues where people mention that keyboard input does not work as expeced.

Proposed Solution

Winit is not the first software that needs to deal with keyboard input on
a variety of platforms. In particular the web platform has a complete
specification how keyboard events should behave which is implemented on
all platforms that Winit aims to support.

While the specification talks about JS objects it can be easily ported
to Rust. Some information is duplicated in KeyboardEvent for
backwards compatibility but this can be omitted in Rust so Winit stays simpler.

See the keyboard-types for how keyboard events can look like in Rust.

  • (shortcoming 1) VirtualKeyCode is replaced with a Key. This is an enum
    with all the values for functional keys and a variant for Unicode values
    that stores printable characters both from the whole Unicode range.
    Specification
  • (shortcoming 2) is also adressed by this. There is just one value for keys
    like "Control" but if necessary one can distinguish left/right or
    keyboard/numpad keys by their location attribute.
  • (shortcoming 3) ScanCode is complemented by Code. Codes describe
    physical key locations in a cross-platform way.
    Specification
  • (shortcoming 4) a repeat attribute is added.
  • (shortcoming 5) All known modifier keys are supported.
    **Specification
    Note: W3C decided to include some keys that are usually handled
    in hardware and don't emit keyboard events (like Fn, FnLock)
  • (shortcoming 6) received characters and keyboard events are now one
    (exceptions see below)
  • (shortcoming 7) to handle dead keys and IMEs a composition event
    is introduced. It describes the text that should be added at
    the current cursor position. Specification
    Note: The introduction composition events makes it a bit harder to
    get "just the text" which is currently emitted by ReceivedCharacter.
    Either ReceivedCharacter is kept around for easier use or a utility
    function is provided that takes keyboard and composition events and
    emits the printable text.

Implementation

This is obviously a breaking change so there needs to be a new release of winit and release notes.
While the proposed events are very expressive it is possible to convert Winit to the new
events first and then improve each backend to emit the additional information about key-codes,
locations, repeating keys etc.

Thank you for writing and maintaining Winit! I hope this helps to get a discussion about keyboard input handling started and maybe some ideas or even the whole proposal is implemented in Winit.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions