Keyman Core Integration for Windows

Table of Contents

1. Introduction
2. Architecture
3. Changes Required to Keyman Engine for Windows

Introduction

A keyboard processor implements keyboarding rules. The current Keyman engine on windows has a native processor implementing keyboarding rules for Keyman formatted keyboards (kmx) only. The Keyman core desktop is a Keyman processor that will allow the processing of different keyboard rule formats. Currently, it is used by Keyman for Linux. This item of work is to integrate this Keyman core processor into Keyman for Windows. Allowing the Keyman core to cross-platform shared logic. This will enable future keyboard formats to be implemented including LDML Keyboards.

Architecture

See the original design document for diagram

Diagram 1.0 gives an overview of how the Keyman Core processor interacts with the Keyman Engine for Windows. The Keyman Keyboard Processor API is consumed by the kmprocess interfacing with its Application Action Queue. The Keyman Core processor shall manage the keyboard rules, state, keyboard options and context. The core processor shall be included as a static library. The context and action queue shall be cached in the Keyman Engine for Windows. This will be used to update the core prior to processing each key event. After each key event, the context cache and action queue in the Keyman Engine for Windows shall be updated from the Keyman Core.

Changes Required to Keyman Engine for Windows

Include the core Keyman Keyboard Processor as a static library.

Use Registry Flag for Initial Integration.

A registry flag for turning on and off the new core processor shall be provided. This will ease initial integration and regression testing. The flag shall be named REGSZ_Flag_UseKeymanCore and be included with the current debug flags. The following changes in the rest of this specification are stated as if the core integration flag is set. If it is not set it will use the existing processor and data structures.

Change to K32_load.cpp

There will be one instantiation of the processor per keyboard.

• LoadlpKeyboard shall call internally km_kbp_keyboard_load of km_kbp_keyboard_api in the Keyman Core Processor library. It shall also set the km_kbp_state which maintains the context and actions together with a reference to the keyboard. km_kbp_state shall be stored in the thread data structure. It will use km_kbp_state_create. See Keyman Change tagKEYMAN64THREADDATA. For Example km_kbp_state_create(_td->lpKeyboards[i].coreKeyboard, test_env_opts, &_td->lpKeyboards[i].lpActiveKBState

• A new function shall be written to load the registry options by using the interface call km_kbp_state_options_update. It shall mimic the current IntLoadKeyboardOptions the registry format needs to be identical to avoid corruption. km_kbp_state shall be passed in as a reference argument to the new function.

• Keyboard options shall be saved to the registry using a new function that will replace the existing IntSaveKeyboardOption which shall use the km_kbp_state_option_lookup interface call. This will be called when processing the km_kbp_state_action_items call after a key event . IntSaveCoreKeyboardOption shall also pass in the km_kbp_state.

• Add km_kbp_keyboard_get_imx_list Which returns the list of IMX libraries and function names that are referenced by
  the keyboard. The windows engine then can load the libraries ( windows DLLs) ready for when the core makes a call back one of the functions defined in the libraries (DLLs).

Change to ProcessHook() in Kmprocess.cpp

kmprocess.cpp shall be modified so that is sending the business logic for keyboard processing to the core processor using the API. The following points cover both the flow of data and the changes required.

• On Keystroke Event:ProcessHook is the entry point where the Keyman Core Processor shall be called to process the keystroke via km_kbp_process_event.

• Initialise and set up the core processor: the state is created when loading the keyboard. Check it has been created.

• Set latest context in core processor: A call shall be made to the Application Interface Text Input Processor (AITIP) to get the current context. Calling the method ReadContext ensures the dead keys are inserted from the existing cached context. The cached context will be used to create a km_kbp_context_item array using km_kbp_context_items_from_utf16. Write a utility function to do this as the api provided function will not recognise the windows deadkey implementation. The formatted context shall then be set calling km_kbp_context_set. This shall be set immediately before calling km_kbp_process_event in the same thread.

• Process keyboard rules for the latest keystroke: km_kbp_process_event shall use the km_kbp_state,the current vkey value and the modifier state. Globals::get_ShiftState() shall be used to set the modifier state. The integer value is defined in compiler.h. Finally, the key up/down state is set.

• Update context in Keyman Engine for Windows from Keyman Core processor: On successful completion of km_kbp_process_event the Keyman Engine for Windows cache context shall NOT be updated directly but rather by applying the actions from the actions queue see next point. to match the context in the core processor engine which has possibly changed due to the processing of the keystroke event. km_kbp_context_items_to_utf16 shall be used to access the context and a method to update the cached context of the AITIP shall be added. The core engine maintains the deadkey markers, which is required.

• Load the actions from the Keyman Core to Keyman Engine for Windows - AppActionQueue. The processor actions shall be accessed by calling km_kbp_state_action_items The list of actions shall be iterated over and applied to the AppActionQueue using the AITIP::QueueAction method.

  • fOutputKeystroke shall be set when an action of type KM_KBP_IT_EMIT_KEYSTROKE is encountered. It shall be commented in code that this can become a local flag set in ProcessHook and not a global flag once we deprecate the existing core implementation. The synthesising of the keystroke should be done in the same iteration.

• Apply the actions to the Application using Keyman Engine for Windows: Call SendActions(): No change is required here at this point the actions and context have been loaded into the Keyman for Windows Engine to execute.

• Special Note the windows api for each keystroke calls the client (keyman engine) up to two times. The first time the is just to see if the client wants to consume the keystroke and apply and or apply an action. If the client returns it does it will be called by the windows API again and it can now apply the action. The call to the km_kbp_process_event will only occur on the first call from the windows api. On the second call the windows engine will apply the cached actions as described above.

Change to tagKEYMAN64THREADDATA

Change to INTKEYBOARDINFO

Add

  km_kbp_keyboard* lpCoreKeyboard;
  km_kbp_option_item* lpCoreKeyboardOptions;
  km_kbp_state* lpCoreKeyboardState;
  km_kbp_keyboard_imx* lpIMXList;

Add these three members at the end of the struct so the first part of the struct aligns with keymanapi.h.

Change to KMSTATE (in keyman64.h)

Add a comment that the following members are needed for Keyman Engine for Windows. Add a TODO #5442 to remove when the old engine is finally removed.

BOOL StopOutput;
int LoopTimes;
LPKEYBOARD lpkb; 
LPGROUP startgroup;

Add

km_kbp_keyboard* lpCoreKb;  

Change to preservedkeymap.cpp

Add km_kbp_keyboard_get_key_list that returns an unordered full set of modifier+virtual keys that are handled by the
keyboard. The matching dispose of call needs to be called to free the memory.
Using this list the windows engine can traverse the list and notify the windows system of any keys it needs to preserve so it will be notified when these keys are pressed.

Change to calldll.cpp

• IM_CallBackCore This is called by the core processor when a call(name) rule is matched. This will have the imx_id for the imx library and function to be called.
• KMQueueActions and KMSetOutput shall use km_kbp_state_queue_action_items to queue action items in the core keyboardprocessors queue.
• KMGetContext and KMSetOutput shall both use kbp_state_get_intermediate_context to get the intermediate context in the core keyboard processor.

Change to kmhook_getmessage.cpp

• When the event wm_keymanim_close is triggered call km_kbp_process_queued_actions for the core keyboard processor to process its queued actions. This is an event outside a keystroke press, which has differences to the km_kbp_process_event call which called on a keystroke.

Related issues and discussions

• Original design document

Limitations - Now a TODO list

There are limitations that will not be addressed in phase one #5443 of the integration of the core processor these are:

• Integrate core feat(windows): Keyman Core integration 🥑 #5443
• feat(windows): Keyman Core Integration - Support for IMX DLLs 🥑 #5650
• feat(windows): Keyman Core Integration - Full support for Caps Lock system stores 🥑 #5652
• feat(windows): Saving/restoring keyboard options before/after processing a key stroke 🥑 #5653
• feat(windows): Implement an API call in common core so that a platform engine can register preserved keys.🥑 #5777
• feat(windows): Emit keystrokes from IMX dll call backs - keystroke processing optimization 🥑 #6099

Keyman for Windows/macOS/Linux:

• OS: Windows
• Keyman version: target 15.0