Feature: pointer movement/scrolling by petejohanson · Pull Request #2027 · zmkfirmware/zmk

petejohanson · 2023-11-17T17:41:06Z

Continuation of the incremental integration of the great work from @krikun98 in #778 with the movement and scrolling pieces next. Based on the rebased work from @caksoylar and and the split acceleration from @bryanforbes

TODO

Docs update
Review acceleration curves/functions configurability
Review @manna-harbour Discord discussion on per-device scaling factors.
Review @manna-harbour Discord note on storing float values, and on send of the report discard whole values, but retain the fractional remainder to be included in the follow up calculations, to avoid jerkiness/jumpiness.

krikun98 · 2023-11-17T22:35:52Z

app/src/mouse/key_listener.c

+    k_work_submit_to_queue(zmk_mouse_work_q(), &mouse_tick);
+}
+
+K_TIMER_DEFINE(mouse_timer, mouse_timer_cb, mouse_clear_cb);


The part that initially made me bang my head against the wall with all this is the fact that Zephyr sometimes seems to de-prioritize these timers if the system is under load.
This is (as far as I was able to understand) the primary cause of the instability with mouse movement - reports can be sent out at random intervals or dropped entirely.
I was exploring re-writing this system to use a dedicated thread, but I wasn't skilled enough at C at the time.
I'm not sure this basis for the tick system is robust enough for general use.

So... timers should be using the hardware timers to trigger, and happen in ISR context. The queue has a certain priority, can be preempted, etc. Tweaking that thread priority might help, but I'll play a bit to see what I can determine. Thanks for the insight!

This is most certainly not definitive - it's a half-remembered conclusion from an investigation I wasn't skilled enough to perform two years ago, just food for thought. I distinctly remember reports being delayed or dropped and that causing instability. hog.c:335 might also be worth taking a look at.

caksoylar · 2023-11-18T00:17:31Z

app/include/dt-bindings/zmk/mouse.h

+
+/* Mouse move behavior */
+#define MOVE_Y(vert) ((vert)&0xFFFF)
+#define MOVE_Y_DECODE(encoded) (int16_t)((encoded)&0x0000FFFF)


I think it might make sense to move these decode defines outside the dt-bindings header, and to either plain mouse.h or the corresponding event headers they are used in.

IMHO, they are complementary and should live together.... I think keeping them here is appropriate.

krikun98 · 2023-12-04T20:45:14Z

Could you please change the target to https://github.com/petejohanson/zmk/tree/core/zephyr-3.5-update here? The review is unnecessarily cluttered.

petejohanson · 2023-12-04T20:47:32Z

Could you please change the target to https://github.com/petejohanson/zmk/tree/core/zephyr-3.5-update here? The review is unnecessarily cluttered.

Unfortunately, that will move the PR into a different GH fork, and make a bit of a mess. I recommend reviewing each commit for clarity for now, until the Zephyr 3.5 bits are merged into ZMK main.

app/src/behaviors/behavior_input_two_axis.c

app/src/mouse/main.c

caksoylar

I think these are some missed updates for the rebase to account for 36eda571b

app/src/behaviors/behavior_mouse_key_press.c

app/src/behaviors/behavior_input_two_axis.c

caksoylar · 2024-07-16T19:26:08Z

This isn't abandoned, I am guessing Studio work is getting more priority right now. Pete mentioned on Discord that he is planning to refactor the endpoints a bit (the current split one has some issues on some OS, like Ubuntu 22.04).

Meanwhile if anyone needs a rebased branch in the very near future, you can cherry-pick from https://github.com/caksoylar/zmk/commits/caksoylar/experimental.

Co-authored-by: Alexander Krikun <krikun98@gmail.com> Co-authored-by: Robert U <urob@users.noreply.github.com> Co-authored-by: Shawn Meier <ftc@users.noreply.github.com>

* Add ability to swap X/Y, invert X and Y values, and apply a scalar multiplier/divisor.

* Remove now-unused mouse work queue and related mouse main file. * Move ticks config into a DTS property on the two axis input behavior.

* Corrected logging for two-axis input timestamps.

* Buffer data from input devices and only surface to HID once synd'd.

* Always import mouse keys behavior and their associated listeners. * Tweak listener code to only add listener nodes when listener and the associated input device are enabled.

proostas · 2024-08-31T14:11:59Z

I built a new firmware for my keyboard after the recent changes had been pushed and I noticed a very strange behavior with certain keys.

I have the following settings for mouse behaviors:

&mmv {
    acceleration-exponent = <1>;  // 1
    time-to-max-speed-ms = <300>; // 300
    delay-ms = <0>;
    trigger-period-ms = <8>;      // 16
};
&msc {
    acceleration-exponent = <1>;  // 0
    time-to-max-speed-ms = <40>;  // 300
    delay-ms = <0>;
    trigger-period-ms = <8>;      // 16
};

In both nodes the delay-ms is set to 0. But I clearly have a delay with (and only with) &msc SCRL_DOWN and &mmv MOVE_LEFT now. ~~Could it be caused by some regression?~~

Sorry. Just tested it on another keyboard and the issue is most likely caused by #2042

Yep. I can confirm it now. Once I add at least one Antecedent Morph behavior to the keymap I end up with a very small delay for &msc SCRL_DOWN and &mmv MOVE_LEFT.

After some more digging I discovered that the key positions of &msc SCRL_DOWN and &mmv MOVE_LEFT in my keymap are shared with these combos:

            combo_key_repeat {
                timeout-ms = <U_COMBO_TIMEOUT>; // = 75
                key-positions = <17 18>;
                bindings = <&key_repeat>;
                require-prior-idle-ms = <U_COMBO_STREAK_DECAY>; // = 300
            };
            combo_caps_word {
                timeout-ms = <U_COMBO_TIMEOUT>;
                key-positions = <16 19>;
                bindings = <&caps_word>;
                require-prior-idle-ms = <U_COMBO_STREAK_DECAY>;
            };

And it appears that the combos are the (possibly main) contributors of these lags. But the funny thing is that the lag was non-existent even with combos until antecedent_morph_keycode_state_changed_listener started to run. So, it looks like both combos and Antecedent Morph listener contribute to these lags.

Excluding Mouse layer from global combos, like:

            combo_key_repeat {
                timeout-ms = <U_COMBO_TIMEOUT>; // = 75
                key-positions = <17 18>; 
                layers = <ENG DIK QWR>; // <-------
                bindings = <&key_repeat>;
                require-prior-idle-ms = <U_COMBO_STREAK_DECAY>; // = 300
            };
            combo_caps_word {
                timeout-ms = <U_COMBO_TIMEOUT>;
                key-positions = <16 19>;
                layers = <ENG DIK QWR>; // <-------
                bindings = <&caps_word>;
                require-prior-idle-ms = <U_COMBO_STREAK_DECAY>;
            };

completely fixes this lag. So, I guess global combos shouldn't overlap with Mouse behaviors.

woodies11 · 2024-09-10T00:33:25Z

docs/docs/behaviors/mouse-emulation.md

+
+### Examples
+
+The following will send a scroll down event to the host when pressed/held:


Did you mean mouse movement down event?

caksoylar · 2024-12-14T21:23:05Z

Closing since #2477 is merged.

hailee0710 · 2024-12-19T03:18:44Z

@caksoylar does it mean that I can use the main zmk for pointing instead of this PR?

screaming-service · 2024-12-19T03:25:33Z

I have been using main since, it is working.

petejohanson · 2024-12-19T03:36:27Z

@caksoylar does it mean that I can use the main zmk for pointing instead of this PR?

Correct. See the documentation on zmk.dev for the details.

petejohanson added enhancement New feature or request pointers Pointer related functionality labels Nov 17, 2023

petejohanson self-assigned this Nov 17, 2023

petejohanson force-pushed the feat/pointers-move-scroll branch from 8b2dc31 to bd9210d Compare November 17, 2023 19:38

krikun98 reviewed Nov 17, 2023

View reviewed changes

petejohanson force-pushed the feat/pointers-move-scroll branch from bd9210d to eab4709 Compare November 17, 2023 23:08

caksoylar reviewed Nov 18, 2023

View reviewed changes

petejohanson force-pushed the feat/pointers-move-scroll branch 9 times, most recently from 32ed133 to c2bf21b Compare November 25, 2023 00:46

englmaxi added a commit to englmaxi/zmk that referenced this pull request Nov 26, 2023

Feature: pointer movement/scrolling zmkfirmware#2027

ff04928

englmaxi added a commit to englmaxi/zmk that referenced this pull request Nov 26, 2023

Feature: pointer movement/scrolling zmkfirmware#2027

78b3b89

petejohanson force-pushed the feat/pointers-move-scroll branch from c2bf21b to 07c181d Compare December 4, 2023 19:47

krikun98 reviewed Dec 4, 2023

View reviewed changes

app/src/behaviors/behavior_input_two_axis.c Outdated Show resolved Hide resolved

petejohanson force-pushed the feat/pointers-move-scroll branch 5 times, most recently from 505343b to d837c95 Compare December 4, 2023 23:01

krikun98 reviewed Dec 4, 2023

View reviewed changes

app/src/mouse/main.c Outdated Show resolved Hide resolved

petejohanson force-pushed the feat/pointers-move-scroll branch from d837c95 to a95b68b Compare December 6, 2023 18:23

caksoylar reviewed Dec 7, 2023

View reviewed changes

hendstephen mentioned this pull request Jul 18, 2024

Feat/pointers move scroll hendstephen/zmk#3

Open

caksoylar and others added 14 commits August 3, 2024 21:07

feat(mouse): Add mouse move and scroll support [WIP]

94c17d6

Co-authored-by: Alexander Krikun <krikun98@gmail.com> Co-authored-by: Robert U <urob@users.noreply.github.com> Co-authored-by: Shawn Meier <ftc@users.noreply.github.com>

feat(mouse): Split move/scroll & x/y acceleration

04022d2

fix(mouse): Bump up default mouse tick duration to prevent crashes

c26b481

refactor: Use input system for pointer events.

e38fd78

feat(mouse): Add input configs for data mods.

b2f6163

* Add ability to swap X/Y, invert X and Y values, and apply a scalar multiplier/divisor.

refactor(mouse): Remove mouse work queue, Kconfig

4e53004

* Remove now-unused mouse work queue and related mouse main file. * Move ticks config into a DTS property on the two axis input behavior.

fix(mouse): Fixes for logging.

77acca1

* Corrected logging for two-axis input timestamps.

refactor: Proper per-device input listeners.

5513584

* Buffer data from input devices and only surface to HID once synd'd.

chore: Minor input behavior clean-up.

f8fa8aa

fix: Testing fixes for listener refactor.

dad5926

refactor(mouse): Tweak behavior inclusion, listener code

0c3536c

* Always import mouse keys behavior and their associated listeners. * Tweak listener code to only add listener nodes when listener and the associated input device are enabled.

fix(mouse): Fix up tests and make ZMK_MOUSE explicit.

37ba9eb

feat(mouse): Updated mouse keys docs.

d45cb3c

fix: Various fixes for building.

bc75de5

petejohanson force-pushed the feat/pointers-move-scroll branch from dbbeb70 to bc75de5 Compare August 12, 2024 18:55

petejohanson requested a review from a team as a code owner August 12, 2024 18:55

proostas mentioned this pull request Sep 1, 2024

Antecedent Morph Behavior aka Adaptive Keys #2042

Open

woodies11 reviewed Sep 10, 2024

View reviewed changes

keymap-editor bot added a commit to bithead2000/Adv360-Pro-ZMK that referenced this pull request Sep 12, 2024

mouse keys removed (see zmkfirmware/zmk#2027)

c7d268b

petejohanson mentioned this pull request Sep 13, 2024

feat(mouse): Add mouse move and scroll support #2477

Merged

4 tasks

chase-hunter mentioned this pull request Oct 1, 2024

Mouse clicks chase-hunter/zmk-config-dance#1

Closed

FrostKiwi mentioned this pull request Oct 17, 2024

add: Cirque Pinnacle trackpad driver #1116

Closed

caksoylar closed this Dec 14, 2024


		### Examples

		The following will send a scroll down event to the host when pressed/held:

Uh oh!

Conversation

petejohanson commented Nov 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO

Uh oh!

krikun98 Nov 17, 2023

Choose a reason for hiding this comment

Uh oh!

petejohanson Nov 17, 2023

Choose a reason for hiding this comment

Uh oh!

krikun98 Nov 17, 2023

Choose a reason for hiding this comment

Uh oh!

caksoylar Nov 18, 2023

Choose a reason for hiding this comment

Uh oh!

petejohanson Aug 3, 2024

Choose a reason for hiding this comment

Uh oh!

krikun98 commented Dec 4, 2023

Uh oh!

petejohanson commented Dec 4, 2023

Uh oh!

Uh oh!

Uh oh!

caksoylar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

caksoylar commented Jul 16, 2024

Uh oh!

proostas commented Aug 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

woodies11 Sep 10, 2024

Choose a reason for hiding this comment

Uh oh!

caksoylar commented Dec 14, 2024

Uh oh!

hailee0710 commented Dec 19, 2024

Uh oh!

screaming-service commented Dec 19, 2024

Uh oh!

petejohanson commented Dec 19, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

15 participants

petejohanson commented Nov 17, 2023 •

edited

Loading

proostas commented Aug 31, 2024 •

edited

Loading