Skip to content

Fully support mobile keyboard in web TextEdit#8068

Closed
cjgriscom wants to merge 6 commits intoemilk:mainfrom
cjgriscom:text-agent-textarea
Closed

Fully support mobile keyboard in web TextEdit#8068
cjgriscom wants to merge 6 commits intoemilk:mainfrom
cjgriscom:text-agent-textarea

Conversation

@cjgriscom
Copy link
Copy Markdown

@cjgriscom cjgriscom commented Apr 5, 2026

Mobile keyboard (e.g. Gboard, MS SwiftKey) support is limited and glitchy (see #8046). The keyboard state resets after every input, causing capitalization bugs on SwiftKey and blocking useful mobile functionality. The old resetting workaround had been applied to solve word duplication bugs, due to the lack of input handlers for mobile autocomplete.

This PR modifies the input handler to fully support mobile text suggestions and autocomplete. It does so by keeping the egui text buffer synchronized with text_agent's DOM to provide the keyboard with the necessary context.

There were two ways to approach this:

  • Adjust text_agent to support full multiline buffer sync using <textarea>
  • Modern EditContext API

I opted for replacing the with a <textarea> because support for EditContext is missing in web-sys and non-Chromium browsers.

Buffer synchronization is challenging due to the JS UTF-16 conversion. I use a UTF-16 code-unit counting routine to ensure the cursor positions remain correct.

I am unable to test on iOS but have confirmed correct functionality on these targets:

Chrome, Firefox on Linux
Chrome mobile and Firefox mobile with Samsung Keyboard, SwiftKey, Gboard

Screenshot_20260405_134704_Firefox

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 5, 2026

Preview available at https://egui-pr-preview.github.io/pr/8068-text-agent-textarea
Note that it might take a couple seconds for the update to show up after the preview_build workflow has completed.

View snapshot changes at kitdiff

@cjgriscom cjgriscom changed the title Fully support mobile keyboard in TextEdit (#8046) Fully support mobile keyboard in TextEdit web (#8046) Apr 5, 2026
@cjgriscom cjgriscom changed the title Fully support mobile keyboard in TextEdit web (#8046) Fully support mobile keyboard in web TextEdit Apr 5, 2026
@umajho
Copy link
Copy Markdown
Contributor

umajho commented Apr 6, 2026

Nice work.

I also considered synchronizing the entire text between egui and TextEdit. However, since the original approach is $O(1)$ while full synchronization is $O(\text{text size})$, I was concerned about potential performance degradation and decided not to go for it.
In practice, though, given that egui is an immediate-mode UI, the impact is likely negligible. Since I haven't benchmarked it, my thought may well be a case of premature optimization. (My approach (#8045) is also $O(N)$, but $N$ is limited to a small chunk of text being written, which should remain modest in typical usage.)

One clear advantage of this PR over both the original approach and my own (#8045) is that it enables auto-completion to continue working when the cursor is moved. While this can be partially worked around on top of my approach (e.g., by prepending text after resetting the text agent), this PR provides a more robust solution.

Copy link
Copy Markdown
Owner

@emilk emilk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on iOS - works much better than the old code. Thank you!

@emilk emilk added this to the 0.35.0 milestone Apr 6, 2026
@cjgriscom cjgriscom force-pushed the text-agent-textarea branch from 8923784 to 6b516c4 Compare April 6, 2026 14:17
@cjgriscom
Copy link
Copy Markdown
Author

@umajho That's a good point. I like your idea, but it wasn't working for me on the latest commit (keyboard is still resetting?). I opted for the simplest approach, but maybe partial text replacement is superior if you can manage the unicode difficulties.

@emilk Thanks for looking at it - I rebased on main to resolve the conflict

@rustbasic
Copy link
Copy Markdown
Contributor

rustbasic commented Apr 6, 2026

@umajho @cjgriscom

Regarding #8068, when typing the Korean word "않았다" on Android (using the Cheonjiin keyboard), a part of the first character is duplicated, resulting in "안않았다."

I’ve confirmed that this issue is resolved in #8045. It seems that some additional work might be needed to apply this fix to #8068 as well.

@cjgriscom
Copy link
Copy Markdown
Author

cjgriscom commented Apr 6, 2026

@umajho @rustbasic Is it just a matter of applying this: f139c60 ?

@umajho
Copy link
Copy Markdown
Contributor

umajho commented Apr 6, 2026

@umajho … but it wasn't working for me on the latest commit (keyboard is still resetting?). …

That's odd. #8045 appears to fix the keyboard reset issue based on my testing:

Prior to #8045 With #8045
2026-04-06 10 46 09 PM 2026-04-06 10 46 25 PM

@rustbasic
Copy link
Copy Markdown
Contributor

rustbasic commented Apr 6, 2026

@umajho @cjgriscom

Yes, it appears that the keyboard reset issue has also been resolved in #8045.

@umajho
Copy link
Copy Markdown
Contributor

umajho commented Apr 6, 2026

… Is it just a matter of applying this: f139c60 ?

The fix in #8045 was under the assumption that only a small portion of the text exists in the text agent. Specifically, the text originally retained in the text agent is compared with the text updated in the text agent during IME composition, and based on that comparison, it determines what action to take.

The code changes to on_composition_end in f139c60 are only part of the fix: changes in places like on_input are also part of it.

To be honest, since the assumption has now changed to the text agent containing the entire synchronized text, I'm not really sure how to migrate the fixes implemented in my PR anymore…

@cjgriscom
Copy link
Copy Markdown
Author

cjgriscom commented Apr 6, 2026

@umajho You're right, it seems to work on the demo now. I must have done something wrong

IME is bigger in scope than what I'd like to address here, so I'd rather defer to your PR in that case. I left a note there about one remaining issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Keyboard layout is reset after every keystroke in text input on mobile

4 participants