Skip to content

Improve robustness of TextAgent in eframe/web#8045

Draft
umajho wants to merge 12 commits intoemilk:mainfrom
umajho:fix-cheonjiin
Draft

Improve robustness of TextAgent in eframe/web#8045
umajho wants to merge 12 commits intoemilk:mainfrom
umajho:fix-cheonjiin

Conversation

@umajho
Copy link
Copy Markdown
Contributor

@umajho umajho commented Mar 29, 2026

This PR primarily aims to address an issue with the Samsung Keyboard Korean Cheonjiin layout found by @rustbasic.
Since I don't have access to the same environment, I'll need to rely on their feedback for validation.

Additionally, I need to gather feedback on whether these changes introduce regressions in other setups, so this PR is not ready.

See also: https://stackblitz.com/edit/egui-text-agent-experiment?file=src%2FApp.tsx

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 29, 2026

Preview available at https://egui-pr-preview.github.io/pr/8045-fix-cheonjiin
Note that it might take a couple seconds for the update to show up after the preview_build workflow has completed.

View snapshot changes at kitdiff

@umajho
Copy link
Copy Markdown
Contributor Author

umajho commented Mar 29, 2026

@rustbasic Hello.

I'd appreciate it if you could test whether this solution works with the Cheonjiin IME you are using.
Since I'm unable to reproduce the bug on my end, there is a fair chance that this first attempt may not fully resolve it, though I believe the overall approach is on the right direction.

It would also be very helpful if you could try this in other web setups to check for any regressions.
(No need to test native setups, as these changes should not affect them.)

Thanks in advance.

///
/// `before_chars` and `after_chars` are the number of characters (not
/// bytes) to delete before and after the cursor, respectively.
DeleteSurrounding {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually not an ad hoc variant introduced just for this PR. A variant with the same name was added to winit in 0.31 (though in this implementation, the fields are defined as *_chars rather than *_bytes).

https://docs.rs/winit/0.31.0-beta.2/winit/event/enum.Ime.html#variant.DeleteSurrounding

@KonaeAkira
Copy link
Copy Markdown
Contributor

I can confirm this PR fixes the 2 issues mentioned in #8046, at least on my device.

@rustbasic
Copy link
Copy Markdown
Contributor

@umajho
I tested it on Android + Cheonjiin IME.
The cursor position keeps moving forward, and character deletion behaves strangely.

rustbasic-2026-03-31.mp4

@umajho
Copy link
Copy Markdown
Contributor Author

umajho commented Mar 31, 2026

Wow, that's bad.

If you go Settings -> System -> Languages & input -> On-screen keyboard, what does it show?
For example, here is mine:

Screenshot Screenshot 2026-03-31 at 6 05 27 PM (The Android Keyboard doesn't provide Korean.)

Could you also check https://stackblitz.com/edit/egui-text-agent-experiment and record what it looks like when you input text?
For example, this is what it looks like with GBoard on Android:

Screencast

2026-03-31 6 30 00 PM

For reference, here is how GBoard behaves on the demo:

Screencast

2026-03-31 6 35 12 PM

@rustbasic
Copy link
Copy Markdown
Contributor

rustbasic commented Mar 31, 2026

Are you asking for the keyboard settings screen first? Here it is.

삼성키보드 = SAMSUNG Keyboard

image blured.

@rustbasic
Copy link
Copy Markdown
Contributor

@umajho

rustbasic-2026-03-31-21-16-53.mp4

@rustbasic
Copy link
Copy Markdown
Contributor

@umajho

With the changes mentioned in the comment, I get the best result when I modify ImeEvent::Commit in builder.rs as follows.

However, there is still an issue: when becomes 않ㅇ, the previous Hangul character is deleted, and the entire text shifts left by one position.

                    ImeEvent::Commit(commit_text) => {
                        if commit_text == "\n" || commit_text == "\r" {
                            None
                        } else {
                            let mut ccursor = cursor_range.primary;
                            state.ime_enabled = false;

                            if !commit_text.is_empty()
                                && cursor_range.secondary.index
                                    == state.ime_cursor_range.secondary.index
                            {
                                ccursor = clear_preedit_text(text, &cursor_range);
                                text.insert_text_at(&mut ccursor, commit_text, char_limit);
                            }

                            Some(CCursorRange::one(ccursor))
                        }
                    }
rustbasic-2026-03-31-21-48-00.mp4

@umajho
Copy link
Copy Markdown
Contributor Author

umajho commented Mar 31, 2026

#8045 (comment)
Are you asking for the keyboard settings screen first? Here it is.

Yes, thanks for the information.
It turns out that it is only available on Samsung phones, which totally makes sense. So this is indeed inaccessible to me.
(There is an unrelated “Samsung Smart Keyboard” App on the Play Store, which is for physical keyboards.)

#8045 (comment)

Sorry, my instructions were not clear. You need to go to the “preview” tab.
That said, I found that stackblitz runs fast on Android Firefox (my default browser), but it is somehow slow on Android Chrome. So, I'm planning to rebuild this experimental page on platforms like jsfiddle using plain HTML+JS that don't require build steps, probably tomorrow.

#8045 (comment)
With the changes mentioned in the comment, I get the best result when I modify ImeEvent::Commit in builder.rs as follows.

Thanks for the suggested changes. I will check them tomorrow.

@rustbasic
Copy link
Copy Markdown
Contributor

@umajho

rustbasic-2026-03-31-23-32-37.mp4

@umajho
Copy link
Copy Markdown
Contributor Author

umajho commented Apr 1, 2026

@rustbasic I have made some changes. does this improve things?

The stackblitz page has also been updated to reflect the changes.

@rustbasic
Copy link
Copy Markdown
Contributor

@umajho

It appears to be fixable, since the issue does not occur when "않았다" is entered consecutively without any problems.
The following are the responses when it is not entered consecutively.

  1. After entering "않", wait briefly for a commit to occur, and then enter the next character.
rustbasic-2026-04-01-18-46-25.mp4
  1. After entering "안", wait briefly for a commit to occur, and then enter the next character.
rustbasic-2026-04-01-18-49-07.mp4

@rustbasic
Copy link
Copy Markdown
Contributor

@umajho

It appears to be fixable, since it works correctly when "않았다" is entered continuously.

However, if you enter "않", wait briefly for a commit to occur, and then enter the next character, "않" gets duplicated.
For example, if you enter "않", wait, and then type "ㅇ", it becomes "않않ㅇ".

Result 1-1) "않않았다"

rustbasic-2026-04-01-19-01-15.mp4

If you enter "안", wait briefly for a commit to occur, and then enter the next character, "안" gets duplicated.
In this case, the final output varies as follows.

Result 2-1) "안않았다"
Result 2-2) "안않않았다"
Result 2-3) "안ㅎㅎ았다"

rustbasic-2026-04-01-19-11-08.mp4

@umajho
Copy link
Copy Markdown
Contributor Author

umajho commented Apr 1, 2026

Is this auto-committing behavior expected?

I will try to figure out how to fix the problem tomorrow.

@umajho
Copy link
Copy Markdown
Contributor Author

umajho commented Apr 5, 2026

Sorry for the delayed response.
I've made a few more changes. Does this improve the situation?

Update:
I realized that the changes I was talking about here are simply reverting my earlier “further simplification” back to the version of the code suggested by rustbasic here.

# Conflicts:
#	crates/eframe/src/web/text_agent.rs
#	crates/egui/src/widgets/text_edit/builder.rs
@rustbasic
Copy link
Copy Markdown
Contributor

rustbasic commented Apr 6, 2026

@umajho

I’ve briefly tested #8045 on Windows 10, WASM, and Android (using the Cheonjiin keyboard), and it works well.
If #8045 is merged, it seems you will have made a significant contribution to egui’s IME.
I will let you know if I encounter any issues.
Thank you for your hard work.

@umajho
Copy link
Copy Markdown
Contributor Author

umajho commented Apr 6, 2026

@rustbasic Glad to hear that!

Could you also try #8068? It also looks promising for addressing the Cheonjiin issue you encountered.
The implementations of these two PRs conflict, so if #8068 works well for you, I'd prefer to close this PR in favor of that one.

@cjgriscom
Copy link
Copy Markdown

There seems to be a new issue, perhaps introduced in #7983

Clicking/dragging to select in the TextEdit now repeatedly causes a refocus. This change is one way to fix it:
9473de9

Screen_Recording_20260406_111123_Firefox.mp4

@umajho
Copy link
Copy Markdown
Contributor Author

umajho commented Apr 6, 2026

There seems to be a new issue, perhaps introduced in #7983

Clicking/dragging to select in the TextEdit now repeatedly causes a refocus. This change is one way to fix it: 9473de9

Screen_Recording_20260406_111123_Firefox.mp4

Oops. This is actually the result of an intentional action.
I made every focus request interrupt IME composition, even if the widget is already focused, to address this bug:
2026-04-06 11 35 27 PM
(Wayland + iBus. The keystrokes are recorded incorrectly due to differences in keyboard layouts between the host OS and the guest OS..)

However, I didn't expect that this would cause problems with virtual keyboards.

A better approach might be to introduce something like OutputCommand::InterruptIme, and let integrations decide how to handle it?

For example, on web, we could reuse this trick to reset the IME without causing flickering:

input.blur().ok();
input.focus().ok();

(It is late here, so my next reply may take a few hours.)

@KonaeAkira
Copy link
Copy Markdown
Contributor

@umajho I am against using that trick because it is the root cause of the 2 issues described in #8046

@cjgriscom
Copy link
Copy Markdown

cjgriscom commented Apr 6, 2026

It's likely that either set_value or set_selection_range on the <input> will reset IME without defocusing.

@umajho
Copy link
Copy Markdown
Contributor Author

umajho commented Apr 7, 2026

@cjgriscom The flickering issue should be fixed by #8078.

emilk pushed a commit that referenced this pull request Apr 8, 2026
… keyboard flickering on web (#8078)

* Closes N/A
* Partially replaces #7983
* Related: #8045
* [x] I have followed the instructions in the PR template

## Details

In #7983, I modified `Memory::request_focus` to interrupt any ongoing
IME composition. This fixed a bug where clicking inside an already
focused `TextEdit` failed to cancel the active composition, resulting in
duplicated text:

#8045 (comment)

To avoid introducing API changes in that PR, I ensured the IME state was
reset by forcing `PlatformOutput::ime` to `None` for at least one frame.
While this works well on desktop platforms, it causes virtual keyboard
flickering on the web:

#8045 (comment)

In this PR, I delegate the responsibility for handling IME composition
interruptions to integrations, allowing each integration to decide how
to interrupt compositions in a flexible manner.

### The new field `should_interrupt_composition` on `IMEOutput`.

Instead of introducing a new `OutputCommand` variant, this PR adds a new
field `should_interrupt_composition` to `IMEOutput`.

Interrupting an active composition is only meaningful when IME remains
allowed. If IME should be disabled altogether, `PlatformOutput::ime` can
simply be set to `None`.
Given this, IMO, it is more appropriate to attach the interrupt signal
to `IMEOutput` (i.e., the type of `PlatformOutput::ime`).
@umajho umajho mentioned this pull request Apr 8, 2026
3 tasks
# Conflicts:
#	crates/eframe/src/web/text_agent.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants