[InputText] decouple text input state from VirtualKeyboard visibility#14901
Conversation
Previously, on SDL targets (Desktop, Ubuntu Touch), the system text input loop was gated by the visibility of the On-Screen Virtual Keyboard. Hiding the VK would call Device:stopTextInput, effectively disabling the physical keyboard. This commit binds the text input state to the lifecycle of the InputText widget instead: * InputText: explicit calls to Device:startTextInput and Device:stopTextInput are now made during focus and unfocus. This ensures the physical keyboard works whenever an input field is active, regardless of UI state. * VirtualKeyboard: removed legacy input signaling from onShow and onCloseWidget, as this is now handled by the input field itself. * Navigation: Improved Back key behavior when the Virtual Keyboard is disabled. Pressing Back now unfocuses the input field and sends a synthetic Down event to pass focus to the dialog buttons, streamlining the navigation flow.
| elseif key["Down"] and G_reader_settings:nilOrTrue("virtual_keyboard_enabled") then | ||
| self:downLine() | ||
| elseif key["Up"] then | ||
| if Device:isSDL() or G_reader_settings:nilOrTrue("virtual_keyboard_enabled") then |
There was a problem hiding this comment.
Like I said, none of these seem like they should say if SDL. Please explain the intent so we can figure out what it should be.
There was a problem hiding this comment.
We are giving control to input text, when the field is in focus, up/down should move you up/down the text, so that moves you easily when reading something with lots of lines, like a patch.
on kindle I didn’t want that, because you’re less likely to do those things, but also because FocusManager can still handle those up/down events, on SDL (for whatever reason, likely the startText thingy) that is not the case.
There was a problem hiding this comment.
So this is in fact a desired Kindle behavior change masquerading as an SDL one?
There was a problem hiding this comment.
No, the kindle stuff is already locked in from when I did the hide the keyboard PR.
this is now to get the emulator working without the vk, you have to see this as a whole, in that long comment @NiLuJe suggests moving those no-ops to ImputDialog, I realised we can avoid many of the issues he describes by locking the it them to InputText instead, hence my response in that other review comment. but we don't have Focus Manager within input text so this needs to add back the cursor functionality.
I strongly suggest to try it, I believe when you see the system at play it will make a bit more sense
in short, when an input dialog pops up, input text takes the stage. We only leave it after an esc press in which case, focus jumps to the next item on input dialog's array...
There was a problem hiding this comment.
What I am trying to say is that none of this is currently SDL-exclusive functionality, and trying to make it so smells, as they say. Most concretely and immediately Android already uses this exact same functionality as we speak,[1] and it Just Works™ because it was purposefully written to Just Work™. Using isSDL anywhere is unlikely to make life easier on our future selves.
diff --git a/frontend/device/android/device.lua b/frontend/device/android/device.lua
index 42142f256..652c8634e 100644
--- a/frontend/device/android/device.lua
+++ b/frontend/device/android/device.lua
@@ -100,6 +100,17 @@ local Device = Generic:extend{
if not link or type(link) ~= "string" then return end
return android.openLink(link)
end,
+ -- Text input control (soft keyboard / IME)
+ startTextInput = function()
+ if A then
+ android.startTextInput()
+ end
+ end,
+ stopTextInput = function()
+ if A then
+ android.stopTextInput()
+ end
+ end,
canImportFiles = function() return android.app.activity.sdkVersion >= 19 end,
hasExternalSD = function() return android.getExternalSdPath() end,
importFile = function(path) android.importFile(path) end,
@@ -241,6 +252,12 @@ function Device:init()
setClipboardText = function(text)
return android.setClipboardText(text)
end,
+ handleSdlEv = function(device_input, ev) -- c/p from SDL for poc
+ local SDL_TEXTINPUT = 771
+ if ev.code == SDL_TEXTINPUT then
+ UIManager:sendEvent(Event:new("TextInput", tostring(ev.value)))
+ end
+ end,
}
-- disable translation for specific models, where media keys follow gravity, see https://github.com/koreader/koreader/issues/12423[1] I believe I already shared the proof of concept diffs previously.
There was a problem hiding this comment.
Right, you were talking about the thing I was asking about (non-touch Kindle) and I thought you were talking about something else. Whoops.
There was a problem hiding this comment.
for back I mean, up/down definitely improve the game massively, even on kindle.
but back should consider vk_enabled.
There was a problem hiding this comment.
So on touch the behavior seems to be broken when the VK shows, unless it's intentional that you can't control it there. That's the background behind what I said above: the arrow keys don't do anything anyway, so it's better if they actually control the focus.
There was a problem hiding this comment.
is not clear to me what is or isn't intentional currently, but I agree, is not good. I think it should be safe to remove the caps, even for kindles
There was a problem hiding this comment.
Yes, the behavior doesn't seem to break on Kindle. I'll push a commit removing them.
| -- NOTE: This effectively stops SDL text input when a keyboard is hidden (... but navigational stuff still works). | ||
| -- If you instead wanted it to be enabled as long as an input dialog is displayed, regardless of VK's state, | ||
| -- this could be moved to InputDialog's onShow/onCloseWidget handlers (but, it would allow input on unfocused fields). | ||
| -- NOTE: But something more complex, possibly based on an in-class ref count would have to be implemented in order to be able to deal |
There was a problem hiding this comment.
This comment has been resolved? It doesn't read like something that could be resolved so easily.
There was a problem hiding this comment.
Strangely, i believe it solves all that. I might be wrong which is why @NiLuJe should come out of hibernation and point out potential pitfalls.
There was a problem hiding this comment.
The comment still applies. The problem is you can't type anymore after hiding the vk. The musings about a possible solution are possibly no longer relevant, but I reintroduced the main comment in the InputText back handling.
There was a problem hiding this comment.
well we solved the problem for when vk is never preset, which is admittedly, what we want most of all ;)
|
I regret not naming this “consciously uncouple text input state from VirtualKeyboard visibility” ;) |
I don't understand what this means. What UI state is there, except for focus? |
FocusManager controls the buttons, so for the dictionary one for example, you have the buttons handled by FM and input text. what I am doing is simply locking keyboard inputs to input text only. so typing words when input text is not in focus should be ignored and/or swallowed by FM |
|
Right, but surely that is making it obey UI state rather than regardless of UI state. |
all I am saying is, focus is exclusive, only one thingy can accept keyboard events at once... which solves all the issues mentioned in the long comment, is brilliant. |
There was a problem hiding this comment.
The behavior strikes me as sufficiently handled like this:
| elseif key["Back"] and self.focused then | |
| self:unfocus() |
There was a problem hiding this comment.
Um, I'm not sure if the suggestion there is working correctly. Like this:
@@ -664,20 +660,8 @@ function InputText:onKeyPress(key)
self:addChars("\n")
elseif key["Tab"] then
self:addChars(" ")
- elseif key["Back"] and Device:isSDL() and G_reader_settings:isFalse("virtual_keyboard_enabled") then
- if self.focused then
- self:unfocus()
- UIManager:nextTick(function()
- local Key = require("device/key")
- local Event = require("ui/event")
- UIManager:sendEvent(Event:new("KeyPress", Key:new("Down", {})))
- end)
- end
- -- as stated before, we also don't need to unfocus when there is no keyboard, one less key press to exit widgets, yay!
- elseif key["Back"] and G_reader_settings:nilOrTrue("virtual_keyboard_enabled") then
- if self.focused then
- self:unfocus()
- end
+ elseif key["Back"] and self.focused then
+ self:unfocus()
else
handled = false
endThere was a problem hiding this comment.
That being said, the desired behavior might be to close the virtual keyboard first without losing focus instead.
Perhaps @NiLuJe's comment relates to something like that? In any case this simply keeps the behavior the same.
There was a problem hiding this comment.
nice changes to up/down I am once again embarrassed to admit, that that hadn't occur to me. Nonetheless this is not a good idea (according to me), the way I see it, either we do the move focus or it kills the whole widget (inputdialog) in one single press. this single unfocus() however is some limbo state best avoided, not to mention is not clear why one needs two back presses to exit the widget.
having said that, and seeing how up/down works much nicer now, in my opinion it is possibly best for the back key to kill the widget in one go.
There was a problem hiding this comment.
elseif key["Back"] and G_reader_settings:nilOrTrue("virtual_keyboard_enabled") then that exists to combat that inputdialog+vk vs only inputdialog, in one, back first kills the vk then a second press kills the widget, in the other it kills the widget directly.
There was a problem hiding this comment.
I am once again embarrassed to admit, that that hadn't occur to me.
Eh, it's easy to think of a thing to improve after someone else already laid the groundwork.
not to mention is not clear why one needs two back presses to exit the widget.
What I really meant was this:
elseif key["Back"] and self.focused then
self:unfocus()
UIManager:show(Notification:new{
text = _("Press back again to close."),
})Because it does strike me as somewhat brusque to do it like this (but we could go ahead with that and see what the feedback is):
elseif key["Back"] then
if self.parent then
UIManager:close(self.parent)
endThere was a problem hiding this comment.
single back press kills widget will seem a bit abrupt, but it will be better in the long run. People will get used to it. and that notification, "hellll no" that will annoy everyone very quickly
|
that start thing does need to go in the last remaining question now is: what's the default? vk off or vk on? |
Hm, they actually share the problem. They both fire regardless if they have focus or not. This remains improper. |
|
It'll do for now. For reference, the behavior I want would look something like this: diff --git a/frontend/ui/widget/focusmanager.lua b/frontend/ui/widget/focusmanager.lua
index 4f7d3dffb..2f0cf83a6 100644
--- a/frontend/ui/widget/focusmanager.lua
+++ b/frontend/ui/widget/focusmanager.lua
@@ -337,6 +341,10 @@ function FocusManager:moveFocusTo(x, y, focus_flags)
logger.dbg("FocusManager: Move focus position to:", x, ",", y)
self.selected.x = x
self.selected.y = y
+ if not self.focus_initialized_sent then
+ target_item:handleEvent(Event:new("FocusInitialized"))
+ self.focus_initialized_sent = true
+ end
-- widget create new layout on update, previous may be removed from new layout.
if bit.band(focus_flags, FocusManager.FORCED_FOCUS) == FocusManager.FORCED_FOCUS or Device:hasDPad() then
-- If FORCED_FOCUS was requested, we want *all* the events: mask out both NOT_ bits
diff --git a/frontend/ui/widget/inputtext.lua b/frontend/ui/widget/inputtext.lua
index c4d3907b4..fd22e66e6 100644
--- a/frontend/ui/widget/inputtext.lua
+++ b/frontend/ui/widget/inputtext.lua
@@ -71,6 +71,10 @@ local InputText = InputContainer:extend{
-- These may be (internally) overloaded as needed, depending on Device capabilities.
function InputText:initEventListener() end
+function InputText:onFocusInitialized()
+ print("InputText:onFocusInitialized()")
+ Device:startTextInput()
+end
function InputText:onFocus() end
function InputText:onUnfocus() end
|
| self:initKeyboard() | ||
| self:initEventListener() | ||
| end | ||
| if self.focused then |
There was a problem hiding this comment.
I don't think that helps, does it? The thing starts as true after all. :-) Best to just keep it simple with a todo comment.
There was a problem hiding this comment.
what if there is a multi input dialog, don't they start one in focus and the others unfocused?
There was a problem hiding this comment.
On the ones I checked it still fires twice as focused even if I set them all to false (three times if I leave the MultiInput as is), but you're right that it might be better than nothing. Please add a todo in any case.
There was a problem hiding this comment.
what do you want it to say exactly? ;)
There was a problem hiding this comment.
--- @todo Investigate why this fires too much. See <https://github.com/koreader/koreader/pull/14901#issuecomment-3837678877>.
There was a problem hiding this comment.
To clarify, personally I'd probably write it like TODO but ldoc integrates nicely with @todo(/fixme/etc.).
|
here's why vk won't work when touch is enabled koreader/frontend/ui/widget/virtualkeyboard.lua Lines 873 to 889 in 323e622 |
|
this is what I think it should be elseif key["Back"] then
if G_reader_settings:nilOrTrue("virtual_keyboard_enabled") and self.focused then
if self.parent and not self.keyboard:isVisible() then
UIManager:close(self.parent)
end
elseif self.parent then
UIManager:close(self.parent)
endlol, which is really just elseif key["Back"] then
if self.parent and not self.keyboard:isVisible() then
UIManager:close(self.parent)
endbut it would require this block to go entirely koreader/frontend/ui/widget/virtualkeyboard.lua Lines 942 to 952 in 323e622 and also doing away with the removing of all FocusManager events as seen here #14901 (comment) |
|
that would mean both touch and non-touch handle vk similarly dialog+vk = vk takes priority and is in focus, first back closes vk and gives back control to inputdialog but physical keyboard continues to work fine... second back closes inputdialog awaiting confirmation |
|
Sounds good to me, but I can't check the code until at least tonight. |
…rdless of vk state, #FreeTheKeyboard
| self:initKeyboard() | ||
| self:initEventListener() | ||
| end | ||
| --- @TODO: Investigate why this fires too much. See <https://github.com/koreader/koreader/pull/14901#issuecomment-3837678877>. |
There was a problem hiding this comment.
Still supposed to be lowercase. ;-) I guess the check I wrote doesn't take that into account.
|
blimey, @Frenzie merging this so fast it reminds me of that cartoon bird that exits the frame so fast it leaves half his feathers behind. Tell me you hate the virtual keyboard without telling me you hate it. ;) |
|
I already wanted to do the focus up/down thing years ago, but I forgot about it. |
|
Something that may be was forgotten when testing: InputDialog as used in Text editor (or Edit book style tweak). |
|
yes, that is a sacrifice for the greater good. We had to remove that trick you @poire-z and @johnbeard implemented a while a go of removing all events when a device had a touch screen. Now it works the same everywhere which should be much easier to maintain, but furthermore, even though it is a slight inconvenience, you can hide the vk so should not be a problem. As far as I am concerned, good riddance virtual keyboard. ;) |
| self:unfocus() | ||
| elseif key["Back"] then | ||
| if self.parent and not self.keyboard:isVisible() then | ||
| UIManager:close(self.parent) |
There was a problem hiding this comment.
a bit late now, but shouldn't this have be self.parent:onCloseDialog() instead?
asking the manager at the bar to kick out your dad seems a bit of an overreach, why not ask him directly ;)
There was a problem hiding this comment.
Well, you're the one who wanted to close it all immediately. ;-)
But if so, I figure it should first include a check like if self.parent.onCloseDialog….
There was a problem hiding this comment.
Well yes, but i didn't mean point a gun to his head and close it all ;) i am okay with being asked, "hey you haven't saved, sure you want to exit?"
I don't think you mentionned that sacrifice, if you knew about it ? I get that Up/Down, when reaching first line or last line, could change focus (it's a little inconvenience, but understandable). |
|
I'm fine reverting the VK changes, but different handling for multiline is probably pretty simple (no time to look the next few hours). |
|
It's not essential for the stable release, it would affect mostly us emulator developpers. |
|
You can also use the enter key on the virtual keyboard, is not unusable. I haven't try to hide anything btw, not sure why such allegations are being made when it is you @poire-z who has lately been covering your eyes in the hopes that the sun goes away ;) But i don’t consider this a major problem given that, as soon as everyone is aware vk can now be hidden for good, no one will have this problem, perhaps what could be done is to focus on the virtual enter key when in those full screen modes.. |
|
There are other little issues. Also, wheter VK enabled or disable, now, just hitting Esc quits the text editor, even if you have made changes. Before this PR, we always got that popup "You have unsaved changes". |
Once non-VK mode works well enough (which is possibly now already) it can be hidden by default on [Edit:]Actually, that might imply extending the setting to on/off/auto to account for dynamic keyboard plugging.[/edit]
We'll look into it. I suspect @Commodore64user and I already landed on the solution earlier. |
|
(Updated a few times my post above. I'm done playing, so hopefully no more live updates.) |
This sounds more like a failure in the previous PR where hiding the vk was introduced, there must be a function somewhere where we don't check for the vk_enabled setting so it simply pops it up. Something that perhaps went unnoticed.
Okay grandad, what exactly is "worse" not wasting half the scree on an unnecessary keyboard??
We are quite careful as to how the setting works, even if on a touch device with a bt keyboard, if you set it to hide and then disconnect the bt keyboard (i.e., you have no physical keyboard anymore) we will show you the vk, all i'm trying to say is, i don’t know how detection should work but the dumb method shouldn't fail
I believe asking the parent politely solves it |
Hey, I'm giving you feedback, I'm not being aggressive. Having some keys working, clicking in the text and getting the keyboard popup, and having nothing much working, and having to put it down each time - is more annoying than the natural editing without any annoyances (keyboard and clicks working) I had before with the VK shown. And on the few lines snippets of CSS, the VK isn't annoying at all. It's good that you make simple inputtexts and focus navigations better for NT, but the "sacrifice" (anticipated or not) of a good experience with text editor on the emulator is a big price to pay. (Haven't checked if the small InputDialog, ie. editing a highlight note, has issues.) PS: Grandad trusts you kids will make it all better. Kisses. |
Okay, so hide itself is already auto. That sounds good.
Yes, that's what I said, including that check in case the parent doesn't meet that assumption. ^_^
Like I said, I'm not attached to that change at all and it can presumably be delayed if we can't find a better solution within the next few days. I merely commented on the discrepancy because it caused me some momentary confusion, not as a silent hint that it should be fixed ASAP. ;-) |
|
koreader/frontend/ui/widget/inputtext.lua Lines 144 to 153 in 9eedae3 this seems to be the reason for the keyboard popping up regardless os state, the only worrisome thing is that comment though, as we DO need |
|
What do you mean by "this" concretely? Since it says |
|
Ahem, it doesn't, or at least not exactly. I had added some code there earlier for testing and forgot about it. function InputText:onShowKeyboard(ignore_first_hold_release)
if (Device:hasKeyboard() or Device:hasScreenKB()) and G_reader_settings:isFalse("virtual_keyboard_enabled") then
return true
end
if self.keyboard then
self.keyboard:showKeyboard(ignore_first_hold_release)
endEdit: diff --git a/frontend/ui/widget/inputtext.lua b/frontend/ui/widget/inputtext.lua
index dae8cbd3c..492b9a0d6 100644
--- a/frontend/ui/widget/inputtext.lua
+++ b/frontend/ui/widget/inputtext.lua
@@ -142,9 +142,7 @@ local function initTouchEvents()
if self.parent.onSwitchFocus then
self.parent:onSwitchFocus(self)
else
- if self.keyboard then
- self.keyboard:showKeyboard()
- end
+ self:onShowKeyboard()
-- Make sure we're flagged as in focus again.
-- NOTE: self:focus() does a full free/reinit cycle, which is completely unnecessary to begin with,
-- *and* resets cursor position, which is problematic when tapping on an already in-focus field (#12444).
@@ -781,6 +779,9 @@ dbg:guard(InputText, "onTextInput",
end)
function InputText:onShowKeyboard(ignore_first_hold_release)
+ if (Device:hasKeyboard() or Device:hasScreenKB()) and G_reader_settings:isFalse("virtual_keyboard_enabled") then
+ return true
+ end
if self.keyboard then
self.keyboard:showKeyboard(ignore_first_hold_release)
endEdit 2: but regardless of the specific solution, the condition should probably be something along those lines. |
|
this is my idea, during vk's init we do, if self.layout and #self.layout > 0 and Device:hasKeyboard()
and self.inputbox and self.inputbox.parent then
local last_row = #self.layout
local last_col = #self.layout[last_row]
self:moveFocusTo(last_col, last_row)
end |
|
Could you clarify what that does? |
|
Sorry, when the vk is visible, it sets focus to the enter key, that way if you press enter on the keyboard (at least assuming you don’t move it) you get an enter press. It’s a hack as much as deleting all the key events was.. |
diff --git a/frontend/ui/widget/inputtext.lua b/frontend/ui/widget/inputtext.lua
index dae8cbd3c8fd..408605ddf147 100644
--- a/frontend/ui/widget/inputtext.lua
+++ b/frontend/ui/widget/inputtext.lua
@@ -142,15 +142,27 @@ local function initTouchEvents()
if self.parent.onSwitchFocus then
self.parent:onSwitchFocus(self)
else
- if self.keyboard then
- self.keyboard:showKeyboard()
+ if (Device:hasKeyboard() or Device:hasScreenKB()) and G_reader_settings:isFalse("virtual_keyboard_enabled") then
+ do end -- luacheck: ignore 541
+ else
+ self:onShowKeyboard()
end
+ Device:startTextInput()
-- Make sure we're flagged as in focus again.
-- NOTE: self:focus() does a full free/reinit cycle, which is completely unnecessary to begin with,
-- *and* resets cursor position, which is problematic when tapping on an already in-focus field (#12444).
-- So, just flip our own focused flag, that's the only thing we need ;).
self.focused = true
end
+ -- We might have a visual focus if we used a DPad, so we need to remove it.
+ if Device:hasDPad() then
+ local x, y = self.parent:getFocusableWidgetXY(self)
+ if x and y then
+ -- Use FORCED_FOCUS to guarantee visual updates (Unfocus old, Focus new)
+ -- even on touch devices where this is usually suppressed.
+ self.parent:moveFocusTo(x, y, FocusManager.FORCED_FOCUS)
+ end
+ end
if self._frame_textwidget.dimen ~= nil -- zh keyboard with candidates shown here has _frame_textwidget.dimen = nil
and #self.charlist > 0 then -- do not move cursor within a hint
local textwidget_offset = self.margin + self.bordersize + self.padding
@@ -669,7 +681,7 @@ function InputText:onKeyPress(key)
self:addChars(" ")
elseif key["Back"] then
if self.parent and not self.keyboard:isVisible() then
- UIManager:close(self.parent)
+ self.parent:onCloseDialog()
end
else
handled = false
diff --git a/frontend/ui/widget/virtualkeyboard.lua b/frontend/ui/widget/virtualkeyboard.lua
index 5aac88c81b1e..9d7eb014c608 100644
--- a/frontend/ui/widget/virtualkeyboard.lua
+++ b/frontend/ui/widget/virtualkeyboard.lua
@@ -880,6 +880,15 @@ function VirtualKeyboard:init()
end
end
end
+ -- Set initial focus to the last key ("Enter"), to facilitate introducing new lines
+ -- on devices with a physical keyboard and the virtual keyboard on display.
+ if self.layout and #self.layout > 0 and Device:hasKeyboard()
+ and self.inputbox and self.inputbox.parent then
+ local last_row = #self.layout
+ local last_col = #self.layout[last_row]
+ self:moveFocusTo(last_col, last_row)
+ end
end
function VirtualKeyboard:_isTextKeyWithoutModifier(seq) |
…ader#14901) Previously, on SDL targets (Desktop, Ubuntu Touch), the system text input loop was gated by the visibility of the On-Screen Virtual Keyboard. Hiding the VK would call Device:stopTextInput, effectively disabling the physical keyboard. This commit binds the text input state to the lifecycle of the InputText widget instead: * Improve up/down behavior. Down at the end of an input moves focus down, up at the beginning moves focus up. * Undo removal of key_event for touch devices, allowing vk to work regardless of vk state, #FreeTheKeyboard
Previously, on SDL targets, the system text input loop was gated by the visibility of the On-Screen Virtual Keyboard. Hiding the VK would call Device:stopTextInput, effectively disabling the physical keyboard.
what's new
This PR binds the text input state to the lifecycle of the InputText widget instead:
InputText:onKeyPress(key), so navigation and closing behave consistently regardless of whether the virtual keyboard is enabled. Now, up/down keys delegate toFocusManagerwhen appropriate, and pressing back closes the parent dialog if the virtual keyboard is not visible.VirtualKeyboard:init()to check for both D-pad and physical keyboard before modifying key event handlers, improving support for devices with both input methods.VirtualKeyboard, streamlining widget lifecycle management.#FREETHEKEYBOARD
screen recording
expand to see
related issues
fixes FR: Ability to disable virtual keyboard #11374
Show/Hide Virtual keyboard and more keyboard shortcuts #12162
continues in [InputText] decouple text input state from VirtualKeyboard visibility pt. 2 #14937
This change is