Sorry to say, no change on macOS... 😞 And it does regress on \ (which is option-shift-7 on German layout on macOS).

Hm, it's strange that Alacritty works so differently. I don't see anything they are doing differently for the base input, just calling text_with_all_modifiers. alacritty/alacritty#6958

I will check if there are some new commits to Winit master that we are missing next.

@clason Also could you provide the logs? It should show the key names we get out from winit.

If you tell me where to look :)

The logfile should be written if you start Neovide with neovide -log https://neovide.dev/command-line-reference.html#log-file. Note that it can contain some sensitive information so don't open any documents with secrets in them.

BTW:
I tested some more using the Finnish multilingual keyboard layout on Wayland https://jkorpela.fi/sfs5966.html, and even the forth level keys alt-gr+shift+key works properly on that. For example alt-gr+shift+4 types «

Edit: I found a bug < does not work. I thought I fixed it at some point, but I must have broken it again.

Yeah, as I said, this is very platform specific. Cross-platform is hard.

Here's the log; I hope you can find the needle in the haystack...
neovide_rCURRENT.log

It should not be hard, winit should take care of all the platform differences.

But for \, I can see that it send M-\. so it sends both the option and the key. alt-gr is totally filtered out on my system. And I see now why it does not happen in Alacritty, they don't send the modifiers to the terminal text. I'm not sure how to deal with it yet, but I'm leaning towards it being a winit bug, since the modifiers should be consumed.

For the dead key, I assume you pressed dead key followed by e. I can only see the e part in the log. So I have no idea what happens there yet.

But for \, I can see that it send M-\.

Which is wrong; it should just be \. (That was the point of the with_all_modifiers dance...)

For the dead key, I assume you pressed dead key followed by e.

That is correct.

I see that we have already discussed the problem with \ with the winit team here rust-windowing/winit#1890 and unfortunately there's no conclusion. They did not do anything about our request.

For the dead keys, I stil don't know what's wrong if it works in Alacritty.

On my system, even crazy mappings like these works map <C-«> :lua print("hello")<cr> and map <C-A-«> :lua print("world")<cr>. Neovide can distinguish between ctrl+shift+altgr+4 and ctrl+shift+alt+altgr+4

@clason, I don't know enough about how Mac keyboards work to know if it's a viable hack or not. But could we always remove option if it's combined with shift, and just send the resulting character? We would still send ctrl and/or command if they are pressed together with the resulting character.

Edit: After reading up, that would not work. Option can also be used alone to produce characters, at least on some layouts.

For shift by itself we always remove it and send the shifted character on all platforms.

I think we will have to do some modifications on windows as well, but I'm not sure, because on that platform ctrl+shift is the same as altgr, and I don't know what winit reports.

@ArturKovacs, @kchibisov if this message reaches you, do you have any suggestions or ideas? And also if you know why the dead keys aren't working, it would be nice. Our code is very simple at the moment as you can see, there shouldn't be anything outside the keyboard_manager.rs affecting it.

I don't know enough about how Mac keyboards work to know if it's a viable hack or not. But could we always remove option if it's combined with shift, and just send the resulting character? We would still send ctrl and/or command if they are pressed together with the resulting character.

Shift does affect the characters, for bindings with Alt one could use the key_without_modifiers, however if you want a text on macOS without just an Alt, that's lacking right now in winit, but I'd need to bring it back because it's required in alacritty (see old OptionAsAlt on macOS).

if this message reaches you, do you have any suggestions or ideas? And also if you know why the dead keys aren't working, it would be nice. Our code is very simple at the moment as you can see, there shouldn't be anything outside the keyboard_manager.rs affecting it.

Could you describe what exactly the issue with the dead keys is? The macOS one? macOS doesn't have AltGr at all, it's all just alt. That's one of the reason I've added lalt/ralt options to modifiers, so users can have a special key getting logic based on modifiers pressed, simply because macOS has weird keyboard input.

Could you describe what exactly the issue with the dead keys is?

They don't work; composing dead keys like ´ or ^ just get eaten (they're dead dead; the pre-compose state isn't triggered).

@clason on macOS, right?

Yes, macOS. (Sorry, I thought that was clear from the context.)

I see now. I did not read properly what our alt_is_meta option was supposed to do. It defaults to false, so option/meta is never sent to Neovim.

Implementing that should be easy and should fix the problem with \.

But I think the suggestion by @kchibisov, would be better to allow configuring the left and right alt differently, so that it works in a similar way to a PC keyboard with alt-gr on the right and alt on the left.

I believe it is a mistake trying to shoehorn option into alt/altgr. (This is a distinction that does not exist on macOS, and macOS doesn't distinguish between left and right option. The expectation is that either option key works as you would expect from other applications.)

I can confirm \ and friends work again now.

I made some improvements to the implementation you should be able to map option+F1 for example. So combinations with special keys are allowed, but not with keys that produce characters. @kchibisov (does winit have a way to get a list of those?), right now we hard-code our own incomplete list)

The reason why the situation is more tricky here and we need that option, compared to other applications is that we don't know what the actual shortcuts are. Normal applications can just try the shortcuts first and then type the produced character(s), just like Alacritty does. This could be fixed by changing the API of Neovim, so that we always send the key and all modifiers and Neovim figures out what to do based on the mappings. So that's probably the way to go in the future actually. Once we have figured out if that's really the case we should open an issue and perhaps even a PR in Neovim with that feature.

I still have no idea why the dead keys don't work on MacOS, so I need help from someone with the actual hardware to debug it, perhaps comparing with Alacritty.

One possibility is that they are actually handled through the IME system, which I haven't implemented yet, but I doubt that.

This could be fixed by changing the API of Neovim, so that we always send the key and all modifiers and Neovim figures out what to do based on the mappings. So that's probably the way to go in the future actually.

No, it is not. é is a literal character that is expected as the input; producing that from actual key events is the UI's (or terminal's) job.

I'm happy to keep testing things if someone tells me what to do exactly, but I don't use Alacritty and can't dive/debug into their sources by myself.

Yes, resolving the dead keys, but not for resolving extra modifiers.

The only ways to resolve \ on your system, is to either have a flag that tells that the option key always produces characters (alt_is_meta, OptionAsAlt), which you said is a mistake.

So the remaining option is to first check if there's a shortcut mapped for S-M-\ and if not process the actual characters. That's something we can't do alone, we need Neovim to do that check.

This is unless the macOS API provides the information for us, but in this message rust-windowing/winit#1890 (comment) @ArturKovacs, tells that he believes it's impossible.

Edit:
As I said, normally it's not a problem since the shortcut processing and keyboard input happens in the same application, but here Neovide+Neovim is a split application.

My point is that you should consider it the other way: some combinations produce a character (which unfortunately is platform and keyboard layout dependent); the rest may -- or may not -- be a shortcut and is forwarded to the application to handle. (I expect that terminals that do not leverage a system IME work the same way; Neovide here -- quite literally -- plays the role of a terminal for Neovim.)

is to either have a flag that tells that the option key always produces characters (alt_is_meta, OptionAsAlt), which you said is a mistake.

Then I was not clear in my remark: I believe this flag is (remains) needed; I just didn't think distinguishing left and right option for that purpose is the way to go.

I believe it is a mistake trying to shoehorn option into alt/altgr. (This is a distinction that does not exist on macOS, and macOS doesn't distinguish between left and right option. The expectation is that either option key works as you would expect from other applications.)

It actually does as part of its modifiers event, that's why winit has it for macOS, it'll all encoded, and even Terminal.app uses it. Iterm has such options for years, and lots of other terminals and applications, macOS has just weird handling, that's what you should accept.

I think for bindings why might want to do what alacritty does, where we take case insansative char from the config and modifiers state, Like <C-S-a> for ctrl+shift+a, not ctrl+shift+A. The same will work for alt. I know that winit will have to provide text_without_alt on macOS, so apps like alacritty could work as before.

Although I don't use macOS myself, I also agree that distinguishing between the left and right option/alt is useful. It's an easy key to hit with the thumb, so you don't really need to use the opposite hand in order to activate it, at least if you have been a life long PC keyboard user. And in Neovim I have plenty of alt mappings, while still using the right hand alt-gr to type special characters. If I was forced to switch to Mac, I would hate to lose that.

I think for bindings why might want to do what alacritty does, where we take case insansative char from the config and modifiers state, Like for ctrl+shift+a, not ctrl+shift+A. The same will work for alt. I know that winit will have to provide text_without_alt on macOS, so apps like alacritty could work as before

Yes, but Neovim will then need to know the keymap in order to to resolve M-S-7 to \ on the macOS German keyboard layout. So I think the only option for us, is to send both variations and let Neovim chose the right one.

Of course if we rely on the OptionAsAlt then that is not a problem, when it's disabled then it would be M-S-7 If OptonAsAlt is enabled on that side we just send \

I still don't know why the dead keys don't work on Mac. On Wayland in addition to typing characters using dead keys normally, even shifted mappings with dead keys work properly. So I can can map something like map <C-É> :lua print("Hello")<cr> to trigger when I press ´ ctrl+shift+e. Since they work in Alacritty there must be something I'm missing here in my implementation.

I still don't know why the dead keys don't work on Mac. On Wayland in addition to typing characters using dead keys normally, even shifted mappings with dead keys work properly. So I can can map something like map <C-É> :lua print("Hello") to trigger when I press ´ ctrl+shift+e. Since they work in Alacritty there must be something I'm missing here in my implementation.

I think the issue is that they are routed via the IME stuff on macOS in winit given that it doesn't distinguish them, really. I've seen some heuristics in web browsers on how to make it work, so maybe I'll just port one of them, because apparently our code looks pretty much what firefox does for example on macOS, except the dead keys handling.

@kchibisov. Thanks, so hopefully it starts working once I get the IME implementation done. It should not be hard comparing to your branch of Alacritty and doing the same here. I just need to figure out how to do some quick local testing of IME first so that I don't write something completely stupid.

Edit:
Yes, it's even in the documentation

/// - **macOS:** IME must be enabled to receive text-input where dead-key sequences are combined.

Edit: I figured it out, see my message below #1899 (comment)
@kchibisov, shouldn't set_ime_allowed be enough to trigger the IME mode?

I'm testing with fcitx5 and mozk on Wayland KDE Plasma and it triggers in all other applications even Wezterm. But not in this branch of Neovide and not in your branch of Alacritty either. I also tested the release Alacritty version and no IME popup is shown there either. Is it supposed to be working?

@clason. I enabled IME now, that should be enough for the macOS dead keys, but as explained in my earlier message something is broken at least on Wayland.

I rebased to fix the conflicts.

I've tested 340571b on MacOS, and everything I use works as before, with the addition that I can insert characters with dead keys, which is awesome! Also tested a couple of MacOS-specific issues:

#1237 (comment) (this PR doesn't change anything).
#1866 (comment) (currently, alt- key bindings work only with dead keys; the PR changes this, so they only work with non-dead keys; this might break some people's keymaps).

Thank you all!

mgax, the next release will be breaking then.