Secondary display improvements#268
Secondary display improvements#268ClassicOldSong merged 39 commits intoClassicOldSong:moonlight-noirfrom
Conversation
|
Thanks for the PR! The refresh rate handling for external display is what I wanted to do but you just got it working, thank you! I'm really busy recently, I'll review it later. |
|
I just realized I have to fix a little condition. I'll fix it later today, so you got some time :-) |
|
Done |
|
I had to make more changes. Due to the streamView being just attached to the Presentation on the SecondScreen, it messed up the InputCatching. It was not possible to prevent systemUi Bars etc from appearing on the edge of the screen with mouse control, as the focused view was initliazed on the mainscreen and then passed to the second screen. Though mouse support and all works now in regular mode, I have added a new fully immersive mode which basically launches the whole GameActivity onto the secondscreen, making everything work perfectly. Gaming, RefreshRate, Resolution, Mouse, Keyboard, Gamepad - perfect Drawback: Phone cant be used as touchpad (can be added later via feature again, but not sure how yet) Users have the option what to pick with the new ConfigSetting Phone can be used while using secondary session, but in order to get the input again onto the Secondary screen, "resume stream" needs to be selected. |
Maybe present the input capturing views back to the main screen? IDK if that's possible but I think it worth a try. |
|
@ClassicOldSong I have added an TouchPad + FullDesktop Keyboard OverlayView that starts with the FullSecondScreen on the phone and sends input events over the connection directly. Works good! This overlay can be closed via little X button, which brings back the input service to the secondscreen (making physical mouse/keyboard work again) If the Overlay is used, physical is not working as lost focus, but thats not an issue.. just close the overlay done. As the SecondScreen connection is now standalone from the app and running in the background, I added a sticky service to make users aware and get focus again by opening the touchpad via the service. And much more... Basically did it mostly from scratch to not mess again with GameActivity. Review is needed and I need more time to fine tune...so far only leftclick,rightclick and scroll is supported by touchpad. |
| conn = Game.instance.conn; | ||
| } | ||
|
|
||
| touchpadView.setOnTouchListener((v, event) -> { |
There was a problem hiding this comment.
Why not directly use the TrackpadContext? It's much improved over the original RelativeTouchContext and we can then manage future trackpad changes in one place.
There was a problem hiding this comment.
The original touch fingers handling logic is very twisted but you don't need to understand, just copy it from there and it'll work...
There was a problem hiding this comment.
I'll try thanks! Just figuring it out on the go so far. Not familiar with the whole app yet.
I'll test!
There was a problem hiding this comment.
I mean copy the touch handling logic from Game.java, not from the TrackpadContext itself, to be clear...
There was a problem hiding this comment.
@ClassicOldSong hm, wasnt I not already using it same as in Game?
There was a problem hiding this comment.
DoubleTap drag (select via mouse) etc is not workin yet.
There was a problem hiding this comment.
In Game.java, it creates several handlers for each finger(xxxxContext) and then calls methods on them. It doesn't handle input emulation directly.
I wrote the TrackpadContext, I can't get it wrong... It's really twisted and counter intuitive.
There was a problem hiding this comment.
Eh I didn't check carefully, you're right, sorry for the confusion...
I'm really busy recently so I wasn't paying attention...
There was a problem hiding this comment.
I also have to add the game menu on backpress to the trackpad or sth similar
There was a problem hiding this comment.
Eh I didn't check carefully, you're right, sorry for the confusion...
I'm really busy recently so I wasn't paying attention...
All good. Im also only using my absolute limited spare time. I use it privately already, but in general it's far from ready to be shipped and no time pressure.
Feel free to take your time, make improvements or tell me if sth is bad.
| scaledY = Math.max(0, Math.min(scaledY, targetHeight - 1)); | ||
|
|
||
| // Debug log | ||
| Log.d("MouseMapper", String.format( |
There was a problem hiding this comment.
Better use the Limelog helper for logs
|
|
||
| gameMenuCallbacks = new GameMenu(this, conn); | ||
| gameMenuCallbacks = new GameMenu(this, conn, this); |
There was a problem hiding this comment.
Why passing this two times?
There was a problem hiding this comment.
I need the activity context of the calling activity, but still need the game instance. Here though on the game itself, it's just the same.
I'll clean up the PR soon with improvements to such quick dirty things.
|
Please merge the latest |
I'm almost done with what I wanted to achieve and I'm testing by using it for the last 3 days. I also got a tester from reddit using it with his monitor and glasses. |
|
I am said tester, and it works great so far :) |
Update main branch artemistics
…yger/artemistics into PR268 # Conflicts: # app/src/main/java/com/limelight/Game.java # app/src/main/java/com/limelight/GameMenu.java
|
Lucky day! I got may glasses early. Several questions to @Janyger :
|
|
One more thing to add: Although you can connect a > 60hz display to Android, but the Android system might always use 60hz no matter what the display mode actually is. That results in the phone outputs 60hz to a non-60hz display causing stutters. Do you have any idea to solve this issue? I tried earlier to set the refresh rate on the external display but it doesn't help. The mode switch "successfully" but Android is still rendering at 60hz. We need a way to detect this behavior. |
Yes, that's the main issue. It's not possible to have any influence on what the System and external Monitor negotiatate. My glasses have 120hz as their default in their EDID, so it's connected with 120 and we can successfully configure 120hz (doesn't matter if the phone screen is 60hz), but if the connection is habdshaked/negotiated, there is nothing we can do. That's why I display with a toast message to the users the actual connected/active refresh rate and resolution before the stream starts. We could limit the hz selection to ONLY the active one and Max resolution the same |
|
The actual problem is, even if it establishes like 72hz or 90hz, Android still renders at 60hz but apps detects the display at the established refresh rate. The XReal glasses defaults to 90hz, on my OnePlus 13 it's rendering at 60 but the toast says 90, but on my Y700 gen 4 you can manually choose the refresh rate in system settings. Only few devices allow configuring refresh rate for the external display unfortunately. |
I can't confirm this behavior. We don't mirror the screen anymore, we show it on the external display, the whole activity is only launched on it, with the set refresh rate directly, so it should be 120hz for example, even though android phone screen uses 60hz. UFO test etc at least work fine. How do you see it not working? |
|
Samsung devices handle external display better because it has DeX mode, but still not perfect. You need to get an average phone to test this behavior. |
Hm, I have different devices. I'll test. However, Samsung is actually the worst... limiting dex also to 60hz and removed support for 120hz 2 years ago. But how do you know it doesn't render the wanted higher frame rate? If I pick 120hz in settings, glasses are active mode 120hz, I'm pretty sure it's actually 120hz. |
Dex Mode needs to be exited for it to behave somewhat normally |
There was no easy toggle one or the other off option. If it was by me, the old can be removed. But both should not be active at the same time.
That sounds good, basically take whatever the connected devices delivers.
Yes, for now I would hide it... The onscreen controller I started but it's annoying and probably not really used. Could be an enhancement later on
Only modes that are relevant can be selected. Trackpad normal and gaming. Or at least any other selected will be overriden. Disabling it I don't see the need, as people might to switch modes all the time. Don't you think that's useful?
? External screen is not rotating on phones rotation. Did you mix up the modes?
I agree with the icon...maybe hitting the touchpad once could already do this action, so we don't need it.
No objections
Wouldn't call it an issue...just that it takes a theme now rather than default which was not available on secondary screen. We might make it look even better? :-) I'm quite busy currently...feel free to also do changes as you like |
|
@ClassicOldSong thanks for taking care of the cleanup, even improving it and making use of my efforts. Highly appreciated. |
|
@Janyger I'm not sure if this is the place to ask this, but I've been messing around with the seperate external display feature implemented in this branch, and is it correct that a local mouse cursor does not work? As in, the mouse cursor is displayed on the phone screen, not the external monitor. If so, is there a way to get the local cursor to function on the external monitor instead of the phone screen? This would be benefitial due to latency reasons whenever a connection is spotty for whatever reason. Outside this minor thing, this feature looks really good! Much better than dex with the annoying task and navigation bars constantly popping up. |
The Corsor should only be on the phone screen when Focus is on the phone, as soon as you get back into the stream, it is and should be in the external one as a normal windows mouse. Can you try clicking the upper left corner icon (focus) on the black touchpad phone screen? That should move the physical mouse Focus into the external screen. |
|
The cursor works fine on the external screen when I disable the local Android cursor, but in that case the cursor displayed is from the host PC, not the client (the phone). This means that when streaming latency increases, the cursor response also slows down. |
Ahh.... Good question... Possible I would say yes, not sure if directly the local cursor but at least a response instant cursor. I'll check when I got time |
|
@Janyger Just out of curiosity, did you figure anything out? If not, is there anything I can do to help? If you haven't had time yet, no worries! |
This PR fixes:
Proper Mouse Tracking for External Displays:
Physical mouse movements are now correctly tracked and displayed using ACTION_HOVER events when a mouse is connected.
Dex like experience with external displays now possible.
Supports Aspect Ratios different from Main Screen on Secondary Displays:
External displays with different aspect ratios (e.g., 3840×1080 for ultra-wide or 3D modes) are now fully supported. This allows proper rendering without being forced to match the main screen's aspect ratio.
*Best for watching 3D content on AR glasses :-)
Samsung (and other devices?) Refresh Rate Quirk Fix:
On Samsung devices, connecting an external display often drops the system refresh rate to 60Hz. Previously, this incorrectly affected Moonlight’s rendering frame rate.
✅ Now, the external display’s actual refresh rate is used when determining the rendering rate.
✅If users pick a lower refreshRate, that one is picked for the stream BUT the actual refreshRate of the external display stays, as we have no way of changing this. So best pick this one :)
Added now TouchPad control's and a real full desktop mode.
Game menu showing up on phone and not on the external display etc.
Phone can be used while stream is running on second monitor
A toast message is shown when streaming starts, displaying the actual resolution and refresh rate of the connected external display