Skip to content

Document that multiple windows are not supported on Android#3505

Open
madsmtm wants to merge 1 commit intomasterfrom
android-note-unsupported-multi-window
Open

Document that multiple windows are not supported on Android#3505
madsmtm wants to merge 1 commit intomasterfrom
android-note-unsupported-multi-window

Conversation

@madsmtm
Copy link
Copy Markdown
Member

@madsmtm madsmtm commented Feb 20, 2024

Not sure exactly what to write here, but I wanted the docs to talk about it somewhere.

Unless my understanding is wrong of course, and it is possible for Winit to support multiple windows / activities at some point?

CC @rib @MarijnS95

@madsmtm
Copy link
Copy Markdown
Member Author

madsmtm commented Feb 20, 2024

One solution I can think of is letting Window have an API for launching a new activity? So maybe the first Window created would be the main one, and then subsequent windows would have to use the launching mechanism?

@kchibisov
Copy link
Copy Markdown
Member

kchibisov commented Feb 20, 2024

I think we discussed that with @MarijnS95 at some point and I think there're issues with NDK stuff. In general window should allow create subview(nsview, subsurface, child window, etc) which is a separate object and allows restacking, etc.

EDIT by @madsmtm: CC #3506

@rib
Copy link
Copy Markdown
Contributor

rib commented Feb 21, 2024

One solution I can think of is letting Window have an API for launching a new activity? So maybe the first Window created would be the main one, and then subsequent windows would have to use the launching mechanism?

This would probably be quite challenging to support currently while our model for making it feel like you're running a regular, native Rust app with a "main" function on Android revolves around dedicated activity subclasses that help us track the lifecycle for the app.

Some work was done to avoid the use of static global state in android-activity so we could try and keep options open for multiple activities that could run in the same process but there would still likely be some thorny details try do this.

In an app I work on we do end up creating additional activities at time to open a web view but this is currently as an un-integrated special case via some Java code where there's no requirement for additional activities to also be driven by native Rust code + Winit.

I'd guess this direction could be more practical if we had our own RustActivity and maybe a separate RustSubActivity or something for managing something akin to multiple windows. I'm not sure this would currently be practical based on NativeActivity / GameActivity.

@rib
Copy link
Copy Markdown
Contributor

rib commented Feb 21, 2024

EDIT by @madsmtm: CC #3505

btw @madsmtm I guess you were trying to link some other issue here? this is pointing back to this PR?

@madsmtm
Copy link
Copy Markdown
Member Author

madsmtm commented Feb 21, 2024

EDIT by @madsmtm: CC #3505

btw @madsmtm I guess you were trying to link some other issue here? this is pointing back to this PR?

Oops, fixed.

@madsmtm
Copy link
Copy Markdown
Member Author

madsmtm commented Feb 21, 2024

we could try and keep options open for multiple activities that could run in the same process but there would still likely be some thorny details try do this.

I think I've been led to believe it was impossible to have two activities in a single process, but that doesn't sound like it is the case, at least not if you use some Java code? Can they be both be on-screen at the same time, e.g. in split screen mode, or one of them on a separate display? Do they have to run on separate threads? Do you register them as separate activities in the application manifest?

I'd guess this direction could be more practical if we had our own RustActivity and maybe a separate RustSubActivity or something for managing something akin to multiple windows. I'm not sure this would currently be practical based on NativeActivity / GameActivity.

Maybe you can expand on how sub-activities work, or just link to some documentation about that parent/child relationship?

@rib
Copy link
Copy Markdown
Contributor

rib commented Feb 21, 2024

we could try and keep options open for multiple activities that could run in the same process but there would still likely be some thorny details try do this.

I think I've been led to believe it was impossible to have two activities in a single process, but that doesn't sound like it is the case, at least not if you use some Java code? Can they be both be on-screen at the same time, e.g. in split screen mode, or one of them on a separate display? Do they have to run on separate threads? Do you register them as separate activities in the application manifest?

far from impossible, it's quite common for a regular Android application to involve more than one Activity which may be started via Intents and they would typically run in the same process.

multiple activities associated with an application would be declared in the AndroidManifest.xml and an Intent could be used to launch those activities.

but more specifically in the context of a typical 'native' application on Android; those are typically implemented via some special Activity sub classes (i.e NativeActivity) which in itself isn't really designed to be to instantiate multiple times (that would be more like trying to run multiple independent programs with main() functions only separated by threads in the same process. (This is one of the reasons why I wanted to remove the use of process::exit() in Winit in case some application would do exactly this, since we can't necessarily assume there aren't multiple Activities running in the same process)

so it's not a fundamental limit of Android that you can't have multiple activities that can stack on top of each, or e.g. be arranged side-by-side on larger screen but the NativeActivity class that's part of the Android SDK isn't geared for trying to do that.

Hypothetically we could implement our own RustActivity subclass of Activity that could somehow be tailored to allow a single native Rust program to follow the lifecycle of multiple activities within a single process.

Some details of that could be quite awkward if trying to think of an Activity like a Window. For instance the launchMode that affects how an activity may replace or stack on other activities doesn't seem to be accessible programmatically, it's something that's only declared in an application's xml manifest. And just the fact that you need to declare activities in your manifest might make them awkward to map to windows in Winit.

Probably in this context (looking for what multiple Windows in Winit could map to - or for something similar to subviews for #3506 ) then I think it's probably SurfaceViews, not Activities that would be more appropriate.

This is also where the built-in NativeActivity class isn't ideal since it has been designed mainly for fullscreen games and only deals with a single SurfaceView that gets advertised to native Rust code as a "Native Window" (a misnomer from the pov of what it corresponds to in Android SDK terms).

Again hypothetically though, with a RustActivity we could potentially try and break down that assumption and make it easier to create multiple SurfaceViews which could then each be cast as a NativeWindow for Rust code.

Even without a special RustActivity it might be possible that we could create a SurfaceView via JNI potentially and then use ANativeWindow_fromSurface to create a 'Native Window' for being able to render to that surface from native Rust code.

There are probably some thorny details that would be involved, but in principle I think perhaps that Winit could do all that via JNI and have good amount of flexibility with adding multiple windows/surface views to the activities root ViewGroup.

I'd guess this direction could be more practical if we had our own RustActivity and maybe a separate RustSubActivity or something for managing something akin to multiple windows. I'm not sure this would currently be practical based on NativeActivity / GameActivity.

Maybe you can expand on how sub-activities work, or just link to some documentation about that parent/child relationship?

Hopefully the above notes give a bit more detail. My naming was probably misleading where I mentioned a 'RustSubActivity' because there's no such special thing as a 'sub activity', but with the way that activities are declared via the manifest for an android application I was vaguely imagining that for a native Rust application wanting to track multiple activities then it might make sense to have a dedicated Activity class for ancillary activities that wouldn't each run a separate android_main() entry point.

/// not be closed by dropping the [`Window`].
/// - **Web:** The window is represented by a `HTMLElementCanvas`, and cannot
/// currently be closed by dropping the [`Window`].
/// - **Android:** Each window is spawned inside its own process, and as such
Copy link
Copy Markdown
Member

@MarijnS95 MarijnS95 Mar 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy-pasting my reply in #696 (comment):

on Android each window is run in its own process

Not at all, anything related to the same app (services, multiple Activities which are probably the closest synonym to Windows) should all run in the same process.

This is actually an open issue, where certain Android system interactions can cause it (the Android system) to launch another Activity one is already open via a callback. Both android-activity and winit are assuming there is only one Activity singleton with (related) state stored across various static globals, and everything falls apart.

It is however unlikely to have multiple windows open at the same time, bar tricks with multiple task stacks, split-window, popout windows and perhaps desktop mode.

I wouldn't mention process boundaries at all. Rather:

  1. Spawning windows in Android most likely doesn't perform in the way the user expects. This could navigate to a different "page" in an app but it's not at all like the desktop experience;
  2. The Android system can spawn new "windows" (Activities) for you (e.g. an app resize or display mode change, or via public Intents launched by other apps) on its own behalf.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, Android multi-resume usually use the same process (but can be configured to do things differently).

@MarijnS95
Copy link
Copy Markdown
Member

MarijnS95 commented Mar 13, 2024

I think I've been led to believe it was impossible to have two activities in a single process

This thought appears to be correct from a Rust+winit+ndk-glue/android-activity perspective, as none of the crates are set up to deal with this, even though it is super common in Android as already elaborated by @rib.

I don't see a compelling use-case in supporting opening multiple Windows from winit on Android: where would this window go? Split-view and pop-up windows are typically dictated by the Android task switcher (and popup overlays like those for video players and navigation are a different beast iirc) EDIT: There appear to be flags to request a new Activity/Window to be started in split-view: https://developer.android.com/reference/android/content/Intent#FLAG_ACTIVITY_LAUNCH_ADJACENT. It could be a different frame in an app that one is navigating to, but structures like that are better left to Java/Kotlin as it requires a lot more machinery.


None of this is however a good reason to not implement support for multiple Activity instances in android-activity and winit. I've seen time and time again (maybe because of missing screenSize/screenLayout/... in the manifests' android:configChanges) that switching an app to split-window or pop-up window causes a new Activity to spawn, shortly before the old one is cleaned up. That case crashes horribly currently.

@daxpedda daxpedda added the DS - android Affects the Android backend label Mar 15, 2024
@MarijnS95 MarijnS95 mentioned this pull request Dec 3, 2024
25 tasks
@InternetOfTofu
Copy link
Copy Markdown

And just the fact that you need to declare activities in your manifest might make them awkward to map to windows in Winit.

I think a possible way of getting around this is to mainly support single activity apps, which is already the recommended way of doing Android development nowadays.

I don't see a compelling use-case in supporting opening multiple Windows from winit on Android

I'm thinking of use-cases related to multi-instancing on large screen platforms. Supporting creating multiple windows in winit can make porting existing desktop applications much easier to Android large screen platforms.

@MarijnS95
Copy link
Copy Markdown
Member

I think a possible way of getting around this is to mainly support single activity apps, which is already the recommended way of doing Android development nowadays.

There's a significant difference between "mainly supporting single-activity apps" and outright misbehaving/crashing when Android tries to open a second Activity (even if with the intent to close a previous one shortly after). I've long planned to fix this, but never got around to finishing and PR'ing it.

I don't see a compelling use-case in supporting opening multiple Windows from winit on Android

I'm thinking of use-cases related to multi-instancing on large screen platforms. Supporting creating multiple windows in winit can make porting existing desktop applications much easier to Android large screen platforms.

Reminder that this is about opening a second window. Doing some recent documentation diving shows https://developer.android.com/reference/android/content/Intent#FLAG_ACTIVITY_LAUNCH_ADJACENT which would be wonderful to extend https://github.com/rust-mobile/android-intent with, but most of that is "out of scope" for winit which should instead focus on first supporting multiple Windows/Activitys in the first place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DS - android Affects the Android backend

Development

Successfully merging this pull request may close these issues.

6 participants