[qtbase] enable ci for android by m-kuhn · Pull Request #35845 · microsoft/vcpkg

m-kuhn · 2023-12-23T16:44:38Z

Enable support for feature egl for android for qtbase.
Specifying it as default-feature for android but at the same time specifying the feature as not supported is inconsistent.

m-kuhn · 2023-12-23T20:17:40Z

ANDROID_SDK_ROOT needs to be available to build qt for android in CI (which was NOT done so far).

vcpkg/ports/qtbase/cmake/qt_install_submodule.cmake

Lines 11 to 13 in 7b3ea91

    
           if(VCPKG_TARGET_IS_ANDROID AND NOT ANDROID_SDK_ROOT) 
        
               message(FATAL_ERROR "${PORT} requires ANDROID_SDK_ROOT to be set. Consider adding it to the triplet." ) 
        
           endif()

The SDK is not installed from what I can see (only the NDK).

vcpkg/scripts/azure-pipelines/android/Dockerfile

Lines 10 to 16 in 7b3ea91

    
           # Download Android NDK 
        
           RUN \ 
        
             wget https://dl.google.com/android/repository/android-ndk-r25c-linux.zip && \ 
        
             unzip android-ndk-r25c-linux.zip && \ 
        
             rm -rf android-ndk-r25c-linux.zip 
        
           ENV ANDROID_NDK_HOME /android-ndk-r25c

We could:

Disable these tests
Install the SDK
- and detect the location from env vars directly in the qt ports (I am starting to wonder if this would be the correct approach. The NDK is also detected from ANDROID_NDK_HOME if I'm not wrong)
- stick to the current approach and require the user to provide ANDROID_SDK_ROOT via triplet, but then we need an overlay triplet for the tests

m-kuhn · 2023-12-26T11:15:05Z

I added the android sdk. It probably needs more to install specific packages with the sdkmanager (tests will tell).

scripts/azure-pipelines/android/Dockerfile

m-kuhn · 2023-12-27T16:07:03Z

Either I am missing something here (I just looked at how ANDROID_NDK_HOME is and tried to apply the same to ANDROID_SDK_ROOT) or the changes in the Dockerfile are not automatically taken into account.
Does someone see why the ENV var from the Dockerfile doesn't make it into the port?

dg0yt · 2023-12-27T16:26:47Z

I guess the docker file is only regenerated once a month.
This is how I currently experiment with NDK r26:
https://github.com/microsoft/vcpkg/pull/35851/files#diff-b810c6816d42fd5c52f27e1ca0da28fd595a2bd00cbb448b443eff8e1739fe57

m-kuhn · 2023-12-27T17:39:47Z

Thanks for the hint.
Still seems not sufficient. I was thinking about VCPKG_ENV_PASSTHROUGH but that's windows specific.

m-kuhn · 2023-12-30T02:20:57Z

Looks like the toolchain file is not loaded at the point where the check happens.

dg0yt · 2023-12-30T07:15:39Z

ports/qtbase/cmake/qt_install_submodule.cmake

-if(VCPKG_TARGET_IS_ANDROID AND NOT ANDROID_SDK_ROOT)
-    message(FATAL_ERROR "${PORT} requires ANDROID_SDK_ROOT to be set. Consider adding it to the triplet." )
+if(VCPKG_TARGET_IS_ANDROID AND NOT ANDROID_SDK_ROOT AND NOT DEFINED ENV{ANDROID_SDK_ROOT}) # The scripts/toolchains/android.cmake file handles writing the env var to a cmake var
+    message(FATAL_ERROR "${PORT} requires ANDROID_SDK_ROOT to be set. Set ANDROID_SDK_ROOT as environment variable or in the triplet." )


qt_install_submodule operates in script mode. It doesn't see results from the toolchain, and AFAICS it uses a cmake variable ANDROID_SDK_ROOT at the moment, not an environment variable. So it would need to be set e.g. in the triplet file.
You may change this file to use the environment variable to initialize the cmake variable iff unset. It is the only use of ANDROID_SDK_ROOT.

Triplets is what I do in my own repo, but I was wondering if we could do this transparently (after all, the ndk path is also picked up from the environment without triplet involved).
But it would certainly be easier to do with the triplet.

That's why I suggested that the portfile could pick the environement variable. (Fully switching to the environment variable might break users.)

Setting it in the triplet ensures the desired (?!) effect on ABI hashes, but is more intrusive on the user side.
The NDK environment variable already affects ABI hashing via the compiler binaries hashes.

Mabye useful in the future: microsoft/vcpkg-tool#1315. This would hash the extra script, and the triplet could remain unchanged.

That's why I suggested that the portfile could pick the environement variable. (Fully switching to the environment variable might break users.)

If it's set to an absolute path, yes. But that would mean every user has to provide a local triplet. And if it's set via env var (as done with the latest iteration of this PR) it means that the ABI hash of the triplet is not really reliable.

dg0yt · 2023-12-30T09:31:53Z

triplets/arm-neon-android.cmake

 set(VCPKG_CMAKE_SYSTEM_NAME Android)
 set(VCPKG_MAKE_BUILD_TRIPLET "--host=armv7a-linux-androideabi")
 set(VCPKG_CMAKE_CONFIGURE_OPTIONS -DANDROID_ABI=armeabi-v7a -DANDROID_ARM_NEON=ON)
+set(ANDROID_SDK_ROOT $ENV{ANDROID_SDK_ROOT})


This doesn't help with ABI hashes: The content of the triplet files remains independent of the content of ENV{ANDROID_SDK_ROOT}.

There is https://learn.microsoft.com/en-us/vcpkg/users/triplets#vcpkg_env_passthrough, but it is documented "for Windows"...

The only way I know to bring this into ABI hashes is by (a) writing it into an overlay triplet, but as you mentioned, this is intrusive on user side.
Or we respect environment variables, either by inspecting them (b) in the triplet (done currently) or (c) in the android toolchain (done before) or (d) in the port and forward it to cmake configure from there.

And if I understand correctly, your preference is (d)?

m-kuhn · 2023-12-30T14:47:51Z

I'm afraid I am hitting a wall with installing the SDK through test-modified-ports.ps1, it seems that it runs with limited permissions and I cannot install java.

m-kuhn · 2024-01-05T00:28:57Z

@dan-shaw @BillyONeal what is the best way to get an update into the android ci docker image? I assume the current changes should be sufficient.

NB: happy to update the version as soon as the path forward is clear, there are currently other PRs waiting for qtbase which are probably merged earlier.

scripts/azure-pipelines/test-modified-ports.ps1

Co-authored-by: Kai Pastor <dg0yt@darc.de>

dg0yt · 2024-07-28T09:17:01Z

Please try m-kuhn#1

Separate script to acquire SDK, limit CI

dg0yt · 2024-08-03T08:55:51Z

Hm, still not successful in CI.

Setting up the Android SDK in /vcpkg/android-sdk
Loading package information...                                                  
Loading local repository...                                                     
[                                       ] 3% Loading local repository...        
[                                       ] 3% Fetch remote repository...         
[=                                      ] 3% Fetch remote repository...         
                                                                                
[=                                      ] 3% Fetch remote repository...         
Warning: Failed to download any source lists!
                                                                                
Warning: IO exception while downloading manifest
[=                                      ] 3% Fetch remote repository...         
                                                                                
Warning: IO exception while downloading manifest
[=                                      ] 3% Fetch remote repository...         
Warning: Still waiting for package manifests to be fetched remotely.
                                                                                
[=                                      ] 3% Fetch remote repository...         
[=                                      ] 3% Computing updates...               
[===                                    ] 8% Computing updates...               
[===                                    ] 10% Computing updates...              
                                                                                
[===                                    ] 10% Computing updates...              
Warning: Failed to find package 'platform-tools'
yes: standard output: Broken pipe
Loading package information...                                                  
Loading local repository...                                                     
[=========                              ] 25% Loading local repository...       
[=========                              ] 25% Fetch remote repository...        
[=======================================] 100% Fetch remote repository...

m-kuhn · 2024-08-03T10:09:02Z

Is network connectivity blocked except through vcpkg tool?

dg0yt · 2024-08-03T13:42:33Z

Is network connectivity blocked except through vcpkg tool?

IDK. Let's try m-kuhn#2.

Fetch packages directly

dg0yt · 2024-08-03T15:45:00Z

Installed packages:
  Path                 | Version | Description                       | Location            
  -------              | ------- | -------                           | -------             
  build-tools;34.0.0   | 34.0.0  | Android SDK Build-Tools 34        | build-tools/34.0.0  
  platform-tools       | 35.0.1  | Android SDK Platform-Tools 35.0.1 | platform-tools      
  platforms;android-34 | 3       | Android SDK Platform 34, rev 3    | platforms/android-34

Good.

CMake Error at ports/qtbase/cmake/qt_install_submodule.cmake:13 (message):
  qtbase requires ANDROID_SDK_ROOT to be set.  Set ANDROID_SDK_ROOT as
  environment variable or in the triplet.

Okay, the CI script sets ANDROID_HOME because this is what is given in https://developer.android.com/tools/variables#envar, but the Qt still wants ANDROID_SDK_ROOT. This should be bridged in the port.

m-kuhn · 2024-08-04T15:36:15Z

Okay, the CI script sets ANDROID_HOME because this is what is given in https://developer.android.com/tools/variables#envar, but the Qt still wants ANDROID_SDK_ROOT. This should be bridged in the port.

I tried to implement this in the port, but it seems ENV{ANDROID_HOME} is not visible there.

Cherry-picked from microsoft#35845

m-kuhn · 2024-11-10T08:57:36Z

Closing in favor of #42080

Adela0814 self-assigned this Dec 25, 2023

Adela0814 added the category:port-feature The issue is with a library, which is requesting new capabilities that didn’t exist label Dec 25, 2023

m-kuhn force-pushed the qtbase_android branch from 74ac394 to e320000 Compare December 26, 2023 11:16

m-kuhn commented Dec 26, 2023

View reviewed changes

scripts/azure-pipelines/android/Dockerfile Outdated Show resolved Hide resolved

m-kuhn force-pushed the qtbase_android branch 5 times, most recently from b0926a3 to 5a7757f Compare December 30, 2023 02:18

m-kuhn force-pushed the qtbase_android branch 2 times, most recently from 7d74cef to a8938b6 Compare December 30, 2023 02:23

dg0yt reviewed Dec 30, 2023

View reviewed changes

m-kuhn force-pushed the qtbase_android branch from c09d856 to e3dfa2b Compare December 30, 2023 09:25

dg0yt reviewed Dec 30, 2023

View reviewed changes

m-kuhn force-pushed the qtbase_android branch 6 times, most recently from 793dc60 to d0e9db8 Compare December 30, 2023 14:46

m-kuhn changed the title ~~[qtbase] enable egl for android~~ [qtbase] enable ci for android Jan 1, 2024

m-kuhn mentioned this pull request Jan 1, 2024

[qtbase] enable egl for android #35961

Closed

m-kuhn added 2 commits July 26, 2024 12:03

license

dba85cf

Align support of qtbase[dbus] with dbus

e232626

m-kuhn force-pushed the qtbase_android branch from 1fa7b08 to e232626 Compare July 26, 2024 10:06

v db

201529f

dg0yt reviewed Jul 26, 2024

View reviewed changes

scripts/azure-pipelines/test-modified-ports.ps1 Outdated Show resolved Hide resolved

m-kuhn and others added 3 commits July 26, 2024 13:52

Update scripts/azure-pipelines/test-modified-ports.ps1

c8e91b6

Co-authored-by: Kai Pastor <dg0yt@darc.de>

Separate script to acquire SDK

30731f5

Limit CI to android

760bf48

dg0yt mentioned this pull request Jul 28, 2024

Separate script to acquire SDK, limit CI m-kuhn/vcpkg#1

Merged

dg0yt and others added 3 commits July 28, 2024 11:57

Fixup

9785d29

Merge pull request #1 from dg0yt/qtbase-android

5d93bf5

Separate script to acquire SDK, limit CI

Merge branch 'master' into qtbase_android

cfe33dc

Fetch packages directly

f1f358f

Merge pull request #2 from dg0yt/qtbase-android

a9d7fae

Fetch packages directly

m-kuhn added 2 commits August 4, 2024 17:09

Check for env var ANDROID_HOME in qtbase

a3d5a77

Remove triplet changes

7e22785

m-kuhn force-pushed the qtbase_android branch from 72d2bae to 7e22785 Compare August 4, 2024 15:11

v db

2250316

dg0yt mentioned this pull request Sep 13, 2024

[qtbase] build failure for x64-android #40762

Closed

JavierMatosD marked this pull request as draft October 8, 2024 21:51

dg0yt pushed a commit to dg0yt/vcpkg that referenced this pull request Nov 9, 2024

Acquire JDK and Android SDK

3ffdaaf

Cherry-picked from microsoft#35845

dg0yt mentioned this pull request Nov 9, 2024

[qt] Enable android #42080

Merged

3 tasks

m-kuhn closed this Nov 10, 2024

Conversation

m-kuhn commented Dec 23, 2023

Uh oh!

m-kuhn commented Dec 23, 2023

Uh oh!

m-kuhn commented Dec 26, 2023

Uh oh!

Uh oh!

m-kuhn commented Dec 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dg0yt commented Dec 27, 2023

Uh oh!

m-kuhn commented Dec 27, 2023

Uh oh!

m-kuhn commented Dec 30, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

m-kuhn commented Dec 30, 2023

Uh oh!

m-kuhn commented Jan 5, 2024

Uh oh!

Uh oh!

dg0yt commented Jul 28, 2024

Uh oh!

dg0yt commented Aug 3, 2024

Uh oh!

m-kuhn commented Aug 3, 2024

Uh oh!

dg0yt commented Aug 3, 2024

Uh oh!

dg0yt commented Aug 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

m-kuhn commented Aug 4, 2024

Uh oh!

m-kuhn commented Nov 10, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

m-kuhn commented Dec 27, 2023 •

edited

Loading

dg0yt commented Aug 3, 2024 •

edited

Loading