Add experimental support for Apple VisionOS platform by komakai · Pull Request #24136 · opencv/opencv

komakai · 2023-08-09T23:46:15Z

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch

This is dependent on cmake support for visionos which is currently in progress.
Creating PR now to test that there are no regressions in iOS and macOS builds

komakai · 2023-08-09T23:49:06Z

Build command for testing visionos support:

./opencv/platforms/apple/build_xcframework.py --out visionostest --visionos_archs arm64 --visionsimulator_archs x86_64 --build_only_specified_archs

CMakeLists.txt

platforms/ios/cmake/Toolchains/common-ios-toolchain.cmake

asmorkalov · 2023-08-10T06:32:28Z

@komakai Great job! Thanks a lot for the patch!

dkurt · 2023-08-10T07:50:58Z

CMakeLists.txt

  VERIFY HAVE_JASPER)
 OCV_OPTION(WITH_OPENJPEG "Include JPEG2K support (OpenJPEG)" ON
-  VISIBLE_IF NOT IOS
+  VISIBLE_IF NOT IOS AND NOT VISIONOS


I see a lot of IOS AND VISIONOS in pair. Is there a united variable for all? Like this:

OCV_OPTION(WITH_OPENJPEG "Include JPEG2K support (OpenJPEG)" ON VISIBLE_IF NOT APPLE

APPLE also includes macOS
I guess we could make something like APPLE_DEVICE but I feel like the names IOS and VISIONOS are going to be easier to understand for maintainers

komakai · 2023-08-24T05:06:55Z

Converting to draft until the necessary changes to cmake are released

asmorkalov · 2023-11-08T13:38:48Z

VisionOS added in CMake 3.28 release: https://cmake.org/cmake/help/latest/release/3.28.html
Related MR: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/8581

asmorkalov · 2023-11-08T13:41:31Z

VisionOS support is not complete in CMake for now: https://gitlab.kitware.com/cmake/cmake/-/issues/25266

komakai · 2023-11-11T03:54:03Z

VisionOS support is not complete in CMake for now: https://gitlab.kitware.com/cmake/cmake/-/issues/25266

I will try to build this PR with CMake-3.28.0-rc4 when I have a minute and check if it builds OK

komakai · 2023-11-12T09:51:58Z

Could build with CMake from https://github.com/Kitware/CMake/releases/download/v3.28.0-rc4/cmake-3.28.0-rc4-macos-universal.dmg with the below change:
On iOS we have the following defines

IPHONEOS in Toolchain-iPhoneOS_Xcode.cmake (OpenCV file)
IPHONESIMULATOR in Toolchain-iPhoneSimulator_Xcode.cmake (OpenCV file)
IOS in iOS-Initialize.cmake (CMake file)

However for visionOS CMake does not have an equivalent define in visionOS-Initialize.cmake so I added a define XROS to Toolchain-XROS_Xcode.cmake and Toolchain-XRSimulator_Xcode.cmake which is effectly equal to VISIONOS or VISIONSIMULATOR and updated all the cmake files to use this define instead of VISIONOS

komakai · 2023-11-12T09:57:05Z

To test this I was able to create a VisionOS project and add the OpenCV framework and add some simple calls to create a Mat and do some operations on the Mat. I could see that the code built OK but I don't have a device and the latest VisionOS simulator only runs on Apple Silicon so I could not actually run the code.
Unless there is someone with an Apple Silicon Mac who can test this I suggest to accept this PR as is since I don't know when I'll get a chance to test it more thoroughly

gdyuldin · 2023-11-14T12:52:24Z

Hi.
Tested under simulator on Apple Silicon (M1) with applying GaussianBlur - it works, when using framework from visionsimulator

Build command:
./opencv/platforms/apple/build_xcframework.py --out visionostest --visionos_archs arm64 --visionsimulator_archs arm64 --build_only_specified_archs

Tools versions:

cmake - 3.28.0-rc4
Xcode. -Version 15.1 beta (15C5028h)
macOS - 13.5.1 (22G90) (Ventura)

With framework from visionos directory build failed with linker error: ld: framework 'opencv2' not found

fengyuentau · 2023-11-27T05:31:51Z

Did I miss something to get a successful build? I tested with:

macOS Sonoma 14.1.1
Xcode 15.0.1
CMake 3.28.0-rc5 (cmake-3.28.0-rc5-macos-universal.tar.gz)

and both commands:

./opencv/platforms/apple/build_xcframework.py --out visionostest --visionos_archs arm64 --visionsimulator_archs arm64 --build_only_specified_archs

./opencv/platforms/apple/build_xcframework.py --out visionostest --visionos_archs arm64 --visionsimulator_archs x86_64 --build_only_specified_archs

but only got errors:

CMake Error at /workspace/cmake-3.28.0-rc5-macos-universal/CMake.app/Contents/share/cmake-3.28/Modules/Platform/visionOS-Initialize.cmake:4 (message):
  xros is not an visionOS SDK
Call Stack (most recent call first):
  /workspace/cmake-3.28.0-rc5-macos-universal/CMake.app/Contents/share/cmake-3.28/Modules/CMakeSystemSpecificInitialize.cmake:34 (include)
  CMakeLists.txt:117 (enable_language)

gdyuldin · 2023-11-28T13:19:44Z

Did I miss something to get a successful build? I tested with:

macOS Sonoma 14.1.1
Xcode 15.0.1
CMake 3.28.0-rc5 (cmake-3.28.0-rc5-macos-universal.tar.gz)

and both commands:

./opencv/platforms/apple/build_xcframework.py --out visionostest --visionos_archs arm64 --visionsimulator_archs arm64 --build_only_specified_archs

./opencv/platforms/apple/build_xcframework.py --out visionostest --visionos_archs arm64 --visionsimulator_archs x86_64 --build_only_specified_archs

but only got errors:

CMake Error at /workspace/cmake-3.28.0-rc5-macos-universal/CMake.app/Contents/share/cmake-3.28/Modules/Platform/visionOS-Initialize.cmake:4 (message):
  xros is not an visionOS SDK
Call Stack (most recent call first):
  /workspace/cmake-3.28.0-rc5-macos-universal/CMake.app/Contents/share/cmake-3.28/Modules/CMakeSystemSpecificInitialize.cmake:34 (include)
  CMakeLists.txt:117 (enable_language)

Looks like xcode 15.0.1 doesn't support VisionOS (link)
You should use beta version of xcode (for example - this )

fengyuentau · 2023-11-29T09:35:40Z

Looks like xcode 15.0.1 doesn't support VisionOS (link)
You should use beta version of xcode (for example - this )

Ok, thanks. Let me try with xcode beta then.

Do you know how to configure CMake to use Xcode beta? I have installed Xcode 15.1 beta but CMake still complains xros is not an visionOS SDK.

komakai · 2023-11-30T01:37:25Z

Do you know how to configure CMake to use Xcode beta? I have installed Xcode 15.1 beta but CMake still complains xros is not an visionOS SDK.

Use xcode-select to select the beta-version - then CMake will use that version

fengyuentau · 2023-12-20T05:58:37Z

Do you know how to configure CMake to use Xcode beta? I have installed Xcode 15.1 beta but CMake still complains xros is not an visionOS SDK.

Use xcode-select to select the beta-version - then CMake will use that version

Thank you for the instructions. I can confirm that build can be done successfully with cmake-3.28.0-rc5 and Xcode-15.1-beta and command ./opencv/platforms/apple/build_xcframework.py --out visionostest --visionos_archs arm64 --visionsimulator_archs arm64 --build_only_specified_archs.

cc @asmorkalov

asmorkalov

👍

klatism · 2024-02-26T03:04:06Z

@komakai I'm getting:

/Users/klatism/Git/opencv/modules/videoio/src/cap_avfoundation.mm:1314:32: error: unrecognized platform name visionOS
if (@available(iOS 11, visionOS 1, *)) {

with:
MacOS: Ventura 13.6.1
XCode: 14.3.1
CMake: 3.27.0

that if statement seems to be within #if TARGET_OS_VISION block. I'm wondering could the check for visionOS be removed from @available?

komakai · 2024-03-06T05:19:27Z

@klatism

I'm wondering could the check for visionOS be removed from @available?

Yes that sounds completely reasonable. I look forward to your Pull Request

klatism · 2024-03-08T18:27:35Z

@komakai please take a look: #25189. Thanks!

komakai · 2024-03-10T02:42:38Z

@komakai please take a look: #25189. Thanks!

Thanks - looks good

Add experimental support for Apple VisionOS platform opencv#24136 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch This is dependent on cmake support for VisionOs which is currently in progress. Creating PR now to test that there are no regressions in iOS and macOS builds

asmorkalov reviewed Aug 10, 2023

View reviewed changes

CMakeLists.txt Outdated Show resolved Hide resolved

platforms/ios/cmake/Toolchains/common-ios-toolchain.cmake Outdated Show resolved Hide resolved

asmorkalov requested a review from VadimLevin August 10, 2023 06:32

asmorkalov added platform: ios/osx feature labels Aug 10, 2023

asmorkalov added this to the 4.9.0 milestone Aug 10, 2023

asmorkalov assigned VadimLevin Aug 10, 2023

dkurt reviewed Aug 10, 2023

View reviewed changes

Add support for visionOS platform

004a910

komakai force-pushed the visionos_support branch from e47aa4a to 004a910 Compare August 11, 2023 03:20

komakai added 3 commits August 11, 2023 12:30

Reinstate extra modules include for iOS build

0c5f549

Fix for APIs not available in visionOS

389a7c8

Additional cmake fixes for visionOS

c523c55

komakai marked this pull request as draft August 24, 2023 05:07

VadimLevin removed their assignment Aug 28, 2023

Add XROS define to mean VISIONOS or VISIONSIMULATOR

415ed50

komakai marked this pull request as ready for review November 12, 2023 09:57

asmorkalov added the community help requested label Nov 13, 2023

asmorkalov approved these changes Dec 20, 2023

View reviewed changes

CMake warning fix.

16a3685

asmorkalov changed the title ~~Add support for visionOS platform~~ Add experimental support for visionOS platform Dec 20, 2023

asmorkalov changed the title ~~Add experimental support for visionOS platform~~ Add experimental support for Apple VisionOS platform Dec 20, 2023

asmorkalov assigned fengyuentau Dec 20, 2023

asmorkalov merged commit 3d9cb53 into opencv:4.x Dec 20, 2023

asmorkalov mentioned this pull request Jan 19, 2024

5.x merge 4.x #24862

Merged

Uh oh!

Conversation

komakai commented Aug 9, 2023

Pull Request Readiness Checklist

Uh oh!

komakai commented Aug 9, 2023

Uh oh!

Uh oh!

Uh oh!

asmorkalov commented Aug 10, 2023

Uh oh!

dkurt Aug 10, 2023

Choose a reason for hiding this comment

Uh oh!

komakai Aug 11, 2023

Choose a reason for hiding this comment

Uh oh!

komakai commented Aug 24, 2023

Uh oh!

asmorkalov commented Nov 8, 2023

Uh oh!

asmorkalov commented Nov 8, 2023

Uh oh!

komakai commented Nov 11, 2023

Uh oh!

komakai commented Nov 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

komakai commented Nov 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gdyuldin commented Nov 14, 2023

Uh oh!

fengyuentau commented Nov 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gdyuldin commented Nov 28, 2023

Uh oh!

fengyuentau commented Nov 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

komakai commented Nov 30, 2023

Uh oh!

fengyuentau commented Dec 20, 2023

Uh oh!

asmorkalov left a comment

Choose a reason for hiding this comment

Uh oh!

klatism commented Feb 26, 2024

Uh oh!

komakai commented Mar 6, 2024

Uh oh!

klatism commented Mar 8, 2024

Uh oh!

komakai commented Mar 10, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

komakai commented Nov 12, 2023 •

edited

Loading

komakai commented Nov 12, 2023 •

edited

Loading

fengyuentau commented Nov 27, 2023 •

edited

Loading

fengyuentau commented Nov 29, 2023 •

edited

Loading