Building iOS apps with M1 hardware #141

shepting · 2021-11-02T21:09:26Z

shepting
Nov 2, 2021

We are all pretty excited to start using Apple's new MacBook Pro devices with the M1 Pro and M1 Max chips, but the current landscape of building for the iOS simulator has quite a bit of extra complexity.

What are folks doing around support for local builds with the new M1 chips? Specifically around:

cross-compiling for x86 with the native ARM compilers in Xcode
running tests with XCTest which will currently crash when given x86 binaries

Would love to see some descriptions and any little snippets of wrapper scripts or pseudo-code that you can share to work around issues!

omarzl · 2021-11-03T00:56:30Z

omarzl
Nov 3, 2021

We are running x86 test bundle binaries via an app host to sort the Failed to load the test bundle, doesn’t contain a version for the current architecture issue, have you tried that?
Only our libraries are being compiled with Bazel so the test bundles are still being compiled in the "classic" way, since we are using Cocoapods we only need to set requires_app_host = true to the test spec to use an app host.

0 replies

keith · 2021-11-04T17:02:59Z

keith
Nov 4, 2021
Maintainer

Right now we're building our apps and tests for x86_64 still as we wait on some vendors to support arm64 simulators, for us this has mostly worked fine so far, we haven't noticed any difference in test / app run experience even though it's running through rosetta.

We hit a few small snags like the ones @omarzl mentioned around running tests. In Xcode as long as we have a test host (we use an empty app) for each unit test target things work fine, but without them they architecture missing issue. We have the same issue from the command line but we are able to work around that with this hack:

platform_developer_dir="$(xcode-select -p)/Platforms/iPhoneSimulator.platform/Developer"
xctest_binary="$platform_developer_dir/Library/Xcode/Agents/xctest"
if [[ $(arch) == arm64 ]]; then
  sliced_xctest_binary=/tmp/xctest_intel
  if [[ ! -x "$sliced_xctest_binary" ]]; then
    lipo -thin x86_64 -output "$sliced_xctest_binary" "$xctest_binary"
  fi

  xctest_binary=$sliced_xctest_binary
fi

SIMCTL_CHILD_DYLD_FALLBACK_LIBRARY_PATH="$platform_developer_dir/usr/lib" \
  SIMCTL_CHILD_DYLD_FALLBACK_FRAMEWORK_PATH="$platform_developer_dir/Library/Frameworks:$platform_developer_dir/Library/Private/Frameworks" \
  SIMCTL_CHILD_LLVM_PROFILE_FILE="$profraw" \
  xcrun simctl \
  spawn \
  "$id" \
  "$xctest_binary" \
  -XCTest All \
  "$test_tmp_dir/$test_bundle_name.xctest"

Instead of forcing everything through a test host (which just has some impact on performance).

We also had to set EXCLUDED_ARCHS[sdk=iphonesimulator*] = arm64 in our xcconfigs to make sure Xcode was happy building for x86_64 (which because of our bazel setup we're doing regardless of this build setting's value)

3 replies

bogo Nov 7, 2021

Attempted to port the above solution directly to xctestrunner here: google/xctestrunner#40. Works great for me - all tests work "out of the box" on M1 now!

shepting Dec 9, 2021
Author

@keith Where does the $id variable get defined? Do you have an example value it can take on?

keith Dec 10, 2021
Maintainer

That's the udid from the simulator, so if you create one manually you can capture that one and pass it here

vfn · 2021-11-04T21:48:53Z

vfn
Nov 4, 2021

I have experienced this issue recently and I got stuck with the GoogleMaps library, that hasn't been updated yet. The solution I found was to wrap it into an xcframework as that appears to be the only way to have two arm64 versions of a framework.

You can repurpose the arm64 version of existing frameworks to also run in the arm64 simulator.

As you can see in the script below, that I forked from another repo and modifed to add the simulator support, after creating the iphoneos archive that uses the arm64 arch, it then calls vtool, changing the architecture of the original framework, and then proceed to archive it for the simulator and finaly package it all into an xcframework containing the arm64 framework for devices and the framework containing amr64 and x86_64 for sims.

https://github.com/vfn/GoogleMaps-SP/blob/main/make_xcframework.sh

function convert_frameworks_arm64_to_iphonesimulator() {
  project_name=$1
  framework_name=$2

  xcrun vtool -arch arm64 \
    -set-build-version 7 11.0 13.7 \
    -replace \
    -output "../Carthage/Build/iOS/$framework_name.framework/$framework_name" \
    "../Carthage/Build/iOS/$framework_name.framework/$framework_name"
}

I mostly followed the instructions in this post https://bogo.wtf/arm64-to-sim-dylibs.html and a bit of trial and error.

Depending on how many dependencies you have on your project, it might be worth to do something similar while you wait for them to be updated.

In my case, I was experiencing a few UI glitches, like scrolling issues and some others when running the simulator using Rosetta.

2 replies

keith Nov 5, 2021
Maintainer

Here's another fun implementation of this bazel-ios/rules_ios#346

jerrymarino Nov 10, 2021

Yep the plan is that the Bazel rules will ultimatley deal with this, to make updating these lagging deps a breeze 🥳

dive · 2021-11-05T09:22:13Z

dive
Nov 5, 2021

In case you missed it, there is an excellent overview from the Swift Package Index – M1 Pro and M1 Max Xcode Build and Test Benchmarks.

0 replies

tapthaker · 2021-11-08T21:35:10Z

tapthaker
Nov 8, 2021

We are building all our artifacts for x86_64 on M1 as well.

Here are some of the challenges that we faced:

As some folks already mentioned, xctest built for x86_64-iphonesimulator fails to execute when running Xcode natively.

However, we couldn't just use a test host to execute our tests inside Xcode. For example, numerous tests have assertions around frame and deep links whose behavior changes if we use test-host.

We ended up creating an xctest_wrapper binary that replaces the xctest inside Xcode. This xctest_wrapper in turn calls xctest_intel similar to what is mentioned in #141 (comment)

Please note that the xctest_wrapper needs to be code-signed and a universal binary for Xcode to accept it. Or else it'll throw NSMachError:

NSMachErrorDomain Code=111

We are still on a forked version of Bazel, which doesn't have some of the changes required to run bazel natively on M1. So as an interim fix to get the most performance out of M1, we wrap our compiler/linker invocations to run natively using the https://www.unix.com/man-page/osx/1/arch/ command.

0 replies

jerrymarino · 2021-11-10T18:01:03Z

jerrymarino
Nov 10, 2021

Some other notes on this, as I've got this up and running on the ARM sim.

rules_ios will soon get a conditional feature to rewrite the macho headers of static libs, frameworks, etc device deps to getup and running on M1 Sim with non-M1-ready deps. Most iOS developers code/build outputs aren't specific to a simulator or a device. The pre-processor defines are one common example where that doesn't hold true and there are others.
Under cocoapods/xcode the script update_in_place.sh in that PR is simple way to get deps up and running on the M1 sim without Bazel but may have caveats. You can of course integrate that script with Xcode to update deps and put/load them from derived data
building with Bazel 5 seems to be a dependency. Specifically, it has patches to handle the new CPU type ios_sim_arm64. There are a handful of issues in the Bazel pre-release builds, one of the largest one being [Bazel 5][objc_library]: NullPointerException thrown during Starlark evaluation (//:main_objc) - at <starlark>.create_compilation_attributes(<builtin>:0) bazelbuild/bazel#14173 which I believe Amber is familiar with the fix and may have a solution soon.
It is possible to build for apple-simulator-arm64 under Bazel 4 with some caveats and hacks. Specifically, use a toolchain that's conditionally enabled to treat the ios_x86_64 cpu type as an arm one. Just override the compiler / linker arguments to be arm64. That's a stopgap of course till Bazel 5 or 6 is cut/fixed.
To generally handle unarchiving, ar doesn't handle duplicate object files. Unfortuntaetly, some build systems will pack duplicate objects that break using ar nativly. I have a granola but bullet-proof implementation of this here https://github.com/bazel-ios/rules_ios/pull/346/files#diff-c88c0cd278946c1673a16524fac11a30a2b5567193b8d918da08e7d81162027cR6

2 replies

keith Nov 10, 2021
Maintainer

To generally handle unarchiving, ar doesn't handle duplicate object files. Unfortuntaetly, some build systems will pack duplicate objects that break using ar nativly. I have a granola but bullet-proof implementation of this here bazel-ios/rules_ios#346 (files)

You probably shouldn't have these for archives built with bazel at least?

jerrymarino Nov 10, 2021

That's spot on - Bazel writes libtool invocations the form $object-$sha256.o - most other build systems just pack them as $object.o

jerrymarino · 2021-11-12T00:26:21Z

jerrymarino
Nov 12, 2021

Today I was looking at more of the edge cases and possibilities of unarchiving and updating ARM64 object files ( e.g. unarchiving ) in the Bazel rule that currently runs arm64-to-sim.

I noticed that there was a code path inside of the linker that will allow this behavior. Specifically if you pass -syslibroot /some/path/ where /some/path/ contains AppleInternal to ld, it will link ARM64 device objects under the simulator target and print a warning instead of the error. You can just run mkdir -p /some/path/AppleInternal to create the directory. This would simplify the Bazel rule, at the cost of having a warning.

Of course you'd still need to handle dynamic, because it failing at runtime. Possibly, dyld may have a similar code path.. If anyone has other approaches to handle dylibs at runtime ( e.g. as opposed to vtool ) I'd be interested to know

0 replies

BalestraPatrick · 2021-11-22T09:44:13Z

BalestraPatrick
Nov 22, 2021

Following the discussion above, we also have this in our xcconfig:

ARCHS[sdk=iphonesimulator*] = x86_64
ARCHS[sdk=watchsimulator*] = x86_64

This works for our iOS app and can give us more time to update all our dependencies to XCFramework with an arm64 iOS Simulator slice.

This doesn't work so great for watchOS though. It seems like the Watch Simulator is very unhappy about this and refuses to install the built bundle because it thinks it's a WatchKit 1 app which it doesn't support anymore.

WatchKit 1 apps aren't supported on this version of watchOS. Quitting "(null)"

I filed FB9768330 with a sample project because this happens on a brand new project too. It seems like there's currently no way to test watchOS apps except when building them natively for arm64, which I guess quite some projects currently can't do.

0 replies

thedavidharris · 2021-12-18T17:29:12Z

thedavidharris
Dec 18, 2021

Has anyone identified if there's a similar thing you have to do to metallib files to make this workable? We were seeing some crashes there and not quite sure if this was even possible. The closed source frameworks have a iphone-simulator-default.metallib and an iphoneos-default.metallib. Was going to experiment a bit (I'm unfamiliar with Metal programming) so see if there's a better way to identify what the differences are. The binaries are definitley different, but I'm not sure if there's anything arch specific or if the right thing to do is continue to use the iphone-simulator-default.metallib

It is possible to decompile those back into .air files, which appear to just be llvm-bitcode, but then I'm not quite sure where to go from there other than just recreate the same metal-lib files, as the .air files also appear to be different from each group.

0 replies

hendych · 2022-02-18T16:05:30Z

hendych
Feb 18, 2022

To support M1:

Converted all 3rd party libraries to use xcframework with arm64 simulator support by updating it or using arm64ToSim just like the other mentioned if the vendor doesn't support it yet.
For static library, arm64ToSim will need ar tool which extract and repack the lib. But I found issues that when running ar to extract, some symbols will be missing in the repacked lib. Details here
We use Buck with Rosetta so we still use x86_64 it will build iphonesimulator-x86_64 and run the tests using xctestrunner with no problem. Some devs used Xcode and we don't see any compile errors in native M1 arm64 since we converted all lib to support both arch. Running XCTest in Xcode somehow successfully compiled but encountered error
xctest (77591) encountered an error (Test runner never began executing tests after launching.. Might be not related to the CPU arch (?).
We are in the mid of migrating Buck to Bazel. Bazel 5 seems to work with arm64 without any problem.

Xcode: 13.1
Simulator iOS 15.0

0 replies

Building iOS apps with M1 hardware #141

Uh oh!

Replies: 10 comments · 7 replies

Uh oh!

Uh oh!

keith Nov 4, 2021 Maintainer

Uh oh!

Uh oh!

shepting Dec 9, 2021 Author

Uh oh!

keith Dec 10, 2021 Maintainer

Uh oh!

Uh oh!

Uh oh!

keith Nov 5, 2021 Maintainer

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

keith Nov 10, 2021 Maintainer

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Replies: 10 comments 7 replies

keith
Nov 4, 2021
Maintainer

shepting Dec 9, 2021
Author

keith Dec 10, 2021
Maintainer

keith Nov 5, 2021
Maintainer

keith Nov 10, 2021
Maintainer