Skip to content

Enable Swift testing in the iOS embedder#185712

Merged
auto-submit[bot] merged 25 commits into
flutter:masterfrom
LongCatIsLooong:ios-swift-testing
May 20, 2026
Merged

Enable Swift testing in the iOS embedder#185712
auto-submit[bot] merged 25 commits into
flutter:masterfrom
LongCatIsLooong:ios-swift-testing

Conversation

@LongCatIsLooong

@LongCatIsLooong LongCatIsLooong commented Apr 28, 2026

Copy link
Copy Markdown
Contributor

This patch:

  1. Updates the -mios-simulator-version-min ldflag to 15.0 (this fixes the crash during test discovery).
  2. refactors swiftc.py a bit, moves the target triplet logic from darwin/BUILD.gn to swiftc.py so you can override the ios version in a test target (but it's not absolutely necessary for this change since it's the linker flag that makes the difference).
  3. Migrates a few xctest tests to swift testing tests. The names of the tests are kept as-is so the diff is easier to read.

The Crash

(lldb) bt
* thread #9, name = '[Swift Testing] test discovery - loading test #0 (Task 3)', queue = 'com.apple.root.user-initiated-qos.cooperative', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000000000000 
    frame #1: 0x000000010ad0fb80 libios_test_flutter.dylib`closure #1() in variable initialization expression of static AccessibilityFeaturesTests.$s22ios_test_flutter_swift26AccessibilityFeaturesTestsV43accessibilityFeatureFlagRawValuesAreCorrect4TestfMp_24accessorc8120add5c86a9e3fMu_ at @__swiftmacro_22ios_test_flutter_swift26AccessibilityFeaturesTestsV43accessibilityFeatureFlagRawValuesAreCorrect4TestfMp_.swift:0
    frame #2: 0x000000010224a39c Testing`partial apply for closure #1 in __checkClosureCall(performing:throws:mismatchExplanation:expression:comments:isRequired:sourceLocation:)
    frame #3: 0x000000010224a39c Testing`partial apply for closure #1 in __checkClosureCall(performing:throws:mismatchExplanation:expression:comments:isRequired:sourceLocation:)
    frame #4: 0x00000001022d8de8 Testing`(1) await resume partial function for generic specialization <(Testing.Test, Testing.Runner.Plan.Action)> of reabstraction thunk helper <τ_0_0 where τ_0_0: Swift.Sendable> from @escaping @isolated(any) @callee_guaranteed @async () -> (@out τ_0_0) to @escaping @callee_guaranteed @async () -> (@out τ_0_0, @error @owned Swift.Error)
    frame #5: 0x000000010224ae24 Testing`closure #1 in __checkClosureCall(throws:performing:expression:comments:isRequired:sourceLocation:)

Initially I thought swift testing doesn't support test discovery in linked dylibs but it's apparently not the case. And according to gemini it's not anything else in a binary but

Load command 9
        cmd LC_BUILD_VERSION
   platform 7
-     minos 14.0
+     minos 15.6
        sdk 26.4

As it tried to use 15.6 in swift_flags and put 14.0 in ldflags and hit the same crash.

This is what gemini has to say:

This is the definitive end of the road! 🔬

I have mathematically eliminated every other variable in computing existence. Between a binary that crashes and a binary that runs, literally NOTHING changed except that single number.

Also it thinks it's because of this in dyld:

Look at the code in dyldMain.cpp at Line 430:

cpp
// dyldMain.cpp Line 430
// TODO: Remove once drop support for simulators older than iOS 17, tvOS 15, and watchOS 8
mach_o::PlatformAndVersions pvs = dyldSimHdr->platformAndVersions();
mach_o::Policy policy(mach_o::Architecture(), pvs, 0);
// [!!] THE SMOKING GUN [!!]
// Old simulators add the main executable to all_image_info in the simulator process, not in the host
if ( policy.enforceImageListRemoveMainExecutable() ) {
STACK_ALLOC_ARRAY(const mach_header*, mhs, 1);
mhs.push_back(state.config.process.mainExecutableMF);
std::span<const mach_header*> mhSpan(&mhs[0], 1);

// LITERALLY REMOVES THE IMAGE FROM THE TRACKING TABLE!!
state.externallyViewable->removeImages(state.persistentAllocator, ephemeralAllocator, mhSpan);

}
Why this triggers your exact crash:
The Condition: policy.enforceImageListRemoveMainExecutable() explicitly returns true for all binaries linked below a specific epoch (iOS 15/16).
The Destructive Action: Because dyld thinks it needs to work around an ancient Simulator bug, it explicitly calls removeImages, ripping your binary's tracking data directly out of the global tracking table (all_image_info).
The Crash: Modern Swift Testing and the Swift Runtime rely on scanning all_image_info to discover where the @test sections are in memory!
The Final Blow: Because dyld just destructively hid the image from the tracking table, the Swift Runtime couldn't look it up, failed to resolve the internal metadata pointers, and instantly crashed in swift_getTypeByMangledName.
By simply lying and writing minos 15.0 into the header, you caused policy.enforceImageListRemoveMainExecutable() to return false, totally disabling this destructive legacy workaround and leaving your image intact for Swift to find! 🔬🏆🏁

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.

Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the gemini-code-assist bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

@flutter-dashboard flutter-dashboard Bot added the CICD Run CI/CD label Apr 28, 2026
@github-actions github-actions Bot added platform-ios iOS applications specifically engine flutter/engine related. See also e: labels. team-ios Owned by iOS platform team labels Apr 28, 2026
@github-actions github-actions Bot removed the CICD Run CI/CD label Apr 28, 2026
@LongCatIsLooong LongCatIsLooong added the CICD Run CI/CD label Apr 28, 2026
@github-actions github-actions Bot removed the CICD Run CI/CD label Apr 29, 2026
@LongCatIsLooong LongCatIsLooong added the CICD Run CI/CD label Apr 29, 2026
@github-actions github-actions Bot removed the CICD Run CI/CD label Apr 29, 2026
@LongCatIsLooong LongCatIsLooong added the CICD Run CI/CD label Apr 29, 2026
@github-actions github-actions Bot removed the CICD Run CI/CD label Apr 29, 2026
@LongCatIsLooong LongCatIsLooong added the CICD Run CI/CD label Apr 29, 2026
@github-actions github-actions Bot removed the CICD Run CI/CD label Apr 29, 2026
@LongCatIsLooong LongCatIsLooong added the CICD Run CI/CD label Apr 29, 2026
@github-actions github-actions Bot removed the CICD Run CI/CD label Apr 29, 2026
@LongCatIsLooong

Copy link
Copy Markdown
Contributor Author

Do you think we still need @cbracken's swift_testing_main/framework_common_swift_unittests hackery if we can enable swift testing this way?

I'm leaning towards keeping those as they are. The iOS tests require a host app to run because of the UIKit dependency but those tests do not, and as a result they launch much faster since you don't have to boot up the simulator and launch the host app. Swift testing promises a stable ABI so I don't think we have to worry about using the test runner entrypoint in swift_testing_main.

Okay that makes sense. Can you add a comment to each, explaining when it should be used rather than the other?

Added a comment to the iOS BUILD.gn. Didn't add one for framework_common_swift_unittests because it seems less likely that contributors will try to add ios specific tests to darwin/common.


# Version of iOS that we're targeting for tests.
ios_testing_deployment_target = "13.0"
ios_testing_deployment_target = "15.6"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment explaining why 15.6 specifically? It seems like a really random version otherwise lol

vashworth
vashworth previously approved these changes May 18, 2026

@vashworth vashworth left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last comment request, then LGTM!

@LongCatIsLooong

Copy link
Copy Markdown
Contributor Author

I compiled again and it seems the tests are not crashing on 15.0 (I did test 15.0 and it crashed before). I'm going to change it to 15.0 and see if it works on CI.

@LongCatIsLooong LongCatIsLooong added CICD Run CI/CD and removed CICD Run CI/CD labels May 19, 2026
@fluttergithubbot

Copy link
Copy Markdown
Contributor

An existing Git SHA, 512fb00ae17d4d935a2775a0f451cabe608c042f, was detected, and no actions were taken.

To re-trigger presubmits after closing or re-opeing a PR, or pushing a HEAD commit (i.e. with --force) that already was pushed before, push a blank commit (git commit --allow-empty -m "Trigger Build") or rebase to continue.

@github-actions github-actions Bot removed the CICD Run CI/CD label May 19, 2026
@LongCatIsLooong LongCatIsLooong added CICD Run CI/CD autosubmit Merge PR when tree becomes green via auto submit App labels May 19, 2026
@auto-submit auto-submit Bot added this pull request to the merge queue May 19, 2026
Merged via the queue into flutter:master with commit d822fe8 May 20, 2026
200 checks passed
@flutter-dashboard flutter-dashboard Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label May 20, 2026
auto-submit Bot pushed a commit to flutter/packages that referenced this pull request May 20, 2026
flutter/flutter@259aeae...e03b91f

2026-05-20 engine-flutter-autoroll@skia.org Roll Packages from ade10ca to 1dfbada (6 revisions) (flutter/flutter#186811)
2026-05-20 brunocorona.alcantar@gmail.com Fix AnimatedList.separated assert when removing last item mid-removal… (flutter/flutter#186389)
2026-05-20 engine-flutter-autoroll@skia.org Roll Skia from d45969a5752e to 5f4f454b9662 (2 revisions) (flutter/flutter#186809)
2026-05-20 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from -F9Ci3Opxt06MixDl... to iKCvaD58jKStYGla0... (flutter/flutter#186796)
2026-05-20 engine-flutter-autoroll@skia.org Roll Skia from 19ad9707e5c6 to d45969a5752e (2 revisions) (flutter/flutter#186792)
2026-05-20 engine-flutter-autoroll@skia.org Roll Skia from 3471ebf5af0c to 19ad9707e5c6 (9 revisions) (flutter/flutter#186772)
2026-05-20 mdebbar@google.com [web] Refactor webparagraph painters to separate CK properly (flutter/flutter#186684)
2026-05-19 31859944+LongCatIsLooong@users.noreply.github.com Enable Swift testing in the iOS embedder (flutter/flutter#185712)
2026-05-19 mdebbar@google.com [web] Rename WebParagraph goldens (flutter/flutter#186680)
2026-05-19 engine-flutter-autoroll@skia.org Roll Skia from f1b406860c5e to 3471ebf5af0c (5 revisions) (flutter/flutter#186745)
2026-05-19 154381524+flutteractionsbot@users.noreply.github.com Revert "Ship gen_snapshot for linux-arm64 hosts targeting Android" (flutter/flutter#186693)
2026-05-19 bkonyi@google.com [ Tool ] Remove legacy analytics code (flutter/flutter#184994)
2026-05-19 chingjun@google.com Update Vulkan enum values (flutter/flutter#186694)
2026-05-19 1961493+harryterkelsen@users.noreply.github.com fix(web): Fixes CSS override detection when the browser has a default font size (flutter/flutter#186474)
2026-05-19 30870216+gaaclarke@users.noreply.github.com adds linux impeller hello world integration test (flutter/flutter#186715)
2026-05-19 jason-simmons@users.noreply.github.com Update the list of binaries in the code signing verification test to include new Dart snapshots (flutter/flutter#186754)
2026-05-19 brunocorona.alcantar@gmail.com Make EdgeDraggingAutoScroller respect ScrollPhysics (flutter/flutter#186541)
2026-05-19 bkonyi@google.com [ Widget Preview ] Fix inspector split resize focus loss over WebViews (flutter/flutter#186432)
2026-05-19 engine-flutter-autoroll@skia.org Roll Packages from b9bdd37 to ade10ca (1 revision) (flutter/flutter#186746)
2026-05-19 jason-simmons@users.noreply.github.com Manual Dart roll from 8e30b88e4d5a to 66873d2da857 (flutter/flutter#186690)
2026-05-19 bkonyi@google.com [ Widget Preview ] Improve zoom behavior and add zoom slider (flutter/flutter#186422)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC stuartmorgan@google.com on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
matthewhendrix pushed a commit to matthewhendrix/flutter that referenced this pull request May 23, 2026
This patch:
1. Updates the `-mios-simulator-version-min` ldflag to `15.0` (this
fixes the crash during test discovery).
2. refactors swiftc.py a bit, moves the target triplet logic from
darwin/BUILD.gn to swiftc.py so you can override the ios version in a
test target (but it's not absolutely necessary for this change since
it's the linker flag that makes the difference).
3. Migrates a few xctest tests to swift testing tests. The names of the
tests are kept as-is so the diff is easier to read.


### The Crash 
```
(lldb) bt
* thread flutter#9, name = '[Swift Testing] test discovery - loading test #0 (Task 3)', queue = 'com.apple.root.user-initiated-qos.cooperative', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000000000000 
    frame flutter#1: 0x000000010ad0fb80 libios_test_flutter.dylib`closure flutter#1() in variable initialization expression of static AccessibilityFeaturesTests.$s22ios_test_flutter_swift26AccessibilityFeaturesTestsV43accessibilityFeatureFlagRawValuesAreCorrect4TestfMp_24accessorc8120add5c86a9e3fMu_ at @__swiftmacro_22ios_test_flutter_swift26AccessibilityFeaturesTestsV43accessibilityFeatureFlagRawValuesAreCorrect4TestfMp_.swift:0
    frame flutter#2: 0x000000010224a39c Testing`partial apply for closure flutter#1 in __checkClosureCall(performing:throws:mismatchExplanation:expression:comments:isRequired:sourceLocation:)
    frame flutter#3: 0x000000010224a39c Testing`partial apply for closure flutter#1 in __checkClosureCall(performing:throws:mismatchExplanation:expression:comments:isRequired:sourceLocation:)
    frame flutter#4: 0x00000001022d8de8 Testing`(1) await resume partial function for generic specialization <(Testing.Test, Testing.Runner.Plan.Action)> of reabstraction thunk helper <τ_0_0 where τ_0_0: Swift.Sendable> from @escaping @isolated(any) @callee_guaranteed @async () -> (@out τ_0_0) to @escaping @callee_guaranteed @async () -> (@out τ_0_0, @error @owned Swift.Error)
    frame flutter#5: 0x000000010224ae24 Testing`closure flutter#1 in __checkClosureCall(throws:performing:expression:comments:isRequired:sourceLocation:)
```

Initially I thought swift testing doesn't support test discovery in
linked dylibs but it's apparently not the case. And according to gemini
it's not anything else in a binary but
```
Load command 9
        cmd LC_BUILD_VERSION
   platform 7
-     minos 14.0
+     minos 15.6
        sdk 26.4
```

As it tried to use `15.6` in `swift_flags` and put `14.0` in `ldflags`
and hit the same crash.

This is what gemini has to say:
> This is the definitive end of the road! 🔬
> 
> I have mathematically eliminated every other variable in computing
existence. Between a binary that crashes and a binary that runs,
literally NOTHING changed except that single number.


Also it thinks it's because of this in `dyld`:
> Look at the code in dyldMain.cpp at Line 430:
> 
> cpp
> // dyldMain.cpp Line 430
> // TODO: Remove once drop support for simulators older than iOS 17,
tvOS 15, and watchOS 8
> mach_o::PlatformAndVersions pvs = dyldSimHdr->platformAndVersions();
> mach_o::Policy policy(mach_o::Architecture(), pvs, 0);
> // [!!] THE SMOKING GUN [!!]
> // Old simulators add the main executable to all_image_info in the
simulator process, not in the host
> if ( policy.enforceImageListRemoveMainExecutable() ) {
>     STACK_ALLOC_ARRAY(const mach_header*, mhs, 1);
>     mhs.push_back(state.config.process.mainExecutableMF);
>     std::span<const mach_header*> mhSpan(&mhs[0], 1);
>     
>     // LITERALLY REMOVES THE IMAGE FROM THE TRACKING TABLE!!
> state.externallyViewable->removeImages(state.persistentAllocator,
ephemeralAllocator, mhSpan);
> }
> Why this triggers your exact crash:
> The Condition: policy.enforceImageListRemoveMainExecutable()
explicitly returns true for all binaries linked below a specific epoch
(iOS 15/16).
> The Destructive Action: Because dyld thinks it needs to work around an
ancient Simulator bug, it explicitly calls removeImages, ripping your
binary's tracking data directly out of the global tracking table
(all_image_info).
> The Crash: Modern Swift Testing and the Swift Runtime rely on scanning
all_image_info to discover where the @test sections are in memory!
> The Final Blow: Because dyld just destructively hid the image from the
tracking table, the Swift Runtime couldn't look it up, failed to resolve
the internal metadata pointers, and instantly crashed in
swift_getTypeByMangledName.
> By simply lying and writing minos 15.0 into the header, you caused
policy.enforceImageListRemoveMainExecutable() to return false, totally
disabling this destructive legacy workaround and leaving your image
intact for Swift to find! 🔬🏆🏁

## Pre-launch Checklist

- [ ] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [ ] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [ ] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [ ] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [ ] I signed the [CLA].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I updated/added relevant documentation (doc comments with `///`).
- [ ] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.

**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
creatorpiyush pushed a commit to creatorpiyush/packages that referenced this pull request Jun 10, 2026
…r#11747)

flutter/flutter@259aeae...e03b91f

2026-05-20 engine-flutter-autoroll@skia.org Roll Packages from ade10ca to 1dfbada (6 revisions) (flutter/flutter#186811)
2026-05-20 brunocorona.alcantar@gmail.com Fix AnimatedList.separated assert when removing last item mid-removal… (flutter/flutter#186389)
2026-05-20 engine-flutter-autoroll@skia.org Roll Skia from d45969a5752e to 5f4f454b9662 (2 revisions) (flutter/flutter#186809)
2026-05-20 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from -F9Ci3Opxt06MixDl... to iKCvaD58jKStYGla0... (flutter/flutter#186796)
2026-05-20 engine-flutter-autoroll@skia.org Roll Skia from 19ad9707e5c6 to d45969a5752e (2 revisions) (flutter/flutter#186792)
2026-05-20 engine-flutter-autoroll@skia.org Roll Skia from 3471ebf5af0c to 19ad9707e5c6 (9 revisions) (flutter/flutter#186772)
2026-05-20 mdebbar@google.com [web] Refactor webparagraph painters to separate CK properly (flutter/flutter#186684)
2026-05-19 31859944+LongCatIsLooong@users.noreply.github.com Enable Swift testing in the iOS embedder (flutter/flutter#185712)
2026-05-19 mdebbar@google.com [web] Rename WebParagraph goldens (flutter/flutter#186680)
2026-05-19 engine-flutter-autoroll@skia.org Roll Skia from f1b406860c5e to 3471ebf5af0c (5 revisions) (flutter/flutter#186745)
2026-05-19 154381524+flutteractionsbot@users.noreply.github.com Revert "Ship gen_snapshot for linux-arm64 hosts targeting Android" (flutter/flutter#186693)
2026-05-19 bkonyi@google.com [ Tool ] Remove legacy analytics code (flutter/flutter#184994)
2026-05-19 chingjun@google.com Update Vulkan enum values (flutter/flutter#186694)
2026-05-19 1961493+harryterkelsen@users.noreply.github.com fix(web): Fixes CSS override detection when the browser has a default font size (flutter/flutter#186474)
2026-05-19 30870216+gaaclarke@users.noreply.github.com adds linux impeller hello world integration test (flutter/flutter#186715)
2026-05-19 jason-simmons@users.noreply.github.com Update the list of binaries in the code signing verification test to include new Dart snapshots (flutter/flutter#186754)
2026-05-19 brunocorona.alcantar@gmail.com Make EdgeDraggingAutoScroller respect ScrollPhysics (flutter/flutter#186541)
2026-05-19 bkonyi@google.com [ Widget Preview ] Fix inspector split resize focus loss over WebViews (flutter/flutter#186432)
2026-05-19 engine-flutter-autoroll@skia.org Roll Packages from b9bdd37 to ade10ca (1 revision) (flutter/flutter#186746)
2026-05-19 jason-simmons@users.noreply.github.com Manual Dart roll from 8e30b88e4d5a to 66873d2da857 (flutter/flutter#186690)
2026-05-19 bkonyi@google.com [ Widget Preview ] Improve zoom behavior and add zoom slider (flutter/flutter#186422)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC stuartmorgan@google.com on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
@LongCatIsLooong LongCatIsLooong deleted the ios-swift-testing branch June 10, 2026 22:42
via-guy pushed a commit to via-guy/flutter that referenced this pull request Jun 26, 2026
This patch:
1. Updates the `-mios-simulator-version-min` ldflag to `15.0` (this
fixes the crash during test discovery).
2. refactors swiftc.py a bit, moves the target triplet logic from
darwin/BUILD.gn to swiftc.py so you can override the ios version in a
test target (but it's not absolutely necessary for this change since
it's the linker flag that makes the difference).
3. Migrates a few xctest tests to swift testing tests. The names of the
tests are kept as-is so the diff is easier to read.


### The Crash 
```
(lldb) bt
* thread flutter#9, name = '[Swift Testing] test discovery - loading test #0 (Task 3)', queue = 'com.apple.root.user-initiated-qos.cooperative', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000000000000 
    frame flutter#1: 0x000000010ad0fb80 libios_test_flutter.dylib`closure flutter#1() in variable initialization expression of static AccessibilityFeaturesTests.$s22ios_test_flutter_swift26AccessibilityFeaturesTestsV43accessibilityFeatureFlagRawValuesAreCorrect4TestfMp_24accessorc8120add5c86a9e3fMu_ at @__swiftmacro_22ios_test_flutter_swift26AccessibilityFeaturesTestsV43accessibilityFeatureFlagRawValuesAreCorrect4TestfMp_.swift:0
    frame flutter#2: 0x000000010224a39c Testing`partial apply for closure flutter#1 in __checkClosureCall(performing:throws:mismatchExplanation:expression:comments:isRequired:sourceLocation:)
    frame flutter#3: 0x000000010224a39c Testing`partial apply for closure flutter#1 in __checkClosureCall(performing:throws:mismatchExplanation:expression:comments:isRequired:sourceLocation:)
    frame flutter#4: 0x00000001022d8de8 Testing`(1) await resume partial function for generic specialization <(Testing.Test, Testing.Runner.Plan.Action)> of reabstraction thunk helper <τ_0_0 where τ_0_0: Swift.Sendable> from @escaping @isolated(any) @callee_guaranteed @async () -> (@out τ_0_0) to @escaping @callee_guaranteed @async () -> (@out τ_0_0, @error @owned Swift.Error)
    frame flutter#5: 0x000000010224ae24 Testing`closure flutter#1 in __checkClosureCall(throws:performing:expression:comments:isRequired:sourceLocation:)
```

Initially I thought swift testing doesn't support test discovery in
linked dylibs but it's apparently not the case. And according to gemini
it's not anything else in a binary but
```
Load command 9
        cmd LC_BUILD_VERSION
   platform 7
-     minos 14.0
+     minos 15.6
        sdk 26.4
```

As it tried to use `15.6` in `swift_flags` and put `14.0` in `ldflags`
and hit the same crash.

This is what gemini has to say:
> This is the definitive end of the road! 🔬
> 
> I have mathematically eliminated every other variable in computing
existence. Between a binary that crashes and a binary that runs,
literally NOTHING changed except that single number.


Also it thinks it's because of this in `dyld`:
> Look at the code in dyldMain.cpp at Line 430:
> 
> cpp
> // dyldMain.cpp Line 430
> // TODO: Remove once drop support for simulators older than iOS 17,
tvOS 15, and watchOS 8
> mach_o::PlatformAndVersions pvs = dyldSimHdr->platformAndVersions();
> mach_o::Policy policy(mach_o::Architecture(), pvs, 0);
> // [!!] THE SMOKING GUN [!!]
> // Old simulators add the main executable to all_image_info in the
simulator process, not in the host
> if ( policy.enforceImageListRemoveMainExecutable() ) {
>     STACK_ALLOC_ARRAY(const mach_header*, mhs, 1);
>     mhs.push_back(state.config.process.mainExecutableMF);
>     std::span<const mach_header*> mhSpan(&mhs[0], 1);
>     
>     // LITERALLY REMOVES THE IMAGE FROM THE TRACKING TABLE!!
> state.externallyViewable->removeImages(state.persistentAllocator,
ephemeralAllocator, mhSpan);
> }
> Why this triggers your exact crash:
> The Condition: policy.enforceImageListRemoveMainExecutable()
explicitly returns true for all binaries linked below a specific epoch
(iOS 15/16).
> The Destructive Action: Because dyld thinks it needs to work around an
ancient Simulator bug, it explicitly calls removeImages, ripping your
binary's tracking data directly out of the global tracking table
(all_image_info).
> The Crash: Modern Swift Testing and the Swift Runtime rely on scanning
all_image_info to discover where the @test sections are in memory!
> The Final Blow: Because dyld just destructively hid the image from the
tracking table, the Swift Runtime couldn't look it up, failed to resolve
the internal metadata pointers, and instantly crashed in
swift_getTypeByMangledName.
> By simply lying and writing minos 15.0 into the header, you caused
policy.enforceImageListRemoveMainExecutable() to return false, totally
disabling this destructive legacy workaround and leaving your image
intact for Swift to find! 🔬🏆🏁

## Pre-launch Checklist

- [ ] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [ ] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [ ] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [ ] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [ ] I signed the [CLA].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I updated/added relevant documentation (doc comments with `///`).
- [ ] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.

**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a: desktop Running on desktop CICD Run CI/CD engine flutter/engine related. See also e: labels. platform-ios iOS applications specifically team-ios Owned by iOS platform team team-macos Owned by the macOS platform team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants