Run all flutter/flutter macOS tests using Xcode 26 and iOS 26 simulator#185651
Conversation
Updates Xcode SDK version from 16c5032a to 17c52 (Xcode 26.2), macOS host from Mac-15.5|Mac-15.7 to Mac-15.7, and simulator runtime from ios-18-2 to ios-26-2. Replaces 18.2 golden images with 26.2 goldens and updates generate_builder_json.dart. device_os remains iOS-18 for devicelab targets (physical devices not yet upgraded).
There was a problem hiding this comment.
Code Review
This pull request updates the CI configuration across multiple files to use macOS 15.7 and a new SDK version (17c52). It also increments the iOS simulator runtime and version from 18.2 to 26.2 within the test runner and updates the corresponding golden image references in the iOS scenario app's Xcode project. Feedback was provided questioning the validity of the version strings 'iOS-26-2' and '26.2', as these appear to be potential typos that could prevent the simulator from launching correctly.
| 'os-runtime', | ||
| help: 'The OS runtime of the iOS simulator device to use.', | ||
| defaultsTo: 'com.apple.CoreSimulator.SimRuntime.iOS-18-2', | ||
| defaultsTo: 'com.apple.CoreSimulator.SimRuntime.iOS-26-2', |
There was a problem hiding this comment.
The iOS simulator runtime version iOS-26-2 appears to be incorrect. Standard Apple versioning for the current cycle (associated with Xcode 16.2) would typically be iOS-18-2. Please verify if this version number is intentional or a typo, as an invalid runtime identifier will cause the tests to fail to launch the simulator.
| 'os-version', | ||
| help: 'The OS version of the iOS simulator device to use.', | ||
| defaultsTo: '18.2', | ||
| defaultsTo: '26.2', |
There was a problem hiding this comment.
The OS version 26.2 appears to be incorrect. If this is intended to match the iOS 18.2 release cycle, it should likely be 18.2. Using a non-existent version number will prevent the test runner from correctly identifying and targeting the simulator.
flutter/flutter@61fca76...81bc3d6 2026-04-28 okorohelijah@google.com Run all flutter/flutter macOS tests using Xcode 26 and iOS 26 simulator (flutter/flutter#185651) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC stuartmorgan@google.com,tarrinneal@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
|
This commit triggered some alerts about changes in multiple performance benchmarks. This link shows the changes attributed to this PR: This link shows a many changes which are attributed to other recent commits. I think these are mis-attributed, and all of these changes caused by this PR: Are these changes expected and/or acceptable? This PR seems to be a change for only the CI config. So maybe the change in the metrics are just showing a difference between the old and new configs. If that's the case, this might not be a real regression that users would see. But some of the changes are very large regressions. A couple examples:
Does this warrant further investigation? cc @gaaclarke |
|
I filed an issue about this: #185762 For tracking purposes let's keep discussion in the issue rather than in this PR. |
…11597) flutter/flutter@61fca76...81bc3d6 2026-04-28 okorohelijah@google.com Run all flutter/flutter macOS tests using Xcode 26 and iOS 26 simulator (flutter/flutter#185651) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC stuartmorgan@google.com,tarrinneal@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Updates Xcode SDK version from 16c5032a to 17c52 (Xcode 26.2), macOS host from Mac-15.5|Mac-15.7 to Mac-15.7, and simulator runtime from ios-18-2 to ios-26-2. Replaces 18.2 golden images with 26.2 goldens and updates generate_builder_json.dart.
device_os remains iOS-18 for devicelab targets (physical devices not yet upgraded).
Fixes #172855
If you had to change anything in the flutter/tests repo, include a link to the migration guide as per the breaking change policy.
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.
If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.
Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the
gemini-code-assistbot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.