Skip to content

Made sure no test() declaration happens asynchronously in fragment shader tests#182306

Merged
gaaclarke merged 1 commit into
flutter:masterfrom
gaaclarke:fix-async-tests
Feb 12, 2026
Merged

Made sure no test() declaration happens asynchronously in fragment shader tests#182306
gaaclarke merged 1 commit into
flutter:masterfrom
gaaclarke:fix-async-tests

Conversation

@gaaclarke

Copy link
Copy Markdown
Member

fixes #182305

test exempt: is a test

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the gemini-code-assist bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

@github-actions github-actions Bot added the engine flutter/engine related. See also e: labels. label Feb 12, 2026
@gaaclarke gaaclarke changed the title Made sure no test() declaration happens asynchronously in fragment shader tests. Made sure no test() declaration happens asynchronously in fragment shader tests Feb 12, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors a set of fragment shader tests to address an issue where test() declarations were made asynchronously. The change introduces a group() and moves the asynchronous initialization of ImageComparer into a setUpAll() block. This ensures that all test() declarations are made synchronously, which is the correct pattern for Dart tests.

@b-luk

b-luk commented Feb 12, 2026

Copy link
Copy Markdown
Contributor

Can you explain how this change fixes the issue?

Is ImageComparer.create() not supposed to be called in the top level main() function, and must be within a setUp/setUpAll?

Or is it not supposed to be called/awaited after earlier test() calls have already run? So it was fine when it was called as the first line in main(), but caused an issue when it was moved below other test() calls?

Other tests call/await ImageComparer.create() at the top of main():

void main() async {
final ImageComparer comparer = await ImageComparer.create();

void main() async {
final ImageComparer comparer = await ImageComparer.create();

void main() async {
final ImageComparer comparer = await ImageComparer.create();

I think we should make sure this test is consistent with all these other test files.

  1. If ImageComparer.create() needs to be within setUp/setUpAll, we should change all these test files too.
  2. Or if things are fine as long as ImageComparer.create() is above all the test() calls, we can have fragment_shader.test do that again to match the other files.

To me it does seem a bit weird to have this setup code outside of setUp/setUpAll. So if both options are valid, maybe option 1 should still be preferred?

@gaaclarke

Copy link
Copy Markdown
Member Author

@b-luk

Can you explain how this change fixes the issue?

Sure. Check out the error message linked in the issue. What it is saying is you can't call the function test() once tests have started to run. In this case we had something like this:

void main() async {
  test("foo", foo());
  final ImageComparer comparer = await ImageComparer.create();
  test("bar", bar(comparer));
}

This generates a race condition where depending on how long ImageComparer.create() takes, the test "foo" may have started executing before the test "bar" is declared. This change makes sure that all test() calls happen synchronously by making comparer a late bound variable that is created by the setUpAll hook.

Is ImageComparer.create() not supposed to be called in the top level main() function, and must be within a setUp/setUpAll?

Any await call between test() calls is problematic. So you could be safe if all of your awaits happen at the very beginning of main.

Or is it not supposed to be called/awaited after earlier test() calls have already run? So it was fine when it was called as the first line in main(), but caused an issue when it was moved below other test() calls?

Exactly.

@b-luk

b-luk commented Feb 12, 2026

Copy link
Copy Markdown
Contributor

Approved to not block. But I think we should follow-up to change all the other test files to put ImageComparer.create() inside setUp/setUpAll too. For consistency, and also to help avoid someone in the future accidentally moving the test calls around and ending up with await ImageComparer.create() after test() calls like what happened here.

@gaaclarke

Copy link
Copy Markdown
Member Author

This is a nasty gotcha, it would be nice to have a lint for this.

@gaaclarke gaaclarke added the autosubmit Merge PR when tree becomes green via auto submit App label Feb 12, 2026
@b-luk

b-luk commented Feb 12, 2026

Copy link
Copy Markdown
Contributor

This is a nasty gotcha, it would be nice to have a lint for this.

Filed dart-lang/test#2593

@gaaclarke gaaclarke added autosubmit Merge PR when tree becomes green via auto submit App and removed autosubmit Merge PR when tree becomes green via auto submit App labels Feb 12, 2026
@auto-submit auto-submit Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Feb 12, 2026
@auto-submit

auto-submit Bot commented Feb 12, 2026

Copy link
Copy Markdown
Contributor

autosubmit label was removed for flutter/flutter/182306, because - The status or check suite Google testing has failed. Please fix the issues identified (or deflake) before re-applying this label.

@gaaclarke gaaclarke added this pull request to the merge queue Feb 12, 2026
Merged via the queue into flutter:master with commit fd817c7 Feb 12, 2026
182 checks passed
@gaaclarke gaaclarke deleted the fix-async-tests branch February 12, 2026 21:21
@gaaclarke

Copy link
Copy Markdown
Member Author

this should improve if not completely address #182261

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 15, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 15, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 15, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 16, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 16, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 16, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2026
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2026
rickhohler pushed a commit to rickhohler/flutter that referenced this pull request Feb 19, 2026
…ader tests (flutter#182306)

fixes flutter#182305

test exempt: is a test

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
mboetger pushed a commit to mboetger/flutter that referenced this pull request Mar 26, 2026
…ader tests (flutter#182306)

fixes flutter#182305

test exempt: is a test

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine flutter/engine related. See also e: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Linux linux_host_engine_test flaking on fragment_shader_test.dart

3 participants