Fix bug in test_golden_comparator, add an e2e test. #174459
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Code Review
This pull request addresses a bug in TestGoldenComparator that caused crashes on golden test failures. The fix involves broadening the exception handling in test_golden_comparator.dart from Exception to Object? to catch any thrown object. Additionally, a new integration test, test_golden_comparator_test.dart, has been added with tests for both successful and failed comparison scenarios to verify the fix. My review includes a suggestion to refactor the new test to reduce code duplication for better maintainability.
packages/flutter_tools/test/integration.shard/test_golden_comparator_test.dart
Outdated
Show resolved
Hide resolved
bkonyi
approved these changes
Aug 26, 2025
| printOnFailure(logger.errorText); | ||
| }); | ||
|
|
||
| // Cannot be in 'setUp' because it implicitly uses [global] state that comes from 'testUsingContext'. |
Contributor
There was a problem hiding this comment.
There's calls to global in this function, so it's hardly implicit 😜
| expect(result, isA<TestGoldenComparisonDone>()); | ||
| }); | ||
|
|
||
| testUsingContext('failed comparison is failed but not crashed', () async { |
Contributor
There was a problem hiding this comment.
Consider leaving a breadcrumb to point to the original issue this regression test was written for.
| print(jsonEncode({'success': success})); | ||
| } on Exception catch (ex) { | ||
| print(jsonEncode({'success': false, 'message': '\$ex'})); | ||
| } on Object? catch (e) { |
Contributor
There was a problem hiding this comment.
Out of curiosity, what's being thrown that wasn't caught by Exception?
Contributor
|
autosubmit label was removed for flutter/flutter/174459, because - The status or check suite Google testing has failed. Please fix the issues identified (or deflake) before re-applying this label. |
Contributor
|
autosubmit label was removed for flutter/flutter/174459, because - The status or check suite Windows plugin_test has failed. Please fix the issues identified (or deflake) before re-applying this label. |
flutteractionsbot
pushed a commit
to flutteractionsbot/flutter
that referenced
this pull request
Aug 26, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
flutteractionsbot
pushed a commit
to flutteractionsbot/flutter
that referenced
this pull request
Aug 26, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
matanlurey
added a commit
to matanlurey/flutter
that referenced
this pull request
Aug 27, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
matanlurey
added a commit
to matanlurey/flutter
that referenced
this pull request
Aug 27, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
auto-submit bot
pushed a commit
to flutter/packages
that referenced
this pull request
Aug 28, 2025
flutter/flutter@c65f01d...da5523a 2025-08-27 dacoharkes@google.com [native assets] Roll dependencies (flutter/flutter#174522) 2025-08-27 engine-flutter-autoroll@skia.org Roll Skia from 8cf2faada2b5 to 7e6da45059c5 (5 revisions) (flutter/flutter#174523) 2025-08-27 43054281+camsim99@users.noreply.github.com [Android] Restrict AOT shared library engine flag to trusted paths (flutter/flutter#173359) 2025-08-27 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from L5zGzsIWIS8N36AFQ... to bHYRvLv2Dg56RWZF2... (flutter/flutter#174518) 2025-08-27 engine-flutter-autoroll@skia.org Roll Packages from 1ef712e to 86fbeec (7 revisions) (flutter/flutter#174521) 2025-08-27 engine-flutter-autoroll@skia.org Roll Skia from 448a0d0e57e3 to 8cf2faada2b5 (1 revision) (flutter/flutter#174510) 2025-08-27 engine-flutter-autoroll@skia.org Roll Skia from 4703976a4dae to 448a0d0e57e3 (3 revisions) (flutter/flutter#174494) 2025-08-27 sokolovskyi.konstantin@gmail.com Fix SliverMainAxisGroup and SliverCrossAxisGroup gestures' local positions. (flutter/flutter#174265) 2025-08-27 robert.ancell@canonical.com Fix lock up when window resized with merged UI and platform threads. (flutter/flutter#172893) 2025-08-27 engine-flutter-autoroll@skia.org Roll Skia from dc2872de506f to 4703976a4dae (1 revision) (flutter/flutter#174489) 2025-08-27 engine-flutter-autoroll@skia.org Roll Dart SDK from db45c0ce46f9 to 89023922f96d (2 revisions) (flutter/flutter#174481) 2025-08-27 32538273+ValentinVignal@users.noreply.github.com Migrate examples and defaults to `WidgetState` (flutter/flutter#174421) 2025-08-27 engine-flutter-autoroll@skia.org Roll Skia from 8e48b9e6d67b to dc2872de506f (7 revisions) (flutter/flutter#174479) 2025-08-27 36861262+QuncCccccc@users.noreply.github.com SnackBar with action no longer auto-dismiss (flutter/flutter#173084) 2025-08-26 engine-flutter-autoroll@skia.org Roll Dart SDK from 9054cd8af73c to db45c0ce46f9 (1 revision) (flutter/flutter#174438) 2025-08-26 engine-flutter-autoroll@skia.org Roll Skia from f941bfab7c09 to 8e48b9e6d67b (4 revisions) (flutter/flutter#174468) 2025-08-26 matanlurey@users.noreply.github.com Fix bug in test_golden_comparator, add an e2e test. (flutter/flutter#174459) 2025-08-26 matt.boetger@gmail.com fix typo in test_profile/README.md (flutter/flutter#174384) 2025-08-26 matanlurey@users.noreply.github.com Remove CP labels on not-merged PRs, and explain why (flutter/flutter#174448) 2025-08-26 1063596+reidbaker@users.noreply.github.com Increase testing coverage and maintainability of android manifest parsing logic (flutter/flutter#174070) 2025-08-26 1961493+harryterkelsen@users.noreply.github.com [web] Add test that pictures are not rasterized when clipped out (flutter/flutter#174452) 2025-08-26 engine-flutter-autoroll@skia.org Roll Skia from 538260c45b4a to f941bfab7c09 (3 revisions) (flutter/flutter#174450) 2025-08-26 30870216+gaaclarke@users.noreply.github.com fixes the vulkan image layout transitions for mipmap generation (flutter/flutter#173884) 2025-08-26 15619084+vashworth@users.noreply.github.com Move flakey iOS tests to bringup (flutter/flutter#174446) 2025-08-26 30870216+gaaclarke@users.noreply.github.com [Impeller] Make sure inline passes always do a clear action. (flutter/flutter#174083) 2025-08-26 engine-flutter-autoroll@skia.org Roll Skia from 21214d63fc40 to 538260c45b4a (2 revisions) (flutter/flutter#174441) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC stuartmorgan@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 28, 2025
mboetger
pushed a commit
to mboetger/flutter
that referenced
this pull request
Sep 18, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
korca0220
pushed a commit
to korca0220/flutter
that referenced
this pull request
Sep 22, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
Jaineel-Mamtora
pushed a commit
to Jaineel-Mamtora/flutter_forked
that referenced
this pull request
Sep 24, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
9 tasks
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Nov 12, 2025
lucaantonelli
pushed a commit
to lucaantonelli/flutter
that referenced
this pull request
Nov 21, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #174267.
The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.