[flutter_tools] Fix flakiness in widget_preview_detection_test#187938
Merged
auto-submit[bot] merged 5 commits intoJun 15, 2026
Conversation
The integration tests in `widget_preview_detection_test.dart` were flaking on slow CI environments because the test was querying DTD for previews immediately after a file modification, without waiting for the background Analysis Server to finish re-analyzing the change. This patch introduces a robust `waitForPreviews` polling helper in the integration test suite. This helper repeatedly queries DTD for previews until a user-specified predicate is met (or a timeout occurs), and gracefully handles temporary DTD or JSON RPC errors during the polling. All tests in `widget_preview_detection_test.dart` have been updated to use this helper, ensuring they wait for the expected state (e.g. preview added, removed, or updated) before asserting. Fixes flutter#187514
Contributor
There was a problem hiding this comment.
Code Review
This pull request refactors the widget preview integration tests to use a new polling helper function, waitForPreviews, which waits for a specific preview condition to be met. Feedback on these changes suggests catching Exception instead of Object in the polling loop to avoid swallowing programming errors, and adding documentation comments to the new public helper function to comply with the repository style guide.
jtmcdole
requested changes
Jun 12, 2026
| if (previews.previews.isNotEmpty) { | ||
| throw StateError('Preview was still detected after deletion!'); | ||
| } | ||
| await waitForPreviews(dtdConnection, (FlutterWidgetPreviews p) => p.previews.isEmpty); |
Member
There was a problem hiding this comment.
Does this one just instantly succeed because the default is empty? Meaning on a slow bot, you could eventually get a preview, but you check ahead of time and just bail out.
i.e. should there be a minimum timeout before bailing?
Contributor
Author
There was a problem hiding this comment.
We're safe from premature success here because of how the test is synchronized.
Before we call waitForPreviews to check for the empty state, we await the reload trigger:
await deleteReloadCompleter.future.timeout(...);This completer only completes when the tool prints 'Triggering reload...' to stdout. The tool only does this after it has detected the deletion, waited for the Analysis Server to finish re-analyzing, and successfully retrieved the updated (empty) previews from DTD.
So by the time the test resumes and calls waitForPreviews(..., isEmpty), the entire deletion and analysis cycle has already completed, and DTD is guaranteed to be in its final empty state. If the bot is slow, the waiting happens at the deleteReloadCompleter line (which has a 60-second timeout), not in waitForPreviews.
| return previews; | ||
| } | ||
| } on Object { | ||
| // Ignore DTD errors or JSON parsing errors during polling, as they might be temporary |
Member
There was a problem hiding this comment.
Did you hit JSON parsing errors? Its free to log these right so we could maybe later figure out if something is happening?
Contributor
Author
There was a problem hiding this comment.
It's more to handle transient failures that we expect to clear. I've added some logging, but the test should eventually fail if the failures don't clear.
…rs, and add explanatory comments
jtmcdole
approved these changes
Jun 15, 2026
This was referenced Jun 16, 2026
auto-submit Bot
pushed a commit
to flutter/packages
that referenced
this pull request
Jun 16, 2026
flutter/flutter@5827d5f...3a0420c 2026-06-16 Rusino@users.noreply.github.com Implement font fallback (flutter/flutter#187520) 2026-06-16 amhurtado@protonmail.com Add FlatBuffers Verifier checks to Impeller asset loading (flutter/flutter#187878) 2026-06-16 engine-flutter-autoroll@skia.org Roll Packages from aa964a3 to 8286d39 (1 revision) (flutter/flutter#188067) 2026-06-16 engine-flutter-autoroll@skia.org Roll Skia from 9c2b83788409 to d7196b0b4939 (1 revision) (flutter/flutter#188066) 2026-06-16 engine-flutter-autoroll@skia.org Roll Skia from ef17057bb776 to 9c2b83788409 (1 revision) (flutter/flutter#188061) 2026-06-16 engine-flutter-autoroll@skia.org Roll Skia from 500025456bb5 to ef17057bb776 (1 revision) (flutter/flutter#188058) 2026-06-16 engine-flutter-autoroll@skia.org Roll Skia from cb1035ff14bf to 500025456bb5 (5 revisions) (flutter/flutter#188057) 2026-06-16 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from TbB86Po_HDe1dvXvT... to VeLhhlDcod09NR4Hb... (flutter/flutter#188055) 2026-06-16 engine-flutter-autoroll@skia.org Roll Skia from 70acf6a5e7c9 to cb1035ff14bf (3 revisions) (flutter/flutter#188054) 2026-06-16 41930132+hellohuanlin@users.noreply.github.com [pv]skip non-tappable web view workaround on ios 26.4 (flutter/flutter#185424) 2026-06-16 mdebbar@google.com [web] RenderParagraph needs paint after a DPR change (flutter/flutter#186968) 2026-06-16 30870216+gaaclarke@users.noreply.github.com Adds gamma correction to windows text. (flutter/flutter#187871) 2026-06-15 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Add a platform view test to android_hardware_smoke_test (#187913)" (flutter/flutter#188051) 2026-06-15 awolff@google.com Add a platform view test to android_hardware_smoke_test (flutter/flutter#187913) 2026-06-15 codefu@google.com feat: linux_analyze in a workflow (flutter/flutter#187889) 2026-06-15 mdebbar@google.com [web] Changes to WebParagraph configuration (flutter/flutter#187188) 2026-06-15 matt.boetger@gmail.com Fail gracefully on Android AVD lock errors during startup (flutter/flutter#187200) 2026-06-15 bkonyi@google.com [flutter_tools] Fix flakiness in widget_preview_detection_test (flutter/flutter#187938) 2026-06-15 jason-simmons@users.noreply.github.com Exclude fuchsia-sdk/sdk/.build-id from the builder cache archive (flutter/flutter#187826) 2026-06-15 engine-flutter-autoroll@skia.org Roll Skia from c8d9f80f13e4 to 70acf6a5e7c9 (4 revisions) (flutter/flutter#188020) 2026-06-15 engine-flutter-autoroll@skia.org Roll Packages from b78ad83 to aa964a3 (7 revisions) (flutter/flutter#188021) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC bmparr@google.com,stuartmorgan@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
via-guy
pushed a commit
to via-guy/flutter
that referenced
this pull request
Jun 26, 2026
…er#187938) ### Description Fixes flakiness in `widget_preview_detection_test.dart` by introducing a robust polling mechanism when querying DTD for previews. Previously, the integration tests were flaking on slow CI environments because they queried DTD immediately after a file modification, without waiting for the background Dart Analysis Server to finish re-analyzing the changes. This led to stale (empty) preview results and assertion failures like `Bad state: Preview was lost after modification!`. ### Changes * **Introduced `waitForPreviews` Helper**: Added a robust polling helper in `widget_preview_test_helpers.dart` that repeatedly queries DTD for previews until a user-specified predicate is met (or a 10-second timeout occurs). * **Error Resilience**: The helper gracefully ignores all `Object`s (including DTD connection issues, RPC errors, or temporary parsing errors) during the polling loop, as these are common transient states while the Analysis Server is re-analyzing. * **Test Refactoring**: Updated all four integration tests in `widget_preview_detection_test.dart` to use `waitForPreviews` with appropriate predicates (waiting for previews to be non-empty, empty, or containing specific updated content). * **Documentation**: Added comprehensive Dartdoc comments to the new public helper. ### Related Issues * Fixes flutter#187514
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes flakiness in
widget_preview_detection_test.dartby introducing a robust polling mechanism when querying DTD for previews.Previously, the integration tests were flaking on slow CI environments because they queried DTD immediately after a file modification, without waiting for the background Dart Analysis Server to finish re-analyzing the changes. This led to stale (empty) preview results and assertion failures like
Bad state: Preview was lost after modification!.Changes
waitForPreviewsHelper: Added a robust polling helper inwidget_preview_test_helpers.dartthat repeatedly queries DTD for previews until a user-specified predicate is met (or a 10-second timeout occurs).Objects (including DTD connection issues, RPC errors, or temporary parsing errors) during the polling loop, as these are common transient states while the Analysis Server is re-analyzing.widget_preview_detection_test.dartto usewaitForPreviewswith appropriate predicates (waiting for previews to be non-empty, empty, or containing specific updated content).Related Issues