Add retry flag to flutter_test #125851

fzyzcjy · 2023-05-02T01:22:14Z

I will add tests, polish code, etc, if this change looks generally OK!

Pre-launch Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
I read the Tree Hygiene wiki page, which explains my responsibilities.
I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
I signed the CLA.
I listed at least one issue that this PR fixes in the description above.
I updated/added relevant documentation (doc comments with ///).
I added new tests to check the change I am making, or this PR is test-exempt.
All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

pdblasi-google · 2023-05-02T16:55:19Z

Another good catch! I'm all for this change, but I'll hold off on official approval until you can add tests and do any polish that you're planning.

While you're polishing, could you write up a quick issue to reference in case this needs to be reverted and relanded for any reason?

fzyzcjy · 2023-05-02T23:07:22Z

Sure! I will add tests etc soon

flutter-dashboard · 2023-05-02T23:08:14Z

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact Hixie on the #hackers channel in Chat (don't just cc him here, he won't see it! He's on Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

fzyzcjy · 2023-05-05T07:27:40Z

I have checked how such arguments are tested (e.g. timeout, retry), but it seems that they are not tested. Could you please give a bit of hint about how this should be tested? I guess it has to be on a higher level, since we are not going to write a normal test, but a test to check the test function itself ;)

pdblasi-google · 2023-05-05T18:25:39Z

Hmmm... I'm sad to hear there aren't existing tests for many of those parameters... I'd assumed there'd at least be something verifying that we're passing them through as expected.

For retry specifically, I think you could do something like this:

void main() {
  bool retried = false;
  testWidgets('retry allows test to run multiple times', (WidgetTester tester) async {
    addTeardown(() => retried = true);
    expect(retried, isTrue);
  }, retry: 1);
}

It looks like whatever test you come up for for this would go in the widget_tester_test.dart file. There's one in there for group and a few for testWidgets variant functionality, so it seems like that's where it'd go.

fzyzcjy · 2023-05-05T23:37:26Z

Looks reasonable, I will try that later! (btw retries will report a failure in intellij idea instead of flaky of success)

goderbauer · 2023-05-30T22:09:36Z

(triage): @fzyzcjy Do you still have plans for this PR?

fzyzcjy · 2023-05-30T23:08:33Z

Yes, just not much time to add tests :) I will do it as soon as possible

fzyzcjy · 2023-05-31T02:31:19Z

CI is green now :)

packages/flutter_test/test/widget_tester_test.dart

pdblasi-google

LGTM!

auto-submit · 2023-06-01T18:46:18Z

auto label is removed for flutter/flutter, pr: 125851, due to This PR has not met approval requirements for merging. You are not a member of flutter-hackers and need 1 more review(s) in order to merge this PR.

Merge guidelines: You need at least one approved review if you are already part of flutter-hackers or two member reviews if you are not a flutter-hacker before re-applying the autosubmit label. Reviewers: If you left a comment approving, please use the "approve" review action instead.

goderbauer

LGTM

Roll Flutter from 3db9504 to 0b74153 (108 revisions) flutter/flutter@3db9504...0b74153 2023-06-03 engine-flutter-autoroll@skia.org Roll Flutter Engine from 54293721816a to c838a1b05924 (2 revisions) (flutter/flutter#128161) 2023-06-03 engine-flutter-autoroll@skia.org Roll Flutter Engine from 8769e9ce3a90 to 54293721816a (8 revisions) (flutter/flutter#128158) 2023-06-03 goderbauer@google.com Prefix for dart:ui multiview change (flutter/flutter#128152) 2023-06-02 aam@google.com Roll dds dependency to 2.8.3 for expression evaluation upcoming fixes. (flutter/flutter#128124) 2023-06-02 engine-flutter-autoroll@skia.org Roll Flutter Engine from f3f6a020595d to 8769e9ce3a90 (3 revisions) (flutter/flutter#128149) 2023-06-02 engine-flutter-autoroll@skia.org Roll Flutter Engine from b4250acbf3ca to f3f6a020595d (3 revisions) (flutter/flutter#128147) 2023-06-02 flar@google.com Reland "Remove obsolete drawShadow bounds workaround (#127052)" (flutter/flutter#127231) 2023-06-02 engine-flutter-autoroll@skia.org Roll Flutter Engine from c6e9383f2143 to b4250acbf3ca (8 revisions) (flutter/flutter#128142) 2023-06-02 goderbauer@google.com Pre-migration for dart:ui multi view changes (flutter/flutter#128092) 2023-06-02 goderbauer@google.com Remove LiveTestRenderView (flutter/flutter#127882) 2023-06-02 engine-flutter-autoroll@skia.org Roll Flutter Engine from 02d6fbb68b69 to c6e9383f2143 (15 revisions) (flutter/flutter#128119) 2023-06-02 5236035+fzyzcjy@users.noreply.github.com Tiny remove unnecessary method in text_selection.dart (flutter/flutter#127480) 2023-06-02 vashworth@google.com Workaround for Dart VM timeout (flutter/flutter#127875) 2023-06-02 goderbauer@google.com Make --flutter-repo analyze whole repo (flutter/flutter#127990) 2023-06-02 engine-flutter-autoroll@skia.org Roll Packages from f0513ae to 75085ed (3 revisions) (flutter/flutter#128121) 2023-06-02 50433979+natsuk4ze@users.noreply.github.com [Refactor] fix quote style in template (flutter/flutter#127762) 2023-06-02 engine-flutter-autoroll@skia.org Roll Flutter Engine from 3a453f2ccb77 to 02d6fbb68b69 (14 revisions) (flutter/flutter#128089) 2023-06-02 goderbauer@google.com Fix typos in labels (flutter/flutter#128093) 2023-06-02 zanderso@users.noreply.github.com Revert "Fix issue where DevTools would not be immediately available when using --start-paused" (flutter/flutter#128117) 2023-06-02 goderbauer@google.com Sync Lints (flutter/flutter#127976) 2023-06-02 hans.muller@gmail.com Updated TabBar and ToggleButtons examples (flutter/flutter#128088) 2023-06-02 36861262+QuncCccccc@users.noreply.github.com Text should still be centered when search bar height is less than 48 (flutter/flutter#128068) 2023-06-02 fluttergithubbot@gmail.com Roll pub packages (flutter/flutter#128053) 2023-06-01 tessertaha@gmail.com Add `FilterChip.elevated`, `ChoiceChip.elevated`, & `ActionChip.elevated` variants (flutter/flutter#128049) 2023-06-01 hans.muller@gmail.com Updated custom ListTile examples (flutter/flutter#128071) 2023-06-01 bkonyi@google.com Fix issue where DevTools would not be immediately available when using --start-paused (flutter/flutter#126698) 2023-06-01 katelovett@google.com Add error message when sliver overlap absorber is missing (flutter/flutter#128075) 2023-06-01 hans.muller@gmail.com Updated Menu examples (flutter/flutter#128080) 2023-06-01 5236035+fzyzcjy@users.noreply.github.com Add retry flag to flutter_test (flutter/flutter#125851) 2023-06-01 christopherfujino@gmail.com [flutter_tools] Use process matcher for multidex test (flutter/flutter#127996) 2023-06-01 hans.muller@gmail.com Updated InputDecoratorExamples for M3 (flutter/flutter#128065) 2023-06-01 engine-flutter-autoroll@skia.org Roll Flutter Engine from 7c0a639efd0a to 3a453f2ccb77 (1 revision) (flutter/flutter#128064) 2023-06-01 christopherfujino@gmail.com migrate the package_autoroller from querying by label to title (flutter/flutter#128066) 2023-06-01 hans.muller@gmail.com Revised Floating Action Button examples (flutter/flutter#128058) 2023-06-01 engine-flutter-autoroll@skia.org Roll Flutter Engine from fdd39c421fb9 to 7c0a639efd0a (2 revisions) (flutter/flutter#128056) 2023-06-01 engine-flutter-autoroll@skia.org Roll Flutter Engine from c8e1a8d6fb57 to fdd39c421fb9 (1 revision) (flutter/flutter#128050) 2023-06-01 chillers@google.com [labeler] Remove use of any (flutter/flutter#128011) 2023-06-01 jonahwilliams@google.com [framework] remove reference to closed issue. (flutter/flutter#128007) 2023-06-01 engine-flutter-autoroll@skia.org Roll Packages from 95bb793 to f0513ae (3 revisions) (flutter/flutter#128045) 2023-06-01 vasilich6107@users.noreply.github.com Add fallback font to IconData class (flutter/flutter#127269) 2023-06-01 mdebbar@google.com [web] Assert route names start with / if using PathUrlStrategy (flutter/flutter#127986) 2023-06-01 andrewrkolos@gmail.com [tool] In `flutter doctor -v`, warn when Android Studio version could not be detected. (flutter/flutter#126395) 2023-06-01 engine-flutter-autoroll@skia.org Roll Flutter Engine from 8339c71a963c to c8e1a8d6fb57 (1 revision) (flutter/flutter#128035) 2023-06-01 engine-flutter-autoroll@skia.org Roll Flutter Engine from bc81133892a6 to 8339c71a963c (1 revision) (flutter/flutter#128034) 2023-06-01 engine-flutter-autoroll@skia.org Roll Flutter Engine from e235f47164fa to bc81133892a6 (1 revision) (flutter/flutter#128026) 2023-06-01 72562119+tgucio@users.noreply.github.com Fixes in Cupertino translations (flutter/flutter#127872) ...

Update test_compat.dart

ffb4a11

flutter-dashboard bot added a: tests "flutter test", flutter_test, or one of our tests framework flutter/packages/flutter repository. See also f: labels. labels May 2, 2023

Update widget_tester.dart

83a9872

fzyzcjy marked this pull request as ready for review May 2, 2023 01:24

goderbauer requested a review from pdblasi-google May 2, 2023 10:08

fzyzcjy mentioned this pull request May 2, 2023

Test bot didn't comment on a PR where it should have #123836

Closed

fzyzcjy mentioned this pull request May 2, 2023

Add retry flag to flutter_test #125920

Closed

Merge branch 'master' into patch-33

bd40606

Merge branch 'master' into patch-33

db6f493

github-actions bot removed the framework flutter/packages/flutter repository. See also f: labels. label May 31, 2023

fzyzcjy and others added 3 commits May 31, 2023 09:03

add tests

a50bd01

more tests

1a8e387

Merge branch 'master' into patch-33

ad74df0

pdblasi-google reviewed May 31, 2023

View reviewed changes

packages/flutter_test/test/widget_tester_test.dart Outdated Show resolved Hide resolved

fzyzcjy and others added 3 commits June 1, 2023 07:07

beautify

a6a9f40

Merge remote-tracking branch 'origin/patch-33' into patch-33

8f7b050

Merge branch 'master' into patch-33

b911890

flutter-dashboard bot added the framework flutter/packages/flutter repository. See also f: labels. label Jun 1, 2023

pdblasi-google approved these changes Jun 1, 2023

View reviewed changes

pdblasi-google added the autosubmit Merge PR when tree becomes green via auto submit App label Jun 1, 2023

auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jun 1, 2023

pdblasi-google requested a review from goderbauer June 1, 2023 18:48

goderbauer approved these changes Jun 1, 2023

View reviewed changes

pdblasi-google added the autosubmit Merge PR when tree becomes green via auto submit App label Jun 1, 2023

auto-submit bot merged commit ffe94a2 into flutter:master Jun 1, 2023

HansMuller mentioned this pull request Jun 2, 2023

Flutter Framework Team: Notable Commits #121415

Closed

engine-flutter-autoroll mentioned this pull request Jun 3, 2023

Roll Flutter from 3db9504dcda2 to 0b7415356e07 (108 revisions) flutter/packages#4135

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jun 3, 2023

ffe94a2 Add retry flag to flutter_test (flutter/flutter#125851)

7650ae5

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 16, 2023

ffe94a2 Add retry flag to flutter_test (flutter/flutter#125851)

b06d97b

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

ffe94a2 Add retry flag to flutter_test (flutter/flutter#125851)

16e6996

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

ffe94a2 Add retry flag to flutter_test (flutter/flutter#125851)

3a2387a

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

ffe94a2 Add retry flag to flutter_test (flutter/flutter#125851)

7e08743

curt-weber mentioned this pull request Jul 16, 2025

[Proposal] Add retry to testWidgets #103011

Open

work4montano mentioned this pull request Sep 21, 2025

Implement retry on failure mechanism leancodepl/patrol#2077

Open

Add retry flag to flutter_test #125851

Add retry flag to flutter_test #125851

Uh oh!

Conversation

fzyzcjy commented May 2, 2023 • edited by pdblasi-google Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-launch Checklist

Uh oh!

pdblasi-google commented May 2, 2023

Uh oh!

fzyzcjy commented May 2, 2023

Uh oh!

flutter-dashboard bot commented May 2, 2023

Uh oh!

fzyzcjy commented May 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pdblasi-google commented May 5, 2023

Uh oh!

fzyzcjy commented May 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

goderbauer commented May 30, 2023

Uh oh!

fzyzcjy commented May 30, 2023

Uh oh!

fzyzcjy commented May 31, 2023

Uh oh!

Uh oh!

pdblasi-google left a comment

Choose a reason for hiding this comment

Uh oh!

auto-submit bot commented Jun 1, 2023

Uh oh!

goderbauer left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fzyzcjy commented May 2, 2023 •

edited by pdblasi-google

Loading

fzyzcjy commented May 5, 2023 •

edited

Loading

fzyzcjy commented May 5, 2023 •

edited

Loading