Disable Devtools tests #406
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
FYI @kenzieschmoll |
This was referenced Aug 29, 2024
Member
|
IIRC, we discussed the runtime of DevTools tests back when we originally added these to the customer test registry, knowing they would take longer than the recommended 5 minutes to run. What I remember from that discussion (which I think Ian was a part of too?) was that we were fine with the extra runtime of DevTools tests since DevTools is a Flutter-owned product that we are responsible for maintaining and not breaking. I tried to track down an issue or a notes doc somewhere but couldn't find one, so please let me know if this doesn't sound accurate to you. |
Contributor
Author
|
I do remember that. Unfortunately today, we do not have the infrastructure resources to support a task running so long. |
kenzieschmoll
approved these changes
Aug 30, 2024
9 tasks
auto-submit bot
pushed a commit
to flutter/flutter
that referenced
this pull request
Sep 3, 2024
Reverts increased timeouts on customer testing from #154339, also removes bringup state from #154293. This was related to #154251, which will be fixed by flutter/tests#406 (so we should land that first).
auto-submit bot
pushed a commit
to flutter/flutter
that referenced
this pull request
Sep 3, 2024
Reverts #154356, and updates logging so that the test suite's name and runtime is always logged, not just on failure. The issue is resolved in flutter/tests#406 Related to #154251
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Devtools test suite is taking over 10 minutes to run on the linux platform.
Registered tests in flutter/tests are expected to complete in less than 5 minutes, and ideally in a matter of seconds.
This is causing the Linux_customer testing shard to timeout repeatedly in flutter/flutter.
Fixes flutter/flutter#154251
Verification through verbose logging enabled in https://ci.chromium.org/ui/p/flutter/builders/staging/Mac%20integration_ui_test_test_macos/2105/overview
Further, this was reenabled last week in, when the flakiness started #402
It had been disabled two weeks ago in #397, which showed the same issue of runtime causing flakes leading up to it being disabled.
See all builds before it was put on bringup here: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20customer_testing
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.