Skip to content

Flaky test bot: Add more debugging and a large regression test. #4482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
auto-submit merged 2 commits into from
Apr 11, 2025
Merged

Flaky test bot: Add more debugging and a large regression test. #4482

auto-submit merged 2 commits into from
Apr 11, 2025

Conversation

@matanlurey
Copy link
Contributor

@matanlurey matanlurey commented Apr 11, 2025

Further work towards debugging why flutter/flutter#166758 occurs in prod.

Locally, with a regression test, it does mark Windows plugin_test as non-flaky, but in prod:

> DEBUG 2025-04-09T09:20:07.673Z Skipping Linux analyzer_benchmark, could not find matching builder line
> DEBUG 2025-04-09T09:20:07.831Z Skipping Linux coverage, could not find matching builder line
> DEBUG 2025-04-09T09:20:07.833Z Skipping Linux_android_emu_unstable android_defines_test, an existing PR is open: https://github.com/flutter/flutter/pull/165961
> DEBUG 2025-04-09T09:20:07.989Z Skipping Linux_pixel_7pro android_semantics_integration_test, could not find matching builder line
> DEBUG 2025-04-09T09:20:08.185Z Skipping Linux_mokey hybrid_android_views_integration_test, could not find matching builder line
> DEBUG 2025-04-09T09:20:08.407Z Skipping Linux_mokey new_gallery__crane_perf, could not find matching builder line
> DEBUG 2025-04-09T09:20:08.415Z Skipping Staging_build_linux analyze, ignore_flakiness specified
> DEBUG 2025-04-09T09:20:08.610Z Skipping Mac_mokey run_debug_test_android, could not find matching builder line
> DEBUG 2025-04-09T09:20:08.792Z Skipping Mac_mokey run_release_test, could not find matching builder line
> DEBUG 2025-04-09T09:20:08.793Z Skipping Mac_arm64_ios imitation_game_flutter, an existing PR is open: https://github.com/flutter/flutter/pull/163113
> DEBUG 2025-04-09T09:20:08.793Z Skipping Mac_arm64_ios imitation_game_swiftui, an existing PR is open: https://github.com/flutter/flutter/pull/163114
> DEBUG 2025-04-09T09:20:09.008Z Skipping Windows plugin_test, could not find matching builder line
> DEBUG 2025-04-09T09:20:09.219Z Skipping Windows_arm64 plugin_test_windows, could not find matching builder line
> DEBUG 2025-04-09T09:20:09.219Z Skipping Linux flutter_packaging, could not find matching builder line
> DEBUG 2025-04-09T09:20:09.221Z Skipping Windows flutter_packaging, could not find matching builder line

I am guessing it's some sort of state error? The extra debugging lines will help.

@matanlurey matanlurey requested a review from jtmcdole April 11, 2025 21:41
@jtmcdole jtmcdole changed the title Flay test bot: Add more debugging and a large regression test. Flaky test bot: Add more debugging and a large regression test. Apr 11, 2025
jtmcdole
jtmcdole approved these changes Apr 11, 2025
View reviewed changes
Copy link
Member

@jtmcdole jtmcdole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

printer2

@matanlurey matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App. label Apr 11, 2025
@auto-submit auto-submit bot merged commit 36b11d9 into flutter:main Apr 11, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jtmcdole jtmcdole jtmcdole approved these changes

Assignees

No one assigned

Labels

autosubmit Merge PR when tree becomes green via auto submit App.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@matanlurey @jtmcdole