Add `--dry-run` to `dev/bots/test.dart`. #158956

matanlurey · 2024-11-14T21:59:40Z

Example output:

% SHARD=tool_integration_tests SUBSHARD=6_6 dart dev/bots/test.dart --dry-run 
▌13:59:18▐ STARTING ANALYSIS
▌13:59:18▐ --dry-run enabled. Tests will not actually be executed.
▌13:59:18▐ SHARD=tool_integration_tests
▌13:59:18▐ SUBSHARD=6_6
▌13:59:18▐ |> bin/flutter: 
  --version
▌13:59:18▐ |> bin/cache/dart-sdk/bin/dart (packages/flutter_tools): 
  run
  test
  --reporter=expanded
  --file-reporter=json:/var/folders/qw/qw_3qd1x4kz5w975jhdq4k58007b7h/T/metrics_1731621558619861.json
  --test-randomize-ordering-seed=20241114
  -j1
  test/integration.shard/shader_compiler_test.dart
  test/integration.shard/overall_experience_test.dart
  test/integration.shard/expression_evaluation_test.dart
  test/integration.shard/isolated/native_assets_without_cbuild_assemble_test.dart
  test/integration.shard/isolated/native_assets_test.dart
  test/integration.shard/isolated/native_assets_agp_version_test.dart
  test/integration.shard/coverage_collection_test.dart
  test/integration.shard/devtools_uri_test.dart
  test/integration.shard/deprecated_gradle_settings_test.dart
  test/integration.shard/batch_entrypoint_test.dart
  test/integration.shard/bash_entrypoint_test.dart
  test/integration.shard/background_isolate_test.dart

reidbaker · 2024-11-15T15:19:28Z

dev/bots/utils.dart

  }
  print(redLine);
  print('You may find the errors by searching for "╡ERROR #" in the logs.');
+  if (dryRun) {


Reading this code I am not sure why dryrun would want a different error here.

This is unused code, removed.

dev/bots/test.dart

christopherfujino · 2024-11-15T00:55:07Z

dev/bots/utils.dart

+/// the process. This is useful for determing what an invocation of `test.dart`
+/// _might_ due if not invoked with `--dry-mode`, or otherwise determine what
+/// the different test shards and sub-shards are configured as.
+bool dryRun = false;


Is making this a stateful global going to make our unit tests fail on certain shuffle settings?

Fair question. The answer is no, as there are no unit tests where this is true (there is an integration test where this is set to true implicitly with --dry-run) but it's an isolated process.

I modified the API a bit to make it more clear how this will be used.

christopherfujino

Ship it

auto-submit · 2024-11-15T23:10:24Z

auto label is removed for flutter/flutter/158956, due to - The status or check suite Windows framework_tests_misc has failed. Please fix the issues identified (or deflake) before re-applying this label.

auto-submit · 2024-11-15T23:23:47Z

auto label is removed for flutter/flutter/158956, due to - The status or check suite Windows build_android_host_app_with_module_aar has failed. Please fix the issues identified (or deflake) before re-applying this label.

auto-submit · 2024-11-19T01:12:22Z

auto label is removed for flutter/flutter/158956, due to - The status or check suite Windows framework_tests_misc has failed. Please fix the issues identified (or deflake) before re-applying this label.

)

Roll Flutter from 8536b96 to 93d772c (37 revisions) flutter/flutter@8536b96...93d772c 2024-11-21 bkonyi@google.com Added additional logging to `_listCoreDevices` (flutter/flutter#159275) 2024-11-21 engine-flutter-autoroll@skia.org Roll Flutter Engine from 78b87f3fe023 to d1a08064e193 (1 revision) (flutter/flutter#159280) 2024-11-21 jason-simmons@users.noreply.github.com Shut down DevTools and DDS processes if flutter_tools is killed by a signal (flutter/flutter#159238) 2024-11-21 matanlurey@users.noreply.github.com Remove `RepaintBoundary` that is no longer needed. (flutter/flutter#159232) 2024-11-21 matanlurey@users.noreply.github.com Try a speculative fix for Gradle OOMs. (flutter/flutter#159234) 2024-11-21 21270878+elliette@users.noreply.github.com On-device Widget Inspector button exits widget selection (flutter/flutter#158219) 2024-11-21 engine-flutter-autoroll@skia.org Roll Flutter Engine from 523d381893c8 to 78b87f3fe023 (2 revisions) (flutter/flutter#159270) 2024-11-21 jessiewong401@gmail.com Remove `firebase_abstract_method_smoke_test` (flutter/flutter#159145) 2024-11-21 engine-flutter-autoroll@skia.org Roll Packages from e95f6d8 to 913b99e (7 revisions) (flutter/flutter#159268) 2024-11-21 engine-flutter-autoroll@skia.org Roll Flutter Engine from 69c325513a65 to 523d381893c8 (3 revisions) (flutter/flutter#159263) 2024-11-21 jonahwilliams@google.com [flutter_tools] opt iOS/macOS apps out of Metal API validation via migrator, update templates in repo. (flutter/flutter#159228) 2024-11-21 jmccandless@google.com Scribe Android handwriting text input (flutter/flutter#148784) 2024-11-21 matanlurey@users.noreply.github.com Terminate non-detached test devices on `flutter run` completion (flutter/flutter#159170) 2024-11-21 matanlurey@users.noreply.github.com Un-skip tests that use `flutter build apk`. (flutter/flutter#159231) 2024-11-20 engine-flutter-autoroll@skia.org Roll Flutter Engine from 2d32cf3a7971 to 69c325513a65 (1 revision) (flutter/flutter#159229) 2024-11-20 engine-flutter-autoroll@skia.org Roll Flutter Engine from 3828681d1f86 to 2d32cf3a7971 (3 revisions) (flutter/flutter#159226) 2024-11-20 engine-flutter-autoroll@skia.org Roll Flutter Engine from 3245c8976976 to 3828681d1f86 (1 revision) (flutter/flutter#159217) 2024-11-20 matanlurey@users.noreply.github.com Add `--dry-run` to `dev/bots/test.dart`. (flutter/flutter#158956) 2024-11-20 34871572+gmackall@users.noreply.github.com Add docs for setting up Android Studio to auto format Kotlin code (flutter/flutter#159209) 2024-11-20 engine-flutter-autoroll@skia.org Roll Flutter Engine from 80d77505fdde to 3245c8976976 (1 revision) (flutter/flutter#159214) 2024-11-20 stanleycocoa@gmail.com Fix: The enableFeedback property of InkWell cannot be set to a nullabâ�¦ (flutter/flutter#158907) 2024-11-20 engine-flutter-autoroll@skia.org Roll Flutter Engine from 3f19207e820e to 80d77505fdde (1 revision) (flutter/flutter#159210) 2024-11-20 engine-flutter-autoroll@skia.org Roll Packages from fc4adc7 to e95f6d8 (6 revisions) (flutter/flutter#159201) 2024-11-20 kustermann@google.com Remove dependency on [Target] and instead operate on [Architecture] (flutter/flutter#159196) 2024-11-20 63278936+hongeSunCoder@users.noreply.github.com Fix git command in Quality-Assurance.md (flutter/flutter#155146) 2024-11-20 engine-flutter-autoroll@skia.org Roll Flutter Engine from 7eb87547cbc6 to 3f19207e820e (4 revisions) (flutter/flutter#159176) 2024-11-20 matanlurey@users.noreply.github.com Make `runner` non-nullable as it always is. (flutter/flutter#159156) 2024-11-19 tessertaha@gmail.com Update Material 3 `CircularProgressIndicator` for new visual style (flutter/flutter#158104) 2024-11-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from 4ff696b555dc to 7eb87547cbc6 (3 revisions) (flutter/flutter#159168) 2024-11-19 magder@google.com Add platform-android label for all flutter_tools *android* files (flutter/flutter#159166) 2024-11-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from d5820a638885 to 4ff696b555dc (1 revision) (flutter/flutter#159164) 2024-11-19 bernaferrari2@gmail.com Reland Add UI Benchmarks (flutter/flutter#153368) 2024-11-19 mohellebiabdessalem@gmail.com fix lint usage of `task` inside `resolve_dependecies.gradle` file (flutter/flutter#158022) 2024-11-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from cff1e751f853 to d5820a638885 (5 revisions) (flutter/flutter#159155) 2024-11-19 mohellebiabdessalem@gmail.com Removing redundant backticks in `flutter\packages\flutter_tools\gradle\gradle.kts` (flutter/flutter#159051) 2024-11-19 50643541+Mairramer@users.noreply.github.com Fixes initial validation with AutovalidateMode.always on first build (flutter/flutter#156708) 2024-11-19 tessertaha@gmail.com Introduce new Material 3 `Slider` shapes (flutter/flutter#152237) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC louisehsu@google.com,stuartmorgan@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose ...

)

Add --dry-run to dev/bots/test.dart.

dec19a9

matanlurey requested review from christopherfujino and jmagman November 14, 2024 21:59

Remove late.

a4dd953

reidbaker approved these changes Nov 15, 2024

View reviewed changes

christopherfujino reviewed Nov 15, 2024

View reviewed changes

matanlurey added 2 commits November 15, 2024 14:13

Address feedback.

1f1313b

++

5374703

matanlurey requested a review from christopherfujino November 15, 2024 22:15

christopherfujino approved these changes Nov 15, 2024

View reviewed changes

matanlurey added 2 commits November 15, 2024 14:47

++

b7b711e

Merge remote-tracking branch 'upstream/main' into dev-bots-test-dry-run

aea2f1d

matanlurey force-pushed the dev-bots-test-dry-run branch from 169d4bb to aea2f1d Compare November 15, 2024 22:47

matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Nov 15, 2024

auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Nov 15, 2024

matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Nov 15, 2024

auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Nov 15, 2024

matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Nov 19, 2024

auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Nov 19, 2024

matanlurey and others added 3 commits November 19, 2024 08:50

Merge branch 'master' into dev-bots-test-dry-run

dc13217

Merge branch 'master' into dev-bots-test-dry-run

fe4efaa

++

3df5dc0

matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Nov 20, 2024

auto-submit bot added this pull request to the merge queue Nov 20, 2024

Merged via the queue into flutter:master with commit bf36437 Nov 20, 2024
150 checks passed

flutter-dashboard bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Nov 20, 2024

engine-flutter-autoroll mentioned this pull request Nov 21, 2024

Roll Flutter from 8536b96ebb3e to 1578d7d5f34e (28 revisions) flutter/packages#8139

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Nov 21, 2024

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

098f52f

)

engine-flutter-autoroll mentioned this pull request Nov 21, 2024

Roll Flutter from 8536b96ebb3e to 7662e7eae548 (29 revisions) flutter/packages#8140

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Nov 21, 2024

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

bb4e6f8

)

engine-flutter-autoroll mentioned this pull request Nov 21, 2024

Roll Flutter from 8536b96ebb3e to 81d2bf5d7934 (30 revisions) flutter/packages#8141

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Nov 21, 2024

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

92b3b82

)

engine-flutter-autoroll mentioned this pull request Nov 21, 2024

Roll Flutter from 8536b96ebb3e to 4e03220a8691 (31 revisions) flutter/packages#8143

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Nov 21, 2024

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

f8a7efd

)

engine-flutter-autoroll mentioned this pull request Nov 21, 2024

Roll Flutter from 8536b96ebb3e to a051be858576 (32 revisions) flutter/packages#8144

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Nov 21, 2024

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

9c4f6cf

)

engine-flutter-autoroll mentioned this pull request Nov 21, 2024

Roll Flutter from 8536b96ebb3e to d1e2c5fba748 (33 revisions) flutter/packages#8145

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Nov 21, 2024

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

d0bb307

)

engine-flutter-autoroll mentioned this pull request Nov 21, 2024

Roll Flutter from 8536b96ebb3e to 399403ae6a38 (36 revisions) flutter/packages#8146

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Nov 21, 2024

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

cf9f9a7

)

engine-flutter-autoroll mentioned this pull request Nov 21, 2024

Roll Flutter from 8536b96ebb3e to 93d772c5cdd8 (37 revisions) flutter/packages#8147

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Nov 21, 2024

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

445a80c

)

reidbaker mentioned this pull request Dec 13, 2024

Cutout rotation integration test #160255

Closed

11 tasks

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 12, 2025

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

e3836ab

)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 13, 2025

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

3d118be

)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 13, 2025

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

662045f

)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 6, 2025

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

a84cc05

)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 7, 2025

bf36437 Add --dry-run to dev/bots/test.dart. (flutter/flutter#158956

4232167

)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add `--dry-run` to `dev/bots/test.dart`. #158956

Add `--dry-run` to `dev/bots/test.dart`. #158956

Uh oh!

matanlurey commented Nov 14, 2024

Uh oh!

reidbaker Nov 15, 2024

Uh oh!

matanlurey Nov 15, 2024

Uh oh!

Uh oh!

christopherfujino Nov 15, 2024

Uh oh!

matanlurey Nov 15, 2024

Uh oh!

christopherfujino left a comment

Uh oh!

auto-submit bot commented Nov 15, 2024

Uh oh!

auto-submit bot commented Nov 15, 2024

Uh oh!

auto-submit bot commented Nov 19, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add --dry-run to dev/bots/test.dart. #158956

Add --dry-run to dev/bots/test.dart. #158956

Uh oh!

Conversation

matanlurey commented Nov 14, 2024

Uh oh!

reidbaker Nov 15, 2024

Choose a reason for hiding this comment

Uh oh!

matanlurey Nov 15, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

christopherfujino Nov 15, 2024

Choose a reason for hiding this comment

Uh oh!

matanlurey Nov 15, 2024

Choose a reason for hiding this comment

Uh oh!

christopherfujino left a comment

Choose a reason for hiding this comment

Uh oh!

auto-submit bot commented Nov 15, 2024

Uh oh!

auto-submit bot commented Nov 15, 2024

Uh oh!

auto-submit bot commented Nov 19, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add `--dry-run` to `dev/bots/test.dart`. #158956

Add `--dry-run` to `dev/bots/test.dart`. #158956