Skip to content

Conversation

@stuartmorgan-g
Copy link
Collaborator

@stuartmorgan-g stuartmorgan-g commented Jan 20, 2023

This creates a new test.dart script for developers to run tests locally, separate from the run_tests.dart entrypoint whose no-argument behavior is constrained by our CI requirements since it's what the repo tooling's custom-tests runs.

Most of the logic is extracted to a new shared file, which is used by run_tests.dart and test.dart.

  • run_tests.dart is now solely intended for CI. It distributes the tests among the host platforms so that everything is run, but not all on the same host, and not more than once. It follows the same logic we use for most of our CI in that it runs everything it can on Linux, and only the things that need to be run on Windows and macOS are run there. It also adds validation that we are running every test suite on at least one host unless explicitly opted out, so that we can't accidentally forget to run things in CI (and so that we can easily see in one place what we aren't running in CI).
  • test.dart is for developers to run locally. It still has the arguments for selecting specific tests, but the default behavior is to run every test that can run on that host, so local runs get the maximum possible coverage. This includes integration tests that we aren't currently set up to be able to run in CI. This means developers don't need to remember what is and isn't run locally by default.

Linux custom-tests is moved to the heavy workload machine configuration in the Cirrus configuration since the Android build is OOMing on the light workload configuration.

Fixes flutter/flutter#115393

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
  • I signed the CLA.
  • The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
  • I listed at least one issue that this PR fixes in the description above.
  • I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
  • I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is test-exempt.
  • All existing and new tests are passing.

@stuartmorgan-g stuartmorgan-g added override: no versioning needed Override the check requiring version bumps for most changes override: no changelog needed Override the check requiring CHANGELOG updates for most changes labels Jan 20, 2023
@stuartmorgan-g
Copy link
Collaborator Author

test_suites.dart is code moved from run_tests.dart with essentially no changes; the individual commits let you see the minor adjustments.

@stuartmorgan-g
Copy link
Collaborator Author

Looks like the light-workload machine config doesn't have enough memory for the Android builds; trying it with the heavy-workload config.

Copy link
Contributor

@tarrinneal tarrinneal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems solid to me.

@tarrinneal tarrinneal added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 21, 2023
@auto-submit auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jan 21, 2023
@auto-submit
Copy link
Contributor

auto-submit bot commented Jan 21, 2023

auto label is removed for flutter/packages, pr: 3076, due to - The status or check suite linux-custom_package_tests CHANNEL:stable has failed. Please fix the issues identified (or deflake) before re-applying this label.

@stuartmorgan-g
Copy link
Collaborator Author

The merge here was incorrect, I'll fix it later.

@stuartmorgan-g stuartmorgan-g added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 21, 2023
@auto-submit auto-submit bot merged commit a4d8c39 into flutter:main Jan 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autosubmit Merge PR when tree becomes green via auto submit App override: no changelog needed Override the check requiring CHANGELOG updates for most changes override: no versioning needed Override the check requiring version bumps for most changes p: pigeon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[pigeon] Rework test distribution in CI

2 participants