Help us understand the severity of this issue
What do you need help with
Almost all of the flutter/packages tests rely on some environment variables being set from .ci.yaml:
- Essentially all of them use
CHANNEL to control which channel of Flutter they are run against (example)
- All of the sharded tests (which is a lot of them) pass shard info in
PACKAGE_SHARDING (example)
I was seeing weird persistent time-outs in sharded tests in some PRs and couldn't figure out why, until I was investigating an even stranger failure in the N-2 stable version analyze tests in several PRs. This PR doesn't change anything about any of our tests or packages, and yet was failing. In particular, it was failing packages that should be skipped due to the --skip-if-not-supporting-flutter-version="$CHANNEL" flag. While checking if the CHANNEL value was wrong somehow, I found that instead it's not there at all.
And from there I realized that the timeouts in the platform tests are because they are supposed to be running 1/6 of the tests, but are instead running all of them.
Unfortunately flutter/packages has been in very bad shape for the last week because of issues with FTL capacity, so I'm not sure exactly when this started. This commit 4 days ago is the last time I'm confident that things worked, because it has the env variables.
I'm assuming this is accidental fallout for hardening CI environments.
Help us understand the severity of this issue
What do you need help with
Almost all of the flutter/packages tests rely on some environment variables being set from .ci.yaml:
CHANNELto control which channel of Flutter they are run against (example)PACKAGE_SHARDING(example)I was seeing weird persistent time-outs in sharded tests in some PRs and couldn't figure out why, until I was investigating an even stranger failure in the N-2 stable version analyze tests in several PRs. This PR doesn't change anything about any of our tests or packages, and yet was failing. In particular, it was failing packages that should be skipped due to the
--skip-if-not-supporting-flutter-version="$CHANNEL"flag. While checking if theCHANNELvalue was wrong somehow, I found that instead it's not there at all.And from there I realized that the timeouts in the platform tests are because they are supposed to be running 1/6 of the tests, but are instead running all of them.
Unfortunately flutter/packages has been in very bad shape for the last week because of issues with FTL capacity, so I'm not sure exactly when this started. This commit 4 days ago is the last time I'm confident that things worked, because it has the env variables.
I'm assuming this is accidental fallout for hardening CI environments.