https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8684718470335776881/+/u/run_test.dart_for_tool_integration_tests_shard_and_subshard_7_10/stdout
This shard runs 60 minutes, 45 minutes in the tests.
https://ci.chromium.org/ui/p/flutter/builders/try/Windows%20tool_integration_tests_8_10/132/overview
This is the fasted shard. 3.5 minutes for the actual tests - and 9 minutes for the total bot. So there's 5.5 minutes of overhead for setting up the bot before running tests.
I think just creating more shards or giving a longer timeout to the shards is the wrong solution here. Instead we should try to get the tests more balanced across the shards.
Some ideas for solving this issue:
- Change tests that run multiple tests in one file to separate test files, so they get distributed more.
- Create a list of long-running tests in _runIntegrationToolTests and forcibly assign them to different shards.
Yeah, I think 1. is the best solution. Realistically, we shouldn't be placing multiple integration tests in the same file, but we haven't really been consistent with that
By @bkonyi
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8684718470335776881/+/u/run_test.dart_for_tool_integration_tests_shard_and_subshard_7_10/stdout
This shard runs 60 minutes, 45 minutes in the tests.
https://ci.chromium.org/ui/p/flutter/builders/try/Windows%20tool_integration_tests_8_10/132/overview
This is the fasted shard. 3.5 minutes for the actual tests - and 9 minutes for the total bot. So there's 5.5 minutes of overhead for setting up the bot before running tests.
I think just creating more shards or giving a longer timeout to the shards is the wrong solution here. Instead we should try to get the tests more balanced across the shards.
Some ideas for solving this issue:
By @bkonyi