Add initial compile tests #109177
Add initial compile tests #109177
Conversation
|
|
||
| final Map<String, dynamic> compileRelease = await _compileApp(reportPackageContentSizes: reportPackageContentSizes); | ||
| final Map<String, dynamic> compileDebug = await _compileDebug( | ||
| // "initial" compile required downloading and creating the `android/.gradle` directory while "full" |
There was a problem hiding this comment.
Factoring out the invocation that we know triggers a download into its own separate benchmark is a good start, but I think the end state we probably want to shoot for is factoring the download out into a step that the recipe can tag as an infra step rather than a test step.
There was a problem hiding this comment.
A question is, do we want to track the gradle downloads as part of the build or do we consider it purely setup. The existing tests have the release_full_compile_millis shouldering the cost and is technically incorrect compared to the other benchmarks that benefit from not needing to download gradle.
Users will hit this when they try to initially build. Do we thing this extra cost is part of the build cost and thus belonging in the xxx_full_compile_millis bench or are we saying one-time initial costs are not a metric we want to track and thus want to separate it out into setup.
There was a problem hiding this comment.
Hitting the network is not useful to include in a benchmark because we don't have any control over it. Said differently, what would we do if it suddenly became much slower to hit the network?
| final Map<String, dynamic> compileDebug = await _compileDebug( | ||
| // "initial" compile required downloading and creating the `android/.gradle` directory while "full" | ||
| // compiles only run `flutter clean` between runs.gi | ||
| final Map<String, dynamic> compileInitialRelease = await _compileApp(deleteGradleCache: true); |
There was a problem hiding this comment.
Is there any way that the initial run can be taken only as far as needed to populate the cache?
There was a problem hiding this comment.
I believe that only the build command triggers the gradle steps that populates the .gradle directory. We could make a flag or command to just populate things like this, but it would likely not be of much use outside of our tests.
|
This was reverted: #110550 |
| final Directory gradleCacheDir = Directory('$testDirectory/android/.gradle'); | ||
| gradleCacheDir.deleteSync(recursive: true); |
There was a problem hiding this comment.
You can use rmTree when you reland this.
flutter/dev/devicelab/lib/framework/utils.dart
Lines 98 to 114 in 7122a00
In discussion for #84868, We are seeing differences when running flutter build depending on if the contents of
android/.gradledirectory is populated or not. This is because gradle will download dependencies and generate the .gradle file on first run of the app.This PR splits the inital compile time into
initialandfull, where initial implies building from a fully untouched never-before-compiled project, whilefullimplies building after just runningflutter clean.This change adds about ~30s of build time to the
hello_world_android__compiletest task