[flutter_tools] Track asset transformer dependencies for hot reload#187947
[flutter_tools] Track asset transformer dependencies for hot reload#187947bkonyi wants to merge 9 commits into
Conversation
Allow asset transformers to register additional input dependencies for hot reload by passing a `--depfile` flag to the transformer subprocess and parsing the resulting Ninja-format depfile. Key changes: - `AssetTransformer` now passes `--depfile` and parses it using `DepfileService`, returning dependencies in a new `AssetTransformationResult`. - `DevelopmentAssetTransformer` tracks these dependencies and provides them to `DevFS`. - `DevFS.updateBundle` does not skip transformed assets on startup, running the transformer to collect dependencies but optimizing by not uploading them to the device unless requested (e.g., for Web). - `DevFS.updateBundle` checks if any tracked dependencies are invalidated and triggers re-transformation. - `DevFS.update` and `WebDevFS.update` append collected dependencies to the watched `sources` set. - `copyAssets` build target collects and adds these dependencies to its inputs to ensure correct rebuilds. - Added unit and integration tests to verify the behavior. Fixes flutter#187051
There was a problem hiding this comment.
Code Review
This pull request updates the asset transformation process to track and monitor dependencies. The AssetTransformer now parses a generated depfile and returns an AssetTransformationResult containing these dependencies. DevFS and WebDevFS use this information to trigger asset re-transformations when dependencies change. Feedback was provided to ensure relative paths in the depfile are correctly resolved relative to the project directory by passing the working directory to depfileService.parse.
| if (depfile.existsSync()) { | ||
| try { | ||
| final depfileService = DepfileService(logger: logger, fileSystem: _fileSystem); | ||
| final Depfile parsedDepfile = depfileService.parse(depfile); |
There was a problem hiding this comment.
When parsing the depfile, any relative paths written by the transformer will be resolved relative to the current working directory of the flutter_tools process (or the depfile's temp directory) rather than the project's working directory where the transformer was executed. To ensure relative paths in the depfile are correctly resolved relative to the project directory, pass _fileSystem.directory(workingDirectory) as the second argument to depfileService.parse.
| final Depfile parsedDepfile = depfileService.parse(depfile); | |
| final Depfile parsedDepfile = depfileService.parse(depfile, _fileSystem.directory(workingDirectory)); |
Update existing tests in `assets_test.dart` to expect the `--depfile` argument in `FakeCommand`s and handle it in `ArgParser`s. Also fix the `Uses processors~/2 to transform assets` test to expect 5 transformed assets instead of 4, which is the correct number of assets (1 primary + 4 variants) created and processed in the test.
Omit obvious local variable types for `processManager` and `environment` in `Uses processors~/2 to transform assets` test.
Update `FakeCommand` and `onRun` in `bundle_builder_test.dart` to support the new `--depfile` argument.
…transformers Redesign `AssetTransformer` to pass the depfile path via `FLUTTER_ASSET_TRANSFORMER_DEPFILE` environment variable instead of `--depfile` command-line argument. This ensures 100% backwards-compatibility with existing or user-defined transformers that do not support the `--depfile` argument and would otherwise crash. Make `invalidatedFiles` optional in `DevFS.updateBundle` to maintain compatibility with existing tests and callers that do not pass it. Update `asset_transformer_test.dart` to verify the new environment variable behavior, using a robust temp directory filtering approach in the depfile parsing test. Fix a latent bug in `assets_test.dart` where the `Uses processors~/2` test expected 4 transformed assets instead of 5 (1 primary + 4 variants). Revert other test files (`devfs_test.dart`, `bundle_builder_test.dart`, and other tests in `assets_test.dart`) to their `origin/main` state since they no longer need to mock the `--depfile` argument.
Update `DepfileService.parse` to accept an optional `baseDirectory` parameter, which is used to resolve relative paths found in the depfile. Pass the `workingDirectory` (project root) as `baseDirectory` to `depfileService.parse` in `AssetTransformer` to ensure relative paths written by transformers are resolved correctly. Add a unit test to `depfile_test.dart` to verify relative path resolution against a base directory.
Rewrite `AssetTransformer` to use an implicit naming convention for depfiles instead of an environment variable. Now, if a transformer wants to register dependencies, it should write its depfile to the output path with `.d` appended (i.e., `<output>.d`). `AssetTransformer` checks for the existence of this file after running the transformer, parses it, and registers the dependencies. This approach: - Is 100% backwards-compatible with existing transformers (no new arguments or env vars are passed). - Eliminates the need for environment variables. - Simplifies the code by removing the second temporary directory. Update `asset_transformer_test.dart` to match this new convention.
In the `Uses processors~/2 to transform assets` test, assert the length of `inputFilePaths` and `outputFilePaths` lists directly instead of calling `toSet()`. `MemoryFileSystem` can reuse deleted temporary directory names sequentially. Since the test runs multiple transformations, some directories are deleted before others start, leading to name reuse (e.g., two runs using `/.tmp_rand0/rand1/`). Calling `toSet()` deduplicated these paths and caused the assertion to fail with a length of 4 instead of 5 in some environments (like CI). Asserting the list length directly is more robust and correctly verifies that all 5 transformations were executed.
[flutter_tools] Track asset transformer dependencies for hot reload
Allow asset transformers to register additional input dependencies for hot reload by passing a
--depfileflag to the transformer subprocess and parsing the resulting Ninja-format depfile.Key changes:
AssetTransformernow passes--depfileand parses it usingDepfileService, returning dependencies in a newAssetTransformationResult.DevelopmentAssetTransformertracks these dependencies and provides them toDevFS.DevFS.updateBundledoes not skip transformed assets on startup, running the transformer to collect dependencies but optimizing by not uploading them to the device unless requested (e.g., for Web).DevFS.updateBundlechecks if any tracked dependencies are invalidated and triggers re-transformation.DevFS.updateandWebDevFS.updateappend collected dependencies to the watchedsourcesset.copyAssetsbuild target collects and adds these dependencies to its inputs to ensure correct rebuilds.Fixes #187051