Skip to content

[flutter_tools] Track asset transformer dependencies for hot reload#187947

Open
bkonyi wants to merge 9 commits into
flutter:masterfrom
bkonyi:track-asset-transformer-dependencies
Open

[flutter_tools] Track asset transformer dependencies for hot reload#187947
bkonyi wants to merge 9 commits into
flutter:masterfrom
bkonyi:track-asset-transformer-dependencies

Conversation

@bkonyi

@bkonyi bkonyi commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

[flutter_tools] Track asset transformer dependencies for hot reload

Allow asset transformers to register additional input dependencies for hot reload by passing a --depfile flag to the transformer subprocess and parsing the resulting Ninja-format depfile.

Key changes:

  • AssetTransformer now passes --depfile and parses it using DepfileService, returning dependencies in a new AssetTransformationResult.
  • DevelopmentAssetTransformer tracks these dependencies and provides them to DevFS.
  • DevFS.updateBundle does not skip transformed assets on startup, running the transformer to collect dependencies but optimizing by not uploading them to the device unless requested (e.g., for Web).
  • DevFS.updateBundle checks if any tracked dependencies are invalidated and triggers re-transformation.
  • DevFS.update and WebDevFS.update append collected dependencies to the watched sources set.
  • copyAssets build target collects and adds these dependencies to its inputs to ensure correct rebuilds.
  • Added unit and integration tests to verify the behavior.

Fixes #187051

Allow asset transformers to register additional input dependencies for hot reload by passing a `--depfile` flag to the transformer subprocess and parsing the resulting Ninja-format depfile.

Key changes:
- `AssetTransformer` now passes `--depfile` and parses it using `DepfileService`, returning dependencies in a new `AssetTransformationResult`.
- `DevelopmentAssetTransformer` tracks these dependencies and provides them to `DevFS`.
- `DevFS.updateBundle` does not skip transformed assets on startup, running the transformer to collect dependencies but optimizing by not uploading them to the device unless requested (e.g., for Web).
- `DevFS.updateBundle` checks if any tracked dependencies are invalidated and triggers re-transformation.
- `DevFS.update` and `WebDevFS.update` append collected dependencies to the watched `sources` set.
- `copyAssets` build target collects and adds these dependencies to its inputs to ensure correct rebuilds.
- Added unit and integration tests to verify the behavior.

Fixes flutter#187051
@flutter-dashboard flutter-dashboard Bot added the CICD Run CI/CD label Jun 12, 2026
@github-actions github-actions Bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Jun 12, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the asset transformation process to track and monitor dependencies. The AssetTransformer now parses a generated depfile and returns an AssetTransformationResult containing these dependencies. DevFS and WebDevFS use this information to trigger asset re-transformations when dependencies change. Feedback was provided to ensure relative paths in the depfile are correctly resolved relative to the project directory by passing the working directory to depfileService.parse.

if (depfile.existsSync()) {
try {
final depfileService = DepfileService(logger: logger, fileSystem: _fileSystem);
final Depfile parsedDepfile = depfileService.parse(depfile);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When parsing the depfile, any relative paths written by the transformer will be resolved relative to the current working directory of the flutter_tools process (or the depfile's temp directory) rather than the project's working directory where the transformer was executed. To ensure relative paths in the depfile are correctly resolved relative to the project directory, pass _fileSystem.directory(workingDirectory) as the second argument to depfileService.parse.

Suggested change
final Depfile parsedDepfile = depfileService.parse(depfile);
final Depfile parsedDepfile = depfileService.parse(depfile, _fileSystem.directory(workingDirectory));

Update existing tests in `assets_test.dart` to expect the `--depfile` argument in `FakeCommand`s and handle it in `ArgParser`s.

Also fix the `Uses processors~/2 to transform assets` test to expect 5 transformed assets instead of 4, which is the correct number of assets (1 primary + 4 variants) created and processed in the test.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 12, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 12, 2026
Omit obvious local variable types for `processManager` and `environment` in `Uses processors~/2 to transform assets` test.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 12, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 12, 2026
Update `FakeCommand` and `onRun` in `bundle_builder_test.dart` to support the new `--depfile` argument.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 12, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 12, 2026
…transformers

Redesign `AssetTransformer` to pass the depfile path via `FLUTTER_ASSET_TRANSFORMER_DEPFILE` environment variable instead of `--depfile` command-line argument. This ensures 100% backwards-compatibility with existing or user-defined transformers that do not support the `--depfile` argument and would otherwise crash.

Make `invalidatedFiles` optional in `DevFS.updateBundle` to maintain compatibility with existing tests and callers that do not pass it.

Update `asset_transformer_test.dart` to verify the new environment variable behavior, using a robust temp directory filtering approach in the depfile parsing test.

Fix a latent bug in `assets_test.dart` where the `Uses processors~/2` test expected 4 transformed assets instead of 5 (1 primary + 4 variants).

Revert other test files (`devfs_test.dart`, `bundle_builder_test.dart`, and other tests in `assets_test.dart`) to their `origin/main` state since they no longer need to mock the `--depfile` argument.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 12, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 12, 2026
Update `DepfileService.parse` to accept an optional `baseDirectory` parameter, which is used to resolve relative paths found in the depfile.

Pass the `workingDirectory` (project root) as `baseDirectory` to `depfileService.parse` in `AssetTransformer` to ensure relative paths written by transformers are resolved correctly.

Add a unit test to `depfile_test.dart` to verify relative path resolution against a base directory.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 19, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 19, 2026
Rewrite `AssetTransformer` to use an implicit naming convention for depfiles instead of an environment variable.

Now, if a transformer wants to register dependencies, it should write its depfile to the output path with `.d` appended (i.e., `<output>.d`). `AssetTransformer` checks for the existence of this file after running the transformer, parses it, and registers the dependencies.

This approach:
- Is 100% backwards-compatible with existing transformers (no new arguments or env vars are passed).
- Eliminates the need for environment variables.
- Simplifies the code by removing the second temporary directory.

Update `asset_transformer_test.dart` to match this new convention.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 19, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 19, 2026
In the `Uses processors~/2 to transform assets` test, assert the length of `inputFilePaths` and `outputFilePaths` lists directly instead of calling `toSet()`.

`MemoryFileSystem` can reuse deleted temporary directory names sequentially. Since the test runs multiple transformations, some directories are deleted before others start, leading to name reuse (e.g., two runs using `/.tmp_rand0/rand1/`). Calling `toSet()` deduplicated these paths and caused the assertion to fail with a length of 4 instead of 5 in some environments (like CI).

Asserting the list length directly is more robust and correctly verifies that all 5 transformations were executed.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 19, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 19, 2026
@bkonyi bkonyi requested a review from chingjun June 23, 2026 17:14
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 25, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CICD Run CI/CD tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[flutter_tools] Allow asset transformers to register additional input dependencies for hot reload

1 participant