Skip to content

Re-use original source file imports for efficient incremental type-checking#54819

Closed
devversion wants to merge 10 commits intoangular:mainfrom
devversion:import-manager
Closed

Re-use original source file imports for efficient incremental type-checking#54819
devversion wants to merge 10 commits intoangular:mainfrom
devversion:import-manager

Conversation

@devversion
Copy link
Member

@devversion devversion commented Mar 11, 2024

See individual commits. The new import manager is more complex, but basically it will allow us to:

  • simplify the consumer side of the manager to generate imports + inserting them into TS transforms/contents
  • improve incremental re-use with type checking— by allowing us to avoid structural source file changes
  • share the same logic for schematics where we are solving similar problems— intending to re-use existing imports

This PR also adds tests for import generation. Something we don't have right now.
In a follow-up we can replace/remove the schematics import manager then.

Related to: #53521 (review)

@devversion devversion requested a review from crisbeto March 11, 2024 16:29
@devversion devversion added action: review The PR is still awaiting reviews from at least one requested reviewer target: patch This PR is targeted for the next patch release area: compiler Issues related to `ngc`, Angular's template compiler labels Mar 11, 2024
@ngbot ngbot bot added this to the Backlog milestone Mar 11, 2024
@devversion devversion changed the title Re-use original source file imports for efficient incremental type-check program re-use Re-use original source file imports for efficient incremental type-checking Mar 11, 2024
@devversion devversion marked this pull request as ready for review March 11, 2024 18:39
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we have some code paths throughout the codebase that check source files by reference which this could break since it returns a new node. Not sure if this runs after them though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is the same as before. The utility addImports used by the Ivy transforms also transformed the sourceFile using updateSourceFile — that should be fine/identical

@devversion
Copy link
Member Author

Passing TGP. I've landed all preparation changes (making magic_string usable in g3)

@devversion devversion added action: merge The PR is ready for merge by the caretaker and removed action: review The PR is still awaiting reviews from at least one requested reviewer labels Mar 15, 2024
This commit introduces a new implementation of `ImportManager` that has
numerous benefits:

- It allows efficient re-use of original source file imports.
  * either fully re-using original imports if matching
  * updating existing import declarations to include new symbols.
- It allows efficient re-use of previous generated imports.
- The manager can be used for schematics and migrations.

The implementation is a rework of the import manager that we originally
built for schematics in Angular Material, but this commit improved it
to be more flexible, more readable, and "correct".

In follow-ups we can use this for schematics/migrations.
…manager

`ImportGenerator` is the abstraction used by the translator functions to
insert imports for `ExternalExpr` in an AST-agnostic way.

This was built specifically for the linker which does not use any of the
complex import managers- but rather re-uses `ngImport` or uses
`ngImport.Bla`.

This commit also switches the linker AST-agnostic generator to follow
the new signatures. This was rather trivial.
This commit switches ngtsc's JS and DTS transform to use the new import
manager. This is a drop-in replacement as we've updated the translator
helpers in the previous commit to align with the new API suggested by
the `ImportManagerV2` (to be renamed then).
…import manager

Updates the type-check block generation code (also for inline type check
blocks) to use the new import manager.

This is now a requirement because the translator utilities from the
reference emit environment expect an import manager that follows the
new contract established via `ImportGenerator<TFile, TExpression>`.

For type check files, we can simply print new imports as we don't expect
existing imports to be updated. That is because type check files do not
have any _original_ source files (or in practice— those are empty).

For type check blocks inline, or constructors, imports _may_ be re-used.
This is great as it helps fixing some incrementality bugs that we were
seeing in the type check code. That is, sometimes the type check block
code may generate imports conditionally for e.g. `TemplateRef`, or
animations. Those then **prevent** incremental re-use if TCB code
switches between those continously. We tried to account for that with
signal inputs by always pre-generating such imports. This fixed the
issue for type-check files, but for inline type check blocks this is
different as we would introduce new imports in user code that would then
be changed back in subsequential edit iterations. See:
angular#53521 (review).

In practice, the assumption was that we would be fine since user code is
most likely containing imports to `@angular/core` already. That is a
true assumption, but unfortunately it doesn't help with incremental
re-use because TypeScript's structural change detection does not dedupe
and expects 1:1 exact imports from their old source files.

microsoft/TypeScript#56845

To improve incremental re-use for the type check integration, we should
re-use original source file imports when possible. This commit enables
this.

To update imports and execute inline operations, we are now uisng
`magic-string` (which is then bundled) as it simplifies the string
manipulatuons.
Switches the JIT transforms to use the new import manager.
…al inputs

Enables the incremental type-checking test that we never enabled when we
landed signal inputs. Now that we fixed incremental re-use by re-using
the existing user imports for inline type check blocks, the test is
passing and can be enabled.
This commit deletes the older and now unused `ImportManager`.
To ease review and to allow for both instances to co-exist, `ImportManagerV2`
was introduced. This commit renames it to `ImportManager` now that we
deleted the older one.
…s/blocks

This commit adds some unit tests verifying the import generation in TCB
files and inline blocks. We don't seem to have any unit tests for these
in general. This commit adds some, verifying some characteristics we
would like to guarantee.
This commit updates the logic for preserving file overview comments
to be more reliable and less dependent on previous transforms.

Previously, with the old import manager, we had a utility called
`addImport` that always separated import statements and non-import
statements. This meant that the non-emitted statement from Tsickle
for the synthetic file-overview comments no longer lived at the
beginning of the file.

`addImports` tried to overcome this by adding another new non-emitted
statement *before* all imports. This then was later used by the
transform (or was assumed!) to attach the synthetic file overview
comments if the original tsickle AST Node is no longer at the top.

This logic can be improved, because the import manager shouldn't need to
bother about this fileoverview non-emitted statement, and the logic for
re-attaching the fileoverview comment should be local. This commit fixes
this and makes it a local transform.
@devversion devversion added target: major This PR is targeted for the next major release and removed target: patch This PR is targeted for the next patch release labels Mar 15, 2024
@devversion devversion added target: minor This PR is targeted for the next minor release and removed target: major This PR is targeted for the next major release labels Mar 15, 2024
@alxhub
Copy link
Member

alxhub commented Mar 15, 2024

This PR was merged into the repository by commit d01576b.

@alxhub alxhub closed this in 9a56bd5 Mar 15, 2024
alxhub pushed a commit that referenced this pull request Mar 15, 2024
…manager (#54819)

`ImportGenerator` is the abstraction used by the translator functions to
insert imports for `ExternalExpr` in an AST-agnostic way.

This was built specifically for the linker which does not use any of the
complex import managers- but rather re-uses `ngImport` or uses
`ngImport.Bla`.

This commit also switches the linker AST-agnostic generator to follow
the new signatures. This was rather trivial.

PR Close #54819
alxhub pushed a commit that referenced this pull request Mar 15, 2024
…54819)

This commit switches ngtsc's JS and DTS transform to use the new import
manager. This is a drop-in replacement as we've updated the translator
helpers in the previous commit to align with the new API suggested by
the `ImportManagerV2` (to be renamed then).

PR Close #54819
alxhub pushed a commit that referenced this pull request Mar 15, 2024
…import manager (#54819)

Updates the type-check block generation code (also for inline type check
blocks) to use the new import manager.

This is now a requirement because the translator utilities from the
reference emit environment expect an import manager that follows the
new contract established via `ImportGenerator<TFile, TExpression>`.

For type check files, we can simply print new imports as we don't expect
existing imports to be updated. That is because type check files do not
have any _original_ source files (or in practice— those are empty).

For type check blocks inline, or constructors, imports _may_ be re-used.
This is great as it helps fixing some incrementality bugs that we were
seeing in the type check code. That is, sometimes the type check block
code may generate imports conditionally for e.g. `TemplateRef`, or
animations. Those then **prevent** incremental re-use if TCB code
switches between those continously. We tried to account for that with
signal inputs by always pre-generating such imports. This fixed the
issue for type-check files, but for inline type check blocks this is
different as we would introduce new imports in user code that would then
be changed back in subsequential edit iterations. See:
#53521 (review).

In practice, the assumption was that we would be fine since user code is
most likely containing imports to `@angular/core` already. That is a
true assumption, but unfortunately it doesn't help with incremental
re-use because TypeScript's structural change detection does not dedupe
and expects 1:1 exact imports from their old source files.

microsoft/TypeScript#56845

To improve incremental re-use for the type check integration, we should
re-use original source file imports when possible. This commit enables
this.

To update imports and execute inline operations, we are now uisng
`magic-string` (which is then bundled) as it simplifies the string
manipulatuons.

PR Close #54819
alxhub pushed a commit that referenced this pull request Mar 15, 2024
…er (#54819)

Switches the JIT transforms to use the new import manager.

PR Close #54819
alxhub pushed a commit that referenced this pull request Mar 15, 2024
…al inputs (#54819)

Enables the incremental type-checking test that we never enabled when we
landed signal inputs. Now that we fixed incremental re-use by re-using
the existing user imports for inline type check blocks, the test is
passing and can be enabled.

PR Close #54819
alxhub pushed a commit that referenced this pull request Mar 15, 2024
This commit deletes the older and now unused `ImportManager`.

PR Close #54819
alxhub pushed a commit that referenced this pull request Mar 15, 2024
…54819)

To ease review and to allow for both instances to co-exist, `ImportManagerV2`
was introduced. This commit renames it to `ImportManager` now that we
deleted the older one.

PR Close #54819
alxhub pushed a commit that referenced this pull request Mar 15, 2024
…s/blocks (#54819)

This commit adds some unit tests verifying the import generation in TCB
files and inline blocks. We don't seem to have any unit tests for these
in general. This commit adds some, verifying some characteristics we
would like to guarantee.

PR Close #54819
alxhub pushed a commit that referenced this pull request Mar 15, 2024
)

This commit updates the logic for preserving file overview comments
to be more reliable and less dependent on previous transforms.

Previously, with the old import manager, we had a utility called
`addImport` that always separated import statements and non-import
statements. This meant that the non-emitted statement from Tsickle
for the synthetic file-overview comments no longer lived at the
beginning of the file.

`addImports` tried to overcome this by adding another new non-emitted
statement *before* all imports. This then was later used by the
transform (or was assumed!) to attach the synthetic file overview
comments if the original tsickle AST Node is no longer at the top.

This logic can be improved, because the import manager shouldn't need to
bother about this fileoverview non-emitted statement, and the logic for
re-attaching the fileoverview comment should be local. This commit fixes
this and makes it a local transform.

PR Close #54819
@angular-automatic-lock-bot
Copy link

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot bot locked and limited conversation to collaborators Apr 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

action: merge The PR is ready for merge by the caretaker area: compiler Issues related to `ngc`, Angular's template compiler target: minor This PR is targeted for the next minor release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants