Skip to content

TY-aware delayed HIR lowering#153489

Draft
aerooneqq wants to merge 23 commits intorust-lang:mainfrom
aerooneqq:two-phase-hir
Draft

TY-aware delayed HIR lowering#153489
aerooneqq wants to merge 23 commits intorust-lang:mainfrom
aerooneqq:two-phase-hir

Conversation

@aerooneqq
Copy link
Contributor

@aerooneqq aerooneqq commented Mar 6, 2026

This PR implements a prototype of TY-aware delayed HIR lowering. Part of #118212.

r? @petrochenkov

Motivation

When lowering delegation in perfect scenario we would like to access the TY-level information, in particular, queries like generics_of, type_of, fn_sig for proper HIR generation with less hacks. For example, we do not want to generate more lifetimes than needed, because without TY-level queries we do not know which delegee's lifetimes are late-bound. Next, consider recursive delegations, for their proper support without TY we would have to either duplicate generics inheritance code in AST -> HIR lowering or create stubs for parts of the HIR and materialize them later. We already use those queries when interacting with external crates, however when dealing with compilation of a local crate we should use resolver for similar purposes. Finally, access to TY-level queries is expected to help supporting delegation to inherent impls, as we can not resolve such calls during AST -> HIR stage.

Benefits

We eliminate almost all code that uses resolver in delegation lowering:

  • Attributes inheritance is done without copying attributes from AST at resolve stage
  • Fn signatures are obtained from tcx.fn_sig
  • Param counts are also obtained from tcx.fn_sig
  • is_method function now uses tcx.associated_item instead of resolver
  • Generics are now inherited through get_external_generics that uses tcx.generics_of. Generics for recursive delegations should also work
  • DelegationIds that stored paths for recursive delegations is removed, we now use only delegee_id
  • Structs that were used for storing delegation-related information in resolver are almost fully removed
  • ast_index is no more used

Next steps

  • Remove creating generic params through AST cloning, proper const types propagation
  • Inherent impls

High level design overview

Queries

We store ids of delayed items to lower in Crate struct. During first stage of lowering, owners that correspond to delayed items are filled with MaybeOwner::Phantom.

Next, we define two new queries: lower_to_hir_delayed and get_delayed_child_owner.

The first query is used when lowering known delayed owner.

The second is feeded with children that were obtained during lowering of a delayed owner (note that the result of lowering of a single LocalDefId is not a single MaybeOwner, its a list of (LocalDefId, MaybeOwner) where the first MaybeOwner corresponds to delayed LocalDefId and others to children that were obtained during lowering (i.e. generic params)). By default get_delayed_child_owner returns MaybeOwner::Phantom. As we do not want to predict the whole list of children which will be obtained after lowering of a single delayed item, we need to store those children somewhere. There are several options:

  • Make the return type of lower_to_hir_delayed to be FxIndexMap<LocalDefId, MaybeOwner> and search children here. Search will be either linear or we can introduce a map somewhere which will track parent-child relations between a single delayed LocalDefId and its children.
  • Try to make query that will lower all delayed items in a loop and return a complete map of all delayed LocalDefIds and their children. In this case there will be problems with delayed items that require information about other delayed items.

By using such atomic queries we handle the second concern, and in case of acyclic dependencies between delayed ids it automatically works, moreover we use queries as cache for delayed MaybeOwners. The only invariant here is that queries which are invoked during delayed HIR lowering of some LocalDefId should not in any way access children of other, yet unlowered, delayed LocalDefId, they should firstly materialize it.

Resolver for lowering

Currently the resolver_for_lowering is stolen in lower_to_hir function, however we want to prolong its life until all delayed LocalDefIds are materialized. For this purpose we borrow resolver_for_lowering in lower_to_hir and drop it after forcing materialization of all delayed LocalDefId in rustc_interface::run_required_analyses.

We also split resolver for lowering into two parts: the first part is a readonly part that came to us from resolve, the second part is a mutable part that can be used to add or overwrite values in the readonly part. Now we modify resolver in place as we steal it, however in this PR we borrow it so we need to split it.

AST index

Lowering uses an AST index. It is created in lower_to_hir function and it references parts of AST. We want to avoid reindexing AST on every delayed LocalDefId lowering, however now it is not clear how to properly do it. As delayed HIR lowering is used only for delegation unstable feature it should not affect other use-cases of the compiler. But it will be reworked sooner or later.

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 6, 2026
@rust-log-analyzer

This comment has been minimized.

@petrochenkov
Copy link
Contributor

cc @oli-obk you may be interested in this, because this likely intersects with incremental AST lowering.

@petrochenkov
Copy link
Contributor

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rust-bors

This comment has been minimized.

rust-bors bot pushed a commit that referenced this pull request Mar 6, 2026
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 6, 2026

#[extension(trait ResolverAstLoweringExt)]
impl ResolverAstLowering {
impl ResolverAstLowering<'_> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went down a slightly different route: c98471e

That gets rid of the extension trait entirely and allows adding custom information that doesn't need to be crate global.

providers.hir_module_items = map::hir_module_items;
providers.local_def_id_to_hir_id = |tcx, def_id| match tcx.hir_crate(()).owners[def_id] {
providers.get_delayed_child_owner = |_, _| MaybeOwner::Phantom;
providers.local_def_id_to_hir_id = |tcx, def_id| match tcx.hir_crate(()).owner(tcx, def_id) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just this change adding an owner method seems sth nice to do on main

@rust-bors
Copy link
Contributor

rust-bors bot commented Mar 6, 2026

☀️ Try build successful (CI)
Build commit: a77c1ae (a77c1ae1722badff1c427c67e07dc1e08989756f, parent: f82485388963286763f8c8a9261ac227627f6a18)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (a77c1ae): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)
0.7% [0.1%, 3.5%] 36
Regressions ❌
(secondary)
0.7% [0.0%, 2.7%] 103
Improvements ✅
(primary)
-0.6% [-1.4%, -0.2%] 9
Improvements ✅
(secondary)
-0.5% [-0.8%, -0.2%] 18
All ❌✅ (primary) 0.4% [-1.4%, 3.5%] 45

Max RSS (memory usage)

Results (primary 4.3%, secondary 3.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
4.3% [0.4%, 27.0%] 57
Regressions ❌
(secondary)
3.4% [1.0%, 24.6%] 83
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-3.7% [-4.2%, -3.2%] 2
All ❌✅ (primary) 4.3% [0.4%, 27.0%] 57

Cycles

Results (primary 2.7%, secondary 3.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
2.7% [2.1%, 3.2%] 2
Regressions ❌
(secondary)
3.9% [1.3%, 8.5%] 11
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.2% [-2.2%, -2.2%] 1
All ❌✅ (primary) 2.7% [2.1%, 3.2%] 2

Binary size

Results (primary 0.0%, secondary 0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
0.0% [0.0%, 0.1%] 11
Regressions ❌
(secondary)
0.1% [0.0%, 0.2%] 16
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 0.0% [0.0%, 0.1%] 11

Bootstrap: 480.442s -> 479.934s (-0.11%)
Artifact size: 397.10 MiB -> 397.38 MiB (0.07%)

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Mar 6, 2026
JonathanBrouwer added a commit to JonathanBrouwer/rust that referenced this pull request Mar 7, 2026
…o-hir-arena, r=petrochenkov

Replace Box<[TraitCandidate]> with &'hir [TraitCandidate<'hir>]

This PR allocates trait candidates on HIR arena and replaces `remove` with `get` in `ResolverAstLowering`. First step for rust-lang#153489.

r? @petrochenkov
rust-bors bot pushed a commit that referenced this pull request Mar 7, 2026
…, r=petrochenkov

Replace Box<[TraitCandidate]> with &'hir [TraitCandidate<'hir>]

This PR allocates trait candidates on HIR arena and replaces `remove` with `get` in `ResolverAstLowering`. First step for #153489.

r? @petrochenkov
JonathanBrouwer added a commit to JonathanBrouwer/rust that referenced this pull request Mar 7, 2026
…o-hir-arena, r=petrochenkov

Replace Box<[TraitCandidate]> with &'hir [TraitCandidate<'hir>]

This PR allocates trait candidates on HIR arena and replaces `remove` with `get` in `ResolverAstLowering`. First step for rust-lang#153489.

r? @petrochenkov
rust-timer added a commit that referenced this pull request Mar 7, 2026
Rollup merge of #153494 - aerooneqq:boxed-trait-candidates-to-hir-arena, r=petrochenkov

Replace Box<[TraitCandidate]> with &'hir [TraitCandidate<'hir>]

This PR allocates trait candidates on HIR arena and replaces `remove` with `get` in `ResolverAstLowering`. First step for #153489.

r? @petrochenkov
@rust-bors
Copy link
Contributor

rust-bors bot commented Mar 7, 2026

☔ The latest upstream changes (presumably #153541) made this pull request unmergeable. Please resolve the merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

perf-regression Performance regression. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants