Skip to content

refactor: identify AST nodes by NodeId instead of Span/Address#9609

Draft
IWANABETHATGUY wants to merge 2 commits into
mainfrom
chore/node-id-migration
Draft

refactor: identify AST nodes by NodeId instead of Span/Address#9609
IWANABETHATGUY wants to merge 2 commits into
mainfrom
chore/node-id-migration

Conversation

@IWANABETHATGUY

@IWANABETHATGUY IWANABETHATGUY commented May 30, 2026

Copy link
Copy Markdown
Member

Summary

Replaces the two ad-hoc cross-pass AST-node identity keys — oxc Span (deduped by PreProcessor) and arena Address — with oxc's post-semantic NodeId for every side table threaded from scan → link → finalize:

  • EcmaView::imports, dummy_record_set, new_url_references, this_expr_replace_map
  • MemberExprRef::node_id / LinkingMetadata::resolved_member_expr_refs
  • DynamicImportExprInfo::node_id / EntryPoint::related_stmt_infos
  • cross-module-optimization state (side-effect-free calls, unreachable dynamic imports)

Cross-module tables are keyed by (ModuleIdx, NodeId) because node ids are unique only within a single AST. Span is retained purely as source-location metadata (diagnostics, source maps, generated replacement spans); import records now carry an explicit importer_span so the TLA import-chain diagnostic no longer needs a reverse lookup over the (formerly span-keyed) imports map.

This removes the implicit "spans are stable and unique within a module" contract along with the PreProcessor synthetic-span workaround: finalizer-generated nodes keep NodeId::DUMMY and can no longer collide with scan-time records.

Notes

  • The incremental-cache (make_copy) and HMR paths finalize a clone of the scanned AST; they rely on clone_in_with_semantic_ids (via EcmaAst::clone_with_another_arena) to preserve node ids across the arena boundary, where plain clone_in would reset them to NodeId::DUMMY.
  • meta/design/ast-mutation.md is rewritten to document the new NodeId contract.

🤖 Generated with Claude Code

Copy link
Copy Markdown
Member Author

How to use the Graphite Merge Queue

Add the label graphite: merge-when-ready to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@netlify

netlify Bot commented May 30, 2026

Copy link
Copy Markdown

Deploy Preview for rolldown-rs canceled.

Name Link
🔨 Latest commit 44cb85c
🔍 Latest deploy log https://app.netlify.com/projects/rolldown-rs/deploys/6a1aaecb57946700088403a6

@IWANABETHATGUY IWANABETHATGUY force-pushed the chore/node-id-migration branch from 698e338 to 77574a0 Compare May 30, 2026 09:19
@IWANABETHATGUY IWANABETHATGUY force-pushed the chore/node-id-migration branch from 77574a0 to 5a92de4 Compare May 30, 2026 09:31
@IWANABETHATGUY IWANABETHATGUY changed the title u refactor: identify AST nodes by NodeId instead of Span/Address May 30, 2026
Comment thread crates/rolldown/src/ast_scanner/impl_visit.rs
@IWANABETHATGUY

Copy link
Copy Markdown
Member Author

@overlookmotel Thanks for the review. However, I still prefer using NodeId to identify nodes, for the following reasons:

Address is more error-prone. I know it is safe to use in most cases, but it can still potentially introduce bugs when unstable_address is involved.
Address comparison is indeed faster than NodeId comparison, but for Rolldown this should be trivial and probably not even observable, since AST traversal is only a small part of the overall pipeline.
If we keep using Address in some places, we would end up with two ways to identify a node, and three ways to inspect a node if we also count Span, which is used for diagnostics. This would make the code a bit messy.

Overall, I would prefer to use NodeId consistently for all AST node identification.

@IWANABETHATGUY IWANABETHATGUY marked this pull request as ready for review June 4, 2026 04:59
@codspeed-hq

codspeed-hq Bot commented Jun 4, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 4 untouched benchmarks
⏩ 10 skipped benchmarks1


Comparing chore/node-id-migration (44cb85c) with main (645ed62)

Open in CodSpeed

Footnotes

  1. 10 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@overlookmotel

overlookmotel commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

OK, fair enough. I tend towards prioritising perf myself, but obviously simplicity has large advantages - your rationale makes total sense.

My understanding of where we're at with NodeId in Oxc is:

  1. AST which has come out of Semantic: all AST nodes have unique NodeIds defined.
  2. New AST nodes created in transformer / minifier: have dummy NodeId (0).

We need to fix the 2nd, but it wasn't on immediate roadmap. We'll likely need to have 2 x AstBuilders with/without NodeId creation. If it's important to Rolldown, we can bump it up the priority list.

This PR: What I haven't figured out yet is whether it's going to hit problems due to the dummy NodeIds post-transform. I'd need to explore Rolldown (with an LLM) and get my head around the overall flow. I'm keen to do that - I've felt for a long time that I need to understand Rolldown much better - and this could be a good opportunity.

So... would you like to wait for me to do that and then review this PR (likely next week)? Or would you like to assess yourselves and get this merged earlier?

Sorry, I had hoped to get into it properly this week, but... other stuff has been going on.

@IWANABETHATGUY

Copy link
Copy Markdown
Member Author

OK, fair enough. I tend towards prioritising perf myself, but obviously simplicity has large advantages - your rationale makes total sense.

My understanding of where we're at with NodeId in Oxc is:

  1. AST which has come out of Semantic: all AST nodes have unique NodeIds defined.
  2. New AST nodes created in transformer / minifier: have dummy NodeId (0).

We need to fix the 2nd, but it wasn't on immediate roadmap. We'll likely need to have 2 x AstBuilders with/without NodeId creation. If it's important to Rolldown, we can bump it up the priority list.

This PR: What I haven't figured out yet is whether it's going to hit problems due to the dummy NodeIds post-transform. I'd need to explore Rolldown (with an LLM) and get my head around the overall flow. I'm keen to do that - I've felt for a long time that I need to understand Rolldown much better - and this could be a good opportunity.

So... would you like to wait for me to do that and then review this PR (likely next week)? Or would you like to assess yourselves and get this merged earlier?

Sorry, I had hoped to get into it properly this week, but... other stuff has been going on.

No rush. I’ll wait until you finish the review before merging it.

@IWANABETHATGUY IWANABETHATGUY marked this pull request as draft June 4, 2026 13:37
graphite-app Bot pushed a commit that referenced this pull request Jun 9, 2026
## Summary

Adds `meta/design/ast-construction.md` documenting how rolldown should construct oxc AST, and cross-links it from `ast-mutation.md`.

The convention:
- **Generic nodes → oxc's `AstBuilder` directly** (handle named `ast`).
- **Rolldown-specific patterns → an `AstBuilderExt` extension trait**, methods prefixed `make_` and named after the operation (e.g. `make_to_esm_wrapper`), mirroring oxc's builder signature style. The call site self-identifies (bare node name = oxc; `make_*` = rolldown) and avoids inherent-vs-trait method shadowing.
- **Build programmatically by default; parsing source is an exception.** Direct construction has no runtime cost; parsing pays lexing+parsing overhead every build. Authoring code as JS and parsing it (`EcmaCompiler::parse`) is reserved for a large fixed blob — in practice just the runtime module.
- The `AstSnippet` facade is retired over time (its author already noted the name was a compromise for `AstBuilder`).

## Background

oxc made `AstBuilder` the single sanctioned construction path (`#[non_exhaustive]` on `NodeId`-bearing nodes, oxc#23046, handled in #9670) and is redesigning it further (oxc#23043, which cites #9609). This doc aligns rolldown with that direction instead of growing a divergent local layer.

## Notes

- **Docs-only; no code changes.**
- The `## Plan` section is **temporary** and will be removed once the migration it describes is implemented.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@hyf0

hyf0 commented Jun 10, 2026

Copy link
Copy Markdown
Member

So... would you like to wait for me to do that and then review this PR (likely next week)? Or would you like to assess yourselves and get this merged earlier?

@overlookmotel

There isn't really a hurry for this, but I also get sometimes we want things to get too pefect to be done. If you could finish the review in time that's good, if you can't that's fine too. On the other hands, merging this would be also a way of getting feedbacks for the node id design. Will plan to merge this after this weeks' release, so we get a full week time to revert this PR if you find some significant downsides.

I wanna push this forward, because I'm looking at the some internal improvement/refactor. I want the future refactor can build on top of the new basis. It might not be perfect, but we still want to fix bugs on top of the new basis instead of the old one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants