Skip to content

Remove dependency of ast_tools on Oxc crates from crates.io #15564

@overlookmotel

Description

@overlookmotel

Currently ast_tools (our codegen) depends on versions of Oxc crates (e.g. oxc_parser) from crates.io, instead of the local repo.

[dependencies]
# NOT `workspace = true`.
# If AST is updated, the local versions of these crates may not compile until after the codegen has run.
# So the codegen itself (this crate) can't use local versions.
# `features = ["serialize"]` on `oxc_span` and `oxc_syntax` is needed to work around a bug in `oxc_index`.
oxc_allocator = { version = "0.96.0" }
oxc_ast = { version = "0.96.0" }
oxc_ast_visit = { version = "0.96.0" }
oxc_codegen = { version = "0.96.0" }
oxc_minifier = { version = "0.96.0" }
oxc_parser = { version = "0.96.0" }
oxc_span = { version = "0.96.0", features = ["serialize"] }
oxc_syntax = { version = "0.96.0", features = ["serialize"] }

Why?

When you change an AST type (e.g. add a field to one of the structs), you need to run ast_tools codegen to update traits and various other code to match that change.

Until you do that, crates like oxc_parser will not compile.

Therefore, the codegen itself cannot depend on local copy of these crates, as then the codegen won't compile either. But the only way to get the codegen to compile... is to run the codegen. Catch 22!

Why is this a problem?

Having multiple versions of our own crates in Cargo.lock complicates dependency management.

Solution

How ast_tools is structured:

  • The codegen contains separate generators for Rust code and JS code.
  • Only the generators that produce JS code rely on oxc_* crates.

Therefore, we can fix this as follows:

  • Add Cargo feature generate-js to ast_tools which enables JS generators.
  • Make all oxc_* crate dependencies optional, only enabled when generate-js feature is enabled.
  • generate-js feature enabled by default.
  • Alter just ast command to:
cargo run -p oxc_ast_tools || {
  cargo run -p oxc_ast_tools --no-default-features
  cargo run -p oxc_ast_tools
}

If the first run fails due to compilation failure (the catch 22 problem), then run again with dependencies on oxc_* crates disabled. This will update the Rust code so that oxc_* crates will now compile. Then run it again - it should compile with the oxc_* crates this time, and generate the JS files too.

Metadata

Metadata

Labels

A-ast-toolsArea - AST toolsC-enhancementCategory - New feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions