This PR contains:

• bugfix
• feature
• refactor
• documentation
• other

Are tests included?

• yes (bugfixes and features will not be merged without tests)
• no

Breaking Changes?

• yes (breaking changes will not be merged unless absolutely necessary)
• no

List any relevant issue numbers:

Description

This is a massive undertaking and there is still a lot to do. The goal is to switch the parser from acorn to swc while actually improving performance. Unfortunately, this does not mean using the JavaScript bindings of swc. At least in my experience, the cost of serializing an deserializing the complex AST nearly eats up the performance benefits of switching the parser. Moreover, swc has a very different AST (and even if you use the ESTree compat module, it is still a Babel AST and not an ESTree AST). Also, and this is critical, it uses utf-8 file positions while Rollup relies on the standard JavaScript utf-16 positions.
Instead, the approach is to directly use the Rust bindings of swc and convert the AST to a binary format before passing it as an (Array)Buffer to JavaScript. While doing this, we account for the AST differences and correct the file positions. Passing an ArrayBuffer is basically a free operation, so we just need to teach the JavaScript side to work on the ArrayBuffer. Also, the buffer is only about a third the size of stringified JSON. And last, this will enable us to easily pass ASTs to different threads or rather, do the parsing in a WebWorker and pass the buffer to the main thread for free.
To interact with the Rust code, I use napi-rs in NodeJS and will probably use wasm-pack for the browser build.
For now, the actual conversion is done. As a by-product of the process, the custom swc AST is converted to an ESTree-compliant AST, which has been verified by deep comparison with the acorn AST for all Rollup test cases.

The goal is to make this the core change in Rollup 4.

There is a lot that needs to be done, though, before this can even be tested.

Most critical changes to enable external testing

• Add WASM browser build
• Fix Node release process to allow the release of beta versions to enable others to test things

Critical for release

• Publish @rollup/browser with included WASM file
• Handle Syntax errors
• Handle other errors that used to be handled by acorn, e.g. duplicate bindings
• Handle PURE and other comments
• Make CI tests work
• Remove acorn options
• Add a Node WASM release for StackBlitz and similar use cases
• Make PRs available in REPL again (currently commented out on master)
• Make build process work across environments. Add instructions at least for Mac and Windows somewhere in CONTRIBUTING.md how to set up Rust nightly.
• Ensure npm install github/branch works again across environments. If necessary, add instructions how to install rustup nightly to PR comment. Remove browser build from prepare script to make this process faster.

Not critical but should be part of Rollup 4.0 to avoid a breaking change

• Switch to final AST structure of import attributes

Nice to have but not critical

• Store binary buffers in Rollup cache instead of AST if available (this is not the case if a custom AST is returned by a plugin, in which case we probably stick with storing the AST itself).
• @lukastaegert Make release comments work from CI in some way
• Enable aarch64-unknown-linux-musl target (currently commented out)
• See if using the browser readString version provides similar performance as using the Buffer method to simplify code
• Deduplicate strings in the binary AST
• Use faster utf-8 to utf-16 conversion by querying positions only in ascending order and iterating code points lazily
• Use one list of numeric ids for fixed strings
• Use one 32bit block for all booleans and fixed strings in any AST node to further reduce size
• Refine estimate for initial buffer size when converting the AST
• Add an easy way to compare the AST against the acorn AST in tests
• Use WebWorkers for parsing
• JSX support that keeps JSX syntax but tracks usages and renamed variables
• JSX support that transpiles JSX
• TypeScript support that removes types
• TypeScript support that keeps types but performs tree-shaking and tracks renames (if possible)
• Improve this.parse API with an asynchronous alternative that does not create the actual AST and supports very fast tree-walking via special APIs.
• Do not create intermediate AST nodes but directly convert AST to Rollup nodes. This means we need a solution if a plugin DOES return an AST. Ideally, we also turn that into binary, and we need to handle unknown nodes.
• Add instructions for copying wasm files when using @rollup/browser.

BREAKING CHANGES

• Acorn plugins are no longer supported, the acornInjectPlugins option has been removed
• The acorn option has been removed
• You can no longer pass additional options to this.parse
• Import assertions now use the new import attribute AST structure
• The "with" syntax for import attributes is not yet supported, awaiting support in SWC
• "assertions" have been replaced with "attributes" in various places of the plugin interface
• output.externalImportAssertions has been deprecated in favor of output.externalImportAttributes
• The INVALID_IMPORT_ASSERTION error code has been replaced with INVALID_IMPORT_ATTRIBUTE