• Closes Create flag for installing web-tree-sitter built with —debug #1566

Problem

The web bindings feel hard to maintain, and also hard to debug. Additionally, there's a few missing components for the web bindings that would aid downstream users immensely - that being bundling the sourcemaps, and having an option to install a "debug" build of web-tree-sitter.

Solution

We've implemented the missing components outlined above, as well as rewriting the library to TypeScript, as it feels much easier to navigate than before with the types being laid out pretty nicely (including the FFI functions!). However, with this change comes some (hopefully) positive changes to the build process.

Because we're using TypeScript, we have a new compilation step added relies on esbuild to bundle our TypeScript files into one JavaScript file with a sourcemap.

However, sourcemaps were a particular pain to implement for the final JS file, because we have two "phases" of compilation - one from TS -> JS (esbuild), and one from JS -> Emscripten-JS. Emscripten doesn't provide a way to generate sourcemaps for the generated JS code, as it just copy-pastes our post-js as is. In theory we should be able to use our TS -> JS source map since our source code wasn't modified, but that's not the case because the line numbers have shifted. As such, we have to run a post-processing step to "relocate" the sourcemap line number information. This ends up working pretty well, but was not fun to figure out.

The above is no longer true, since I've rewritten the way the higher-level API interacts with the WASM module. Instead of gluing it all together inside the module with pre-js and post-js hacks, we're instead consuming the WASM module like a normal module, and bundling that with the final JS code. This is way easier, and doesn't require hacks to the sourcemaps.

Debug Info

For debug information, we build the wasm files with features such as assertions and debug symbols. We store these in the debug folder, and optionally allow a user to use them with a sub-import, web-tree-sitter/debug. Effectively, tacking on /debug to your import will let you automatically use the debug build to see what's going on.

In GH releases, these debug files would show up as tree-sitter-debug.{js|wasm}(.map)?.

TypeScript, CommonJS, and ESM

Rewriting the library in TypeScript also helped me improve the public type definitions, as we can autogenerate them now, and writing the tests in TypeScript was just a much smoother process overall. Hopefully others feel the same way as they peruse and navigate the codebase (and potentially contribute back, as I've added a nice little contributing.md file).

Some notes for readers - we now actually support both CommonJS and ESM modules, by invoking emcc with EXPORT_ES6=1, and esbuild with the format set to cjs and esm.

The default JS file is now an ES6 module, and will be the file that's uploaded to GH releases. For users using the NPM registry, this is automatically taken care of for you with conditional exports (including for the debug sub-export).

I'll let this sit for a bit so people can try it out locally.

cc @verhovsky @wenkokke I think you'd both be interested in these changes (sorry for the ping if that's no longer true)