Why?

this stdlib will be used when users write a scanner.c themselves and targeting wasm32

Can you point to a single scanner where those are used?

Can you point to a single scanner where those are used?

Yeah, I just fix a scanner that doesn't compile on wasm32, which use UINT8_MAX...

https://github.com/polarmutex/tree-sitter-beancount/pull/127/files#diff-bbcf6c785b007d9655b443abc204cc3cc32aa03cea190cb865fe44252dc6f1df

The Rust bindings are auto-generated and not meant to target wasm. To build a WASM-compliant parser, use tree-sitter build --wasm.

(This isn't saying it's impossible to add wasm support to the bindings, but this is not the way.)

(This isn't saying it's impossible to add wasm support to the bindings, but this is not the way.)

The Rust bindings are auto-generated and not meant to target wasm. To build a WASM-compliant parser, use tree-sitter build --wasm.

not really, we already support it #4820 , and it's currently in our testing matrix.

we just have a very small stdlib that users need to be careful when they write their own scanner.c

I don't think we need to support large set of stdlib, for example, there is no need to support printf, but add these static def to stdint.h won't do any harm

My point is that this needs to be done carefully -- you also need to provide the implementation for any stdlib function you want to add. (The stdlib is minimal by design; we can't simply add any function somebody thinks they may want -- we'd rather invest in providing a better purpose-built infrastructure for external scanners.)

(This is a general discussion to be had and not specific to this PR, mind you.)

I'm ok to not add strlen, I just find that it's already in the stdio.h, I simply move it. so I won't argue for it if someone think it's a bad idea to add it.

I mainly want to add missing defs on stdint.h, which are simple #define

@maxbrunsfeld this change looks innocuous so I am leaning towards merging it, but I think it would be helpful to have some sort of "design goal" for the wasm stdlib (since wasm parsers are the future -- and it would be very good if we could start supporting multiple runtimes for which this could serve as a specification).

I'm ok with all of these changes. Overall, I just want to provide a minimal subset of libc that enables writing external scanners, so that those scanners compile and behave the same when they are compiled to native code, and to wasm. I don't know exactly what the subset should be. I do want the code size to be reasonable. Constants like don't have any cost AFAICT.

I agree; this was a more general question (including for guidance on, e.g., the iswpunct addition).

Successfully created backport PR for release-0.26:

• fix(wasm): add common definitions to stdlib #5208

@trim21 you (and I) forgot to actually add this to the stdlib (wasm-stdlib.h).

(we really need some documentation about this... @WillLillis )

@trim21 you (and I) forgot to actually add this to the stdlib (wasm-stdlib.h).

What's inside this file? contents of stdlib files?

Yes. That's the part that actually ends up in the wasm blob for the parser.

If it's generated from other sources we may need a ci to check it or update it

Something like that, yes.