-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Document all token types #2849
Copy link
Copy link
Open
Labels
Description
Motivation
Themes depend on Prism producing tokens with specific types (or aliases). Right now, we do not guarantee or document any of those types.
Description
Document all standard token types (e.g. keyword, comment, ...). It should explain the general concept behind each token type and give at least one example.
The documentation should also include how languages are embedded. Example.
We should also guarantee that these concepts within languages are guaranteed to use these token types. (E.g. we guarantee that keywords always have a keyword class.) It might sound like we already do this but this is not the case right now. (E.g. we have many languages with operator keywords (e.g. NOT in SQL) that do not have a keyword class.)
Reactions are currently unavailable