Skip to content

Document all token types #2849

@RunDevelopment

Description

@RunDevelopment

Motivation
Themes depend on Prism producing tokens with specific types (or aliases). Right now, we do not guarantee or document any of those types.

Description
Document all standard token types (e.g. keyword, comment, ...). It should explain the general concept behind each token type and give at least one example.

The documentation should also include how languages are embedded. Example.

We should also guarantee that these concepts within languages are guaranteed to use these token types. (E.g. we guarantee that keywords always have a keyword class.) It might sound like we already do this but this is not the case right now. (E.g. we have many languages with operator keywords (e.g. NOT in SQL) that do not have a keyword class.)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions