Skip to content

Hint when text file encoding is not standard UTF-8 #5440

@cyckl

Description

@cyckl

Description

Currently, typst will output a vague error when reading a file with a byte order mark (BOM).

watching report.typ
writing to report.pdf

[17:42:00] compiled with errors

error: failed to parse JSON (expected value at line 1 column 1)
    ┌─ report.typ:216:23
    │
216 │ #let data-stock = json("data/stock.json")
    │                        ^^^^^^^^^^^^^^^^^

Adding a more descriptive error in this case would be ideal in order to give the user some direction on how to correct it—this could be done by detecting a BOM and indicating it:

watching report.typ
writing to report.pdf

[17:42:00] compiled with errors

error: failed to parse JSON (byte order mark present at start of file)
    ┌─ report.typ:216:23
    │
216 │ #let data-stock = json("data/stock.json")
    │                        ^^^^^^^^^^^^^^^^^

While this is the best solution, this already approaches the amount of work required to automatically handle BOM itself anyways (#3157). Alternatively, the error message could simply indicate that there was something unexpected at the beginning of the file stream and allude to an encoding error:

watching report.typ
writing to report.pdf

[17:42:00] compiled with errors

error: failed to parse JSON (expected value at line 1 column 1, verify encoding is UTF-8)
    ┌─ report.typ:216:23
    │
216 │ #let data-stock = json("data/stock.json")
    │                        ^^^^^^^^^^^^^^^^^

Use Case

While not an issue in a lot of cases, some software does export files with the BOM (Microsoft Excel CSV export). The existing error message doesn't indicate any specific problem with the file, so it's up to a user to investigate and become aware of how typst interacts with BOM text files.

Metadata

Metadata

Assignees

No one assigned

    Labels

    diagnosticsImprovements to compiler errorsfeature requestNew feature or requestgood contributionSuitable for contribution, but unlike "good first issue", it may need prior experiencescriptingAbout Typst's coding capabilities

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions