Skip to content

Treat all YAML templates files as UTF-8 encoded#449

Merged
rmilecki merged 1 commit intoinvoice-x:masterfrom
rmilecki:yaml-utf-8
Jan 25, 2023
Merged

Treat all YAML templates files as UTF-8 encoded#449
rmilecki merged 1 commit intoinvoice-x:masterfrom
rmilecki:yaml-utf-8

Conversation

@rmilecki
Copy link
Copy Markdown
Collaborator

Treat all YAML templates files as UTF-8 encoded

Encoding detection based on chardet is unreliable. For UTF-8 encoded
files with few non-ASCII chars it often fails to detect UTF-8. That
results in invoices not being parsed correctly.

Read every YAML file as UTF-8 encoded. It is used by all project
internal templates. It should be used by all external ones too.

FWIW YAML documentation also says that every *processor* must support
the UTF-8, UTF-16 and UTF-32:
https://yaml.org/spec/1.2.2/#52-character-encodings

Encoding detection based on chardet is unreliable. For UTF-8 encoded
files with few non-ASCII chars it often fails to detect UTF-8. That
results in invoices not being parsed correctly.

Read every YAML file as UTF-8 encoded. It is used by all project
internal templates. It should be used by all external ones too.

FWIW YAML documentation also says that every *processor* must support
the UTF-8, UTF-16 and UTF-32:
https://yaml.org/spec/1.2.2/#52-character-encodings

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
@rmilecki
Copy link
Copy Markdown
Collaborator Author

This fixes:

It's an alternative solution to the #446 . I'm not sure we should / have to support any other encoding.

Copy link
Copy Markdown
Collaborator

@bosd bosd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review, LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants