Hello Łukasz!

I have been working on extending black for Cython.
Here is my approach:
Use a different grammar file, for the Cython grammar.
Start with the Python grammar and add Cython grammar rules to it one by one.
I have added most "simple" rules to the grammar so far, and I'm adding formatting rules for the blindingly obvious cases.

Here are some issues I faced in my endeavor:

As a result of using a different grammar file, there is a symbol mismatch (the same symbol may be represented by a different integer in the Python grammar and Cython grammar). This requires functions and top level variables using type_repr and syms to be parameterized with regards to the grammar's symbols. This does not affect tokens. (this may require encapsulating some of the global state)
Because of the way pgen works right now, string literals in the grammar cannot match NAME in the grammar. This is not that big of an issue in some cases and can be solved by disabling the keywords, however in other cases it's much harder as it conflicts with other rules. (like with the new expression: new = 5 vs new vector[int]()) I plan on deferring this issue for later.

Also I've expanded the tokenizer to accept integer literals ending in an arbitrary amount of (l, L, u, U), which affects affects python parsing as well. (single (l/L)-suffixed integer literals were already tokenized, even though they are invalid in Python3.)

Once I gather a sizable amount of rules, I'll let you know so you can inform me of your preferred formatting rules
I'd love to hear any suggestions/recommendations you have with regards to anything I've implemented so far, or any preference on how to proceed on the mentioned issues.

Cython grammar support #359

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions