Skip to content

Improve Cython Lexer #2932

@Spill-Tea

Description

@Spill-Tea

There are a few features missing from the Cython Lexer, or where definitions from the Python Lexer differ (e.g. Name.Builtin.Pseudo). Here is a list of various enhancements that can be made:

  1. Include the missing keyword "new", used in C++ mode, within "keyword" context.
  2. Define Keyword.Constant token (within "keyword" context) to include:
(words(("True", "False", "None", "NULL"), suffix=r"\b"), Keyword.Constant)
  1. Update Name.Builtin.Pseudo token (within "builtins" context) to exclude Keyword.Constant values:
(r"(?<!\.)(self|cls|Ellipsis|NotImplemented)\b", Name.Builtin.Pseudo)
  1. Include Missing data types in Builtin token defintion:
    a. char
    b. size_t
    c. ssize_t
    d. double

  2. Modify "cdef" context to discriminate between class, function, and variable name tokens. Currently, everything defaults to Name.Function token. Also include missing "packed" and "cppclass" keywords. Something like this:

[
(r"(public|readonly|extern|api|inline|packed)\b", Keyword.Reserved),
(
    r"(struct|enum|union|class|cppclass)\b(\s+)([a-zA-Z_]\w*)",
    bygroups(Keyword, Whitespace, Name.Class),
    "#pop",
),
(r"([a-zA-Z_]\w*)(\s*)(?=\()", bygroups(Name.Function, Whitespace), "#pop"),
(r"([a-zA-Z_]\w*)(\s*)(?=[:,=#\n]|$)", bygroups(Name.Variable, Whitespace), "#pop"),
(r"([a-zA-Z_]\w*)(\s*)(,)", bygroups(Name.Variable, Whitespace, Punctuation)),
... # the rest of context would remain the same
]

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-lexingarea: changes to individual lexers

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions