Skip to content

Custom tokens don't backtrack to the lower level #1986

@isidentical

Description

@isidentical

Normally styles backtrack to the lower level tokens when doing style resolution. For example if there is no custom styling for Name.Function, it will try to find the style of Name and use it. But this doesn't work with custom tokens:

import pygments
from pygments.lexer import RegexLexer
from pygments.token import *
from pygments.style import Style
from pygments.formatters.terminal256 import Terminal256Formatter

class TestLexer(RegexLexer):
    tokens = {
        'root': [
            (r'A', Name.Function),
            (r'B', Name.Function.CUSTOM),
        ]
    }

class Style1(Style):
    styles = {
        Name: "#FA9BFA"
    }

class Style2(Style):
    styles = {
        Name.Function: "#FA9BFA",
        Name.Function.CUSTOM: "#FA9BFA"
    }


for style in [Style1, Style2]:
    print(pygments.highlight(
        code="AB",
        lexer=TestLexer(),
        formatter=Terminal256Formatter(style=style),
    ).strip())

I would expect both of these to work and output the same colors, but the initial style only highlights the Name.Function, and does not do anything for Name.Function.CUSTOM.
image

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-formattingarea: changes to formatters

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions