Skip to content

Conversation

@blu-base
Copy link
Contributor

@blu-base blu-base commented Oct 1, 2021

It seems likely, that the original issuer had a highlighting theme where the token Text has been highlighted.

Whereever applicable, the Text token is replaced with the more specific Whitespace token - not just the class keyword, but also similar occurrences, such as enums and structs.

This should fix #1237, and should generally be more precise lexing as well.

@Anteru Anteru self-assigned this Oct 2, 2021
@Anteru Anteru added the A-lexing area: changes to individual lexers label Oct 2, 2021
@blu-base
Copy link
Contributor Author

blu-base commented Oct 2, 2021

I did not run the tests initially... The proposed change impacts more places than i thought initially.

It seems there are also more cases of Keyword + Text instead of Keyword + Text.Whitespace. (run grep -r "Keyword, Text" in pygments/lexers)
That raises the question, is this a design choice?

@Anteru
Copy link
Collaborator

Anteru commented Oct 2, 2021

I've been trying to slowly clean up existing lexers and be more attentive to Text vs. Whitespace in new lexers. It's a pervasive problem to be honest, I wasn't aware how much of a problem it actually already is. This is definitely not by design, but mostly because we didn't pay enough attention to it. If you're interested in fixing this in more places this would be most welcome!

@Anteru Anteru merged commit 320853d into pygments:master Oct 2, 2021
@blu-base
Copy link
Contributor Author

blu-base commented Oct 2, 2021

I've been trying to slowly clean up existing lexers and be more attentive to Text vs. Whitespace in new lexers. It's a pervasive problem to be honest, I wasn't aware how much of a problem it actually already is. This is definitely not by design, but mostly because we didn't pay enough attention to it. If you're interested in fixing this in more places this would be most welcome!

This sounds like a problem i can handle. Just a question for the process, would you like individual PRs per changed lexers?

@Anteru
Copy link
Collaborator

Anteru commented Oct 2, 2021

Merged, thanks a lot for working on this!

@Anteru
Copy link
Collaborator

Anteru commented Oct 2, 2021

Thanks for picking this up! I think one PR for everything should be fine given it's fairly easy to review. If you're building any scripts to automate it, it would be great if you could submit them as well should we ever need to go through this exercise again.

@blu-base blu-base deleted the cpp_whitespace branch October 8, 2021 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-lexing area: changes to individual lexers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Broken C++ Class Highlighting

2 participants