Skip to content

Fix for lexing Python raw f-strings with backslashes#1683

Merged
Anteru merged 4 commits intopygments:masterfrom
jmd-dk:fix1681
Jan 17, 2021
Merged

Fix for lexing Python raw f-strings with backslashes#1683
Anteru merged 4 commits intopygments:masterfrom
jmd-dk:fix1681

Conversation

@jmd-dk
Copy link
Copy Markdown
Contributor

@jmd-dk jmd-dk commented Jan 14, 2021

Fix for #1681

  • Introduce a new 'rfstringescape', used for raw f-strings.
  • Build 'fstringescape' as a composite of 'rfstringescape' and 'stringescape'.

@jmd-dk jmd-dk changed the title Fix #1681 Fix for lexing Python raw f-strings with backslashes Jan 14, 2021
@Anteru
Copy link
Copy Markdown
Collaborator

Anteru commented Jan 14, 2021

Thanks for preparing the fix! Can you please add an unit test for this as well?

@jmd-dk
Copy link
Copy Markdown
Contributor Author

jmd-dk commented Jan 14, 2021

Thanks for preparing the fix! Can you please add an unit test for this as well?

I've added three tests; one for just "rawness", one for just usual f-string behavior and one for the previously missing combined behavior.

@jmd-dk
Copy link
Copy Markdown
Contributor Author

jmd-dk commented Jan 16, 2021

@Anteru Note that I've rewritten the old 'fstringescape' in terms of the new 'rfstringescape', as to not have the code

(r'\{\{', String.Escape),
(r'\}\}', String.Escape),

twice. Is this done correctly?

@Anteru
Copy link
Copy Markdown
Collaborator

Anteru commented Jan 17, 2021

Sounds reasonable to me, thanks for the contribution!

@Anteru Anteru added this to the 2.8 milestone Jan 17, 2021
@Anteru Anteru requested a review from birkenfeld January 17, 2021 07:52
@Anteru Anteru added A-lexing area: changes to individual lexers changelog-update Items which need to get mentioned in the changelog labels Jan 17, 2021
Copy link
Copy Markdown
Member

@birkenfeld birkenfeld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-lexing area: changes to individual lexers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants