TNTLexer: Don't crash on unexpected EOL.#1570
Merged
Anteru merged 3 commits intopygments:masterfrom Oct 14, 2020
Merged
Conversation
Catch IndexErrors in each line and error the rest of the line, leaving whatever tokens were found.
Contributor
Author
|
Force pushed to just the one commit (the others are already in the repo) |
Anteru
requested changes
Oct 12, 2020
Collaborator
Anteru
left a comment
There was a problem hiding this comment.
Please add a test case for this (it should crash without the change applied.)
Contributor
Author
|
(Seen 4 hours ago, been writing tests for the whole lexer ever since) |
pygments/lexers/tnt.py: * Fix indentation on import * Fix: TNTLexer.cur is class-level reference if not initialized in get_tokens_unprocessed, so init it in __init__ too * Fix: Fantasy markers are not allowed as components of other formulas, so have a dedicated check for them in the body of get_tokens_unprocessed which disables the normal formula handling if present * Clarify TNTLexer.lineno docstring * Attempt to discard tokens before an IndexError +tests/test_tnt.py: * Test every method, and test both +ve and -ve matches for most * Lexer fixture is test-level to reinitialize cur clean each time * Don't test actual get_tokens_unprocessed method (besides for fantasy markers) because the text testing is left to examplefiles AUTHORS: + Add myself to credits :)
Collaborator
|
That looks fairly comprehensive, thanks! I'll wait for the tests to finish, then merge -- thanks for your contribution. |
Contributor
Author
|
Glad to be of service! Sorry for the original issue I caused. At least I got good tests in :) |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1568 which is blocking #1425