Skip to content

Allow token-based rules to work on source code with syntax errors #11915

@dhruvmanila

Description

@dhruvmanila

Currently, the rules which work with the tokens doesn't emit diagnostics after the location of the first syntax error. For example:

foo;

"hello world

bar;

This will only raise the useless-semicolon for foo and not bar because there's an unterminated string literal in between them.

Playground: https://play.ruff.rs/4c17a92c-0189-4b98-b961-27b04db14599

The task here is to disable this limit and allow token-based rules to check all the tokens. This raises a question: Now that the parser can now recover from an unclosed parenthesis (#11845), how to make sure that the rule logic knows about this and has the correct information about the nesting level? Should we reduce the nesting level if we encounter a Newline token?

We also need to make sure that this doesn't panic in any scenarios (valid or invalid source code). This can be done via lots of fuzzing.

Metadata

Metadata

Assignees

Labels

linterRelated to the linter

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions