PHP 8.2 | Tokenizer/PHP: add support for DNF types#461
Merged
Conversation
This commit adds tokenizer support for DNF types as per the proposal outlined in 387.
This means that:
* Two new tokens are introduced `T_TYPE_OPEN_PARENTHESIS` and `T_TYPE_CLOSE_PARENTHESIS` for the parentheses used in DNF types.
This allows for sniffs to specifically target those tokens and prevents sniffs which are looking for the "normal" open/close parenthesis tokens from acting on DNF parentheses.
* These new tokens, like other parentheses, will get the `parenthesis_opener` and `parenthesis_closer` token array indexes and the tokens between them will have the `nested_parenthesis` index.
Based on the currently added tests, the commit safeguards that:
* The `|` in types is still tokenized as `T_TYPE_UNION`, even in DNF types.
* The `&` in types is still tokenized as `T_TYPE_INTERSECTION`, even in DNF types.
* The `static` keyword for properties is still tokenized as `T_STATIC`, even when right before a DNF type (which could be confused for a function call).
* The arrow function retokenization to `T_FN` with a `T_FN_ARROW` scope opener is handled correctly, even when DNF types are involved and including when the arrow function is declared to return by reference.
* The keyword tokens, like `self`, `parent`, `static`, `true` or `false`, when used in DNF types are still tokenized to their own token and not tokenized as `T_STRING`.
* The `array` keyword when used in DNF types is still tokenized as `T_STRING` and not as `T_ARRAY`.
* A `?` intended as an (illegal) nullability operator in combination with a DNF type is still tokenized as `T_NULLABLE` and not as `T_INLINE_THEN`.
* A function declaration open parenthesis before a typed parameter isn't accidentally retokenized to `T_TYPE_OPEN_PARENTHESIS`.
Includes ample unit tests.
Even so, strenuous testing of this PR is recommended as there are so many moving parts involved, it is very easy for something to have been overlooked.
Related to 105
Closes 387
Closes squizlabs/PHP_CodeSniffer 3731
ceab210 to
5ce28e9
Compare
Member
Author
|
Rebased without changes. Even though there has been no feedback so far, I am going to merge this PR to improve the chances of people using the |
This was referenced Apr 28, 2024
This was referenced May 2, 2024
4 tasks
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This commit adds tokenizer support for DNF types as per the proposal outlined in #387.
This means that:
T_TYPE_OPEN_PARENTHESISandT_TYPE_CLOSE_PARENTHESISfor the parentheses used in DNF types. This allows for sniffs to specifically target those tokens and prevents sniffs which are looking for the "normal" open/close parenthesis tokens from acting on DNF parentheses.parenthesis_openerandparenthesis_closertoken array indexes and the tokens between them will have thenested_parenthesisindex.Based on the currently added tests, the commit safeguards that:
|in types is still tokenized asT_TYPE_UNION, even in DNF types.&in types is still tokenized asT_TYPE_INTERSECTION, even in DNF types.statickeyword for properties is still tokenized asT_STATIC, even when right before a DNF type (which could be confused for a function call).T_FNwith aT_FN_ARROWscope opener is handled correctly, even when DNF types are involved and including when the arrow function is declared to return by reference.self,parent,static,trueorfalse, when used in DNF types are still tokenized to their own token and not tokenized asT_STRING.arraykeyword when used in DNF types is still tokenized asT_STRINGand not asT_ARRAY.?intended as an (illegal) nullability operator in combination with a DNF type is still tokenized asT_NULLABLEand not asT_INLINE_THEN.T_TYPE_OPEN_PARENTHESIS.Includes ample unit tests.
Even so, strenuous testing of this PR is recommended as there are so many moving parts involved, it is very easy for something to have been overlooked.
Suggested changelog entry
Added: tokenizer support for Disjunctive Normal Form types.
Related issues/external references
Related to #105
Closes #387
Closes squizlabs/PHP_CodeSniffer#3731
Types of changes