-
-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PHP 8.2 | Tokenizer/PHP: add support for DNF types #461
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds tokenizer support for DNF types as per the proposal outlined in 387. This means that: * Two new tokens are introduced `T_TYPE_OPEN_PARENTHESIS` and `T_TYPE_CLOSE_PARENTHESIS` for the parentheses used in DNF types. This allows for sniffs to specifically target those tokens and prevents sniffs which are looking for the "normal" open/close parenthesis tokens from acting on DNF parentheses. * These new tokens, like other parentheses, will get the `parenthesis_opener` and `parenthesis_closer` token array indexes and the tokens between them will have the `nested_parenthesis` index. Based on the currently added tests, the commit safeguards that: * The `|` in types is still tokenized as `T_TYPE_UNION`, even in DNF types. * The `&` in types is still tokenized as `T_TYPE_INTERSECTION`, even in DNF types. * The `static` keyword for properties is still tokenized as `T_STATIC`, even when right before a DNF type (which could be confused for a function call). * The arrow function retokenization to `T_FN` with a `T_FN_ARROW` scope opener is handled correctly, even when DNF types are involved and including when the arrow function is declared to return by reference. * The keyword tokens, like `self`, `parent`, `static`, `true` or `false`, when used in DNF types are still tokenized to their own token and not tokenized as `T_STRING`. * The `array` keyword when used in DNF types is still tokenized as `T_STRING` and not as `T_ARRAY`. * A `?` intended as an (illegal) nullability operator in combination with a DNF type is still tokenized as `T_NULLABLE` and not as `T_INLINE_THEN`. * A function declaration open parenthesis before a typed parameter isn't accidentally retokenized to `T_TYPE_OPEN_PARENTHESIS`. Includes ample unit tests. Even so, strenuous testing of this PR is recommended as there are so many moving parts involved, it is very easy for something to have been overlooked. Related to 105 Closes 387 Closes squizlabs/PHP_CodeSniffer 3731
jrfnl
force-pushed
the
php-8.2/387-tokenizer-add-support-for-dnf-types
branch
from
April 28, 2024 11:16
ceab210
to
5ce28e9
Compare
Rebased without changes. Even though there has been no feedback so far, I am going to merge this PR to improve the chances of people using the |
This was referenced Apr 28, 2024
This was referenced May 2, 2024
4 tasks
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This commit adds tokenizer support for DNF types as per the proposal outlined in #387.
This means that:
T_TYPE_OPEN_PARENTHESIS
andT_TYPE_CLOSE_PARENTHESIS
for the parentheses used in DNF types. This allows for sniffs to specifically target those tokens and prevents sniffs which are looking for the "normal" open/close parenthesis tokens from acting on DNF parentheses.parenthesis_opener
andparenthesis_closer
token array indexes and the tokens between them will have thenested_parenthesis
index.Based on the currently added tests, the commit safeguards that:
|
in types is still tokenized asT_TYPE_UNION
, even in DNF types.&
in types is still tokenized asT_TYPE_INTERSECTION
, even in DNF types.static
keyword for properties is still tokenized asT_STATIC
, even when right before a DNF type (which could be confused for a function call).T_FN
with aT_FN_ARROW
scope opener is handled correctly, even when DNF types are involved and including when the arrow function is declared to return by reference.self
,parent
,static
,true
orfalse
, when used in DNF types are still tokenized to their own token and not tokenized asT_STRING
.array
keyword when used in DNF types is still tokenized asT_STRING
and not asT_ARRAY
.?
intended as an (illegal) nullability operator in combination with a DNF type is still tokenized asT_NULLABLE
and not asT_INLINE_THEN
.T_TYPE_OPEN_PARENTHESIS
.Includes ample unit tests.
Even so, strenuous testing of this PR is recommended as there are so many moving parts involved, it is very easy for something to have been overlooked.
Suggested changelog entry
Added: tokenizer support for Disjunctive Normal Form types.
Related issues/external references
Related to #105
Closes #387
Closes squizlabs/PHP_CodeSniffer#3731
Types of changes