Corresponding code:

            case Integer i && i > 40:
                System.out.println("Integer");

Should we add a new token, PATTERN_DEF ?

Example:

|-PATTERN_DEF -> [47:27]        
|   |--LAND -> && [47:27]
|   |   |--PATTERN_VARIABLE_DEF -> PATTERN_VARIABLE_DEF [47:17]
|   |   |   |--MODIFIERS -> MODIFIERS [47:17]
|   |   |   |--TYPE -> TYPE [47:17]
|   |   |   |   `--IDENT -> Integer [47:17]
|   |   |   `--IDENT -> i [47:25]
|   |   `--GT -> > [47:32]
|   |       |--IDENT -> i [47:30]
|   |       `--NUM_INT -> 40 [47:34]

This would apply only to applications of the pattern production rule, and would have no effect on existing o instanceof String s definitions (causing no changes to existing AST structure). As pattern definitions become more prevalent (and complex) in Java grammar, it may prove helpful to have a dedicated PATTERN_DEF node.

Relevant grammar:

Pattern:
  PrimaryPattern
  GuardedPattern

GuardedPattern:
  PrimaryPattern && ConditionalAndExpression

PrimaryPattern:
  TypePattern
  ( Pattern )

@romani @pbludov @rnveach @strkkk please share your thoughts.

Am I got it right,

            case Integer i && i < 40:
                System.out.println("Small integer");
                break;

will be

|-PATTERN_DEF -> [47:27]        
|   |--LAND -> && [47:27]
|   |   |--PATTERN_VARIABLE_DEF -> PATTERN_VARIABLE_DEF [47:17]
|   |   |   |--MODIFIERS -> MODIFIERS [47:17]
|   |   |   |--TYPE -> TYPE [47:17]
|   |   |   |   `--IDENT -> Integer [47:17]
|   |   |   `--IDENT -> i [47:25]
|   |   `--LT -> > [47:32]
|   |       |--IDENT -> i [47:30]
|   |       `--NUM_INT -> 40 [47:34]

and

            case Integer i:
                System.out.println("Any integer");
                break;

will be also a PATTERN_DEF:

|-PATTERN_DEF -> [47:27]        
|   `--PATTERN_VARIABLE_DEF -> PATTERN_VARIABLE_DEF [47:17]
|       |--MODIFIERS -> MODIFIERS [47:17]
|       |--TYPE -> TYPE [47:17]
|       |   `--IDENT -> Integer [47:17]
|       `--IDENT -> i [47:25]

?

will be also a PATTERN_DEF:
?

We can do this.

Or, we can have a simple PATTERN_VARIABLE_DEF with no PATTERN_DEF parent node for switch, just like we do for instanceof. I am not sure what will prove useful from a static analysis viewpoint, because I can't really think of how any current checks would benefit from more easily accessing patterns in the AST (by adding a new token). I can work up some more examples if we want to consider adding a new token.

@pbludov can you think of any existing checks (or new check ideas) that would benefit from having a PATTERN_DEF node?

I think having PATTERN_DEF over LAND makes sense. This will allow some checks to distinguish between && for patterns and conditions.

Adding PATTERN_DEF as the parent of PATTERN_VARIABLE_DEF will be a breaking change for some third-party checks.

I like simple PATTERN_VARIABLE_DEF for switch.

One more thought: the if has no PATTERN_DEF:

if (o instanceof Integer i && i < 40) { // LAND, LITERAL_INSTANCEOF, PATTERN_VARIABLE_DEF, LT
 ...
}

switch (o) {
  case Integer i && i < 40: // LAND, PATTERN_VARIABLE_DEF, LT ?
    break;
}

Should there be an extra node for case?

why we think that it is not CASE specific syntax for pattern

It is not. This syntax can be used anywhere enhanced instance of with pattern matching can be used now, as long as expression is parenthesized.

public class Test {
    void method1(Object o) { // should have violation
        if (o instanceof String s) {
            // ^ pattern variable introduced (simple type pattern)
        }
        
        if (o instanceof String s && s.length() > 6) {
            // ^ pattern variable introduced (simple type pattern),
            // then evaluated in boolean expression
            // 's.length() > 6' is usual boolean expression, not part of pattern
        }
        
        if (o instanceof (String s && s.length() > 4)) {
            // ^ pattern variable introduced in guarded pattern; this refines the pattern itself
            // 's.length() > 4` is part of the pattern
        }
        
        int x = o instanceof (String s && s.length() > 6) ? 4 : 5;
        int y = o instanceof String s && s.length() > 4 ? 2 : 3;
        int z = o instanceof String s ? 1 : 2;
        
        while (o instanceof (String s && s.length() > 6)) {
            
        }
        while (o instanceof String s && s.length() > 4) {
            
        }
        while (o instanceof String s) {
            
        }
        
        for (int i = 0; o instanceof Integer myInt && myInt > 5;) {
            
        }
        for (int i = 0; o instanceof (Integer myInt && myInt > 5);) {
            
        }
        for (int i = 0; o instanceof Integer myInt; ) {
            
        }
    }
}

to keep it specific...

To me, specificity is more important for the difference between a simple pattern variable introduction, like:

        if (o instanceof String s) {
            // ^ pattern variable introduced (simple type pattern)
        }

And

        if (o instanceof (String s && s.length() > 4)) {
            // ^ pattern variable introduced in guarded pattern; this refines the pattern itself
            // 's.length() > 4` is part of the pattern
        }

In simple pattern variable introduction, whatever surrounding context we need can already be gathered by a check (and probably already is). This does not require (or benefit from) a new parent PATTERN_DEF (and would cause breaking changes to AST.

In guarded and parenthesized patterns, we should be able to know exactly all that this pattern encompasses easily. Why? Because pattern definitions are going to get much more complicated soon due to pattern matching for records and arrays, in JEP405:

static void printXCoordOfUpperLeftPointWithPatterns(Rectangle r) {
    if (r instanceof Rectangle(ColoredPoint(Point(var x, var y), var c), var lr)) {
        System.out.println("Upper-left corner: " + x);
    }
}

static void printSumOfFirstTwoXCoords(Object o) {
    if (o instanceof Point[] { Point(var x1, var y1), Point(var x2, var y2), ... }) {
        System.out.println(x1 + x2);
    }
}

In this case, we can easily have entire AST for Rectangle(ColoredPoint(Point(var x, var y), var c), var lr) in one go, and have a parent to look for when analyzing scopes of pattern variables, etc.

One more thought: the if has no PATTERN_DEF

Without parenthesized pattern, it would not. It's not really necessary, since && i < 40 is not part of the pattern. If a parenthesized pattern was used (o instanceof (Integer i && i < 40)), then it would now be a PATTERN_DEF by my proposal, because && i < 40 is now part of the pattern.

How about this: I will make a few small/simple input files and printed AST files, with examples from above for evaluation and add to this PR using proposed implementation. This will make it easier to visualize exact AST for certain syntax. Thumbs up on this for agreement.

TLDR of proposed implementation:

Integer i -> no PATTERN_DEF parent, in any case.

All other patterns have PATTERN_DEF parent.

No breaking changes to AST.

OK, implementation and tests updated. I am marking this ready for review. If we are good with AST structure, I will begin generating reports.

From https://openjdk.java.net/jeps/406:

We also change the grammar for instanceof expressions to:

InstanceofExpression:
  RelationalExpression instanceof ReferenceType
  RelationalExpression instanceof PrimaryPattern

This means that we can have a parenthesized pattern here now:

PrimaryPattern:
  TypePattern
  ( Pattern )

this is not breacking changes. parenthesized pattern not supported in jdk less jdk17. So this updates comes with "switch" of jdk17. I tested it.

Existing pattern did not allow for expected AST output files in non-compilable. I think it is better to keep input files and expected output together.