[kotlin] Fix kotlin grammar for parsing multidollar interpolation by stokpop · Pull Request #6497 · pmd/pmd

stokpop · 2026-03-10T20:06:46Z

Fix kotlin grammar for parsing multidollar interpolation

Newer syntax in Kotlin fails to parse with messages being written to standard err.

As reported: the multi dollar syntax gives errors during parsing. Example:

val productName = "carrot"
val requestedData =
    $$$"""{
      "currency": "$",
      "enteredAmount": "42.45 $$",
      "$${serviceField.length()}": "none",
      "product": "$$${productName + "-fixed-postfix"}"
    }
    """

Related issues

Fix [kotlin] Support multidollar interpolation (Kotlin 2.2) #6003 - the parsing errors as reported should not be there anymore - it is not a full update to kotlin 2.2

Ready?

Added unit tests for fixed bug/feature
Passing all unit tests
Complete build ./mvnw clean verify passes (checked automatically by github actions)
Added (in-code) documentation (if needed)

Background and discussion

This is a fix with intention to minimal changes to the current Kotlin grammar
[update: this is fixed, see below] This fix does not distinguish between the number of dollars, both the $${ and $$${ in example are turned into expressions now, so it is up to the rules to check the number of $$$.
The current Kotlin version is still 1.8, but now with a fix for a syntax introduced in version 2.1
To aid in debugging, Kotlin nodes display a human readable toString() now: needed code update in pmd-core/BaseAntlrInnerNode.java to make that possible
A couple of unit tests with specific Kotlin syntax are added for future regression testing
Helper methods for testing Kotlin parsing have been added to KotlinParsingHelper.
- These can maybe also be moved to more generic test helper code (in pmd-test module?)
In another branch we tried to upgrade to a newer Kotlin grammar as found here: https://github.com/antlr/grammars-v4
- that works but it changes the generated Antlr4 code in a non-compatible way
- this newer grammar does not solve the multi dollar issue!
- should we create PR to see if that is way forward?
- also see if a new Kotlin version can/should be added next to 1.8?

…g tests, now with failing Multiline Multi dollar issue in test

… used to match the number of tripple quote prefix dollars

pmd-actions-helper · 2026-03-10T20:26:45Z

Documentation Preview

No relevant source code has been changed, pmdtester skipped.

(comment created at 2026-04-23 19:00:24+00:00 for 1328d18)

… fixes in grammar

stokpop · 2026-03-11T08:28:31Z

Went down the number of dollars rabbit hole: now the grammar takes into consideration the number of dollars to match actual refs and expressions in the multi line templates. Added unit tests for regular and "split" dollar cases.

adangel

Thanks for the PR and sorry for the late response.

I think, there are a couple of things, we need to fix, see my comments.

Apart from that:

We probably should introduce a new kotlin language version 2.2.0. This would be just for information purposes. In Java, we actually fail to parse code, that as newer language constructs than the select version allows, but I think, this would go too far for Kotlin. So, regardless of the language version, the parser behaves the same. It is just to document, we support (partially) version 2.2.0 now.
The language feature is officially called "Multidollar interpolation: improved handling of $ in string literals". And that's the only language feature, that will be added by this PR. Note: it was preview in 2.1.0 and stabilized in 2.2.0. To make this clear, we should document this also on https://github.com/pmd/pmd/blob/main/docs/pages/pmd/languages/kotlin.md . Something along the lines: "PMDs parser lacks behind ... the following features are supported... other features of 2.0.0, 2.1.0, 2.2.0, 2.3.0 are not supported. PRs are welcome."
Unfortunately, there is currently no official antlr grammar anymore, see https://youtrack.jetbrains.com/issue/KT-84579/Outdated-ANTLR-grammar-in-Kotlin-specification - but the best overview about the language features I found, is this one: https://kotlinlang.org/docs/kotlin-language-features-and-proposals.html#stable . So, it seems we need to keep our grammar up to date on our own unfortunately.

stokpop · 2026-03-25T09:45:30Z

@adangel thanks for your review! Will start working on the improvements.

stokpop · 2026-03-25T09:56:48Z

Unfortunately, there is currently no official antlr grammar anymore, see https://youtrack.jetbrains.com/issue/KT-84579/Outdated-ANTLR-grammar-in-Kotlin-specification - but the best overview about the language features I found, is this one: https://kotlinlang.org/docs/kotlin-language-features-and-proposals.html#stable . So, it seems we need to keep our grammar up to date on our own unfortunately.

Thanks for asking jetbrains youtrack! Hopefully they are able to sync.

Did you also have a look at the two alternative grammars here: https://github.com/antlr/grammars-v4/tree/master/kotlin?
My experiment shows they are a bit different and have missing and added "features". So not sure we can switch/extend these.

adangel · 2026-03-27T08:03:11Z

Did you also have a look at the two alternative grammars here: https://github.com/antlr/grammars-v4/tree/master/kotlin?
My experiment shows they are a bit different and have missing and added "features". So not sure we can switch/extend these.

I didn't have a detailed look. But I guess, we should stick to the grammar we have - if we switch the entire grammar, there might be a risk that all custom rules need to be rewritten, if the parser outputs a different parse tree.

What I would suggest is this:

make sure, we use the latest official grammar from https://github.com/Kotlin/kotlin-spec/tree/release/grammar/src/main/antlr . Hopefully, we did this already and there is nothing to be changed. That would be the starting point.
Keep on working adding one feature at a time, like in this PR, which only adds multidollar interpolation.
I'll update [kotlin] Support multidollar interpolation (Kotlin 2.2) #6003 to be only about the multidollar interpolation feature and not Kotlin 2.2 in general anymore.
When adding another feature (different PR!), you might want to look at these alternative grammars to see, how/if they solved it and you can reimplement it in PMD's own grammar.

In general, I wouldn't expect an updated, official upstream grammar from Jetbrains soon. AFAIK, they've rewritten their parser/compiler (see K2) and it looks like, there is no formal grammar used anywhere anymore. If it comes, we need to see, whether we are still able to use it, since we are now implementing our own grammar.

…de. Prevent NullPointerException in getTextRegion()

…tead of just printing to standard err so test case fails when parsing fails.

…keep behaviour same. Use custom parser that throws ParserException for running tests and getting notified for parse issues.

codacy-production · 2026-04-03T12:28:05Z

Not up to standards ⛔

🔴 Issues 6 medium

Alerts:
⚠ 6 issues (≤ 0 issues of at least minor severity)

Results:
6 new issues

Category Results

BestPractice 6 medium

View in Codacy

🟢 Metrics 11 complexity · 1 duplication

Metric Results

Complexity 11

Duplication 1

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer}
_{TIP This summary will be updated as you push new changes.}

stokpop · 2026-04-03T12:36:10Z

Hi @adangel, I requested a new review of this PR via the button on top.

Can you have another look please and I think the process is that you close the conversations that are ok?

stokpop · 2026-04-16T06:06:06Z

Hello @adangel, can you please have a look at the updates on your review comments? (Or should I maybe turn it to "resolved" myself?)

adangel

Thanks, looks good. You can fix the minor things I mentioned, otherwise I'll do it myself.

I won't have time to merge it before next week though.

added refererence to kotlin feature and fix a typo Co-authored-by: Andreas Dangel <andreas.dangel@adangel.org>

…Test for future regression tests

stokpop · 2026-04-20T10:44:39Z

@adangel please have another look at the requested changes above, there is a remaining question about the toString(): leave it as is in this PR or revert back to Object.toString()?

adangel · 2026-04-23T18:38:58Z

remaining question about the toString(): leave it as is in this PR or revert back to Object.toString()?

I've looked at it and yes, seems we need it. I'll add it to AntlrTerminalPmdAdapter as well, as there we are missing the delegation to pmdNode - then we get also a nice reminder "!debug only!" for terminals.

Had to make change to getTextRegion() to avoid NullPointException.

FYI: This might be caused by parse errors (then there might be no stop token).

Actually, some nodes are created by the parser, even if there are no tokens/terminals. E.g. we always get a "packageHeader" node, even if there is no package statement. Or we always get a "importList" node, even if there are no import statements. In these cases, the stop token is null.

stokpop added 4 commits March 10, 2026 16:36

improve toString() output for Kotlin nodes, add Kotlin grammar parsin…

6469920

…g tests, now with failing Multiline Multi dollar issue in test

minimal fix in grammar to allow multi dollar prefix to multi line blocks

304d478

add all dollars to the MultiLineExpression so that can potentially be…

fd291f2

… used to match the number of tripple quote prefix dollars

Merge branch 'main' into fix-kotlin-grammar-minimal

179997b

stokpop added 2 commits March 11, 2026 08:55

fix dollar count in multiline refs and expressions

3c5227a

more test cases for multi dollar multiline refs and expressions, plus…

37fb82a

… fixes in grammar

adangel requested changes Mar 21, 2026

View reviewed changes

adangel changed the title ~~[kotlin] Fix kotlin grammar for parsing multiline multi dollar~~ [kotlin] Fix kotlin grammar for parsing multidollar interpolation Mar 21, 2026

stokpop added 4 commits March 30, 2026 17:05

added "informational" Kotlin version 2.2.0

9669433

added information about new Kotlin grammar in kotlin.md

f469829

Added "!debug only!" toString() to BaseAntlrNode and BaseAntrlInnerNo…

d06eba6

…de. Prevent NullPointerException in getTextRegion()

Cleanup of PmdKotlinParser for testing. Throws ParseException now ins…

a8e9807

…tead of just printing to standard err so test case fails when parsing fails.

stokpop requested a review from adangel March 30, 2026 21:32

stokpop added 2 commits March 31, 2026 09:01

Added Kotlin 2.2.0 to binary distribution IT

484f22e

Revert PmdKotlinParser to throw ParserException in non-test calls to …

ee2381c

…keep behaviour same. Use custom parser that throws ParserException for running tests and getting notified for parse issues.

adangel approved these changes Apr 17, 2026

View reviewed changes

stokpop and others added 3 commits April 17, 2026 18:25

Apply suggestions from code review

c9752e8

added refererence to kotlin feature and fix a typo Co-authored-by: Andreas Dangel <andreas.dangel@adangel.org>

Simplified KotlinParserMultiLineStringTest by using aseKotlinTreeDump…

50d4a0b

…Test for future regression tests

Merge branch 'main' into fix-kotlin-grammar-minimal

4d8aa9a

stokpop requested a review from adangel April 20, 2026 10:43

[core] antlr: delegate toString to pmdNode's toString

17afbf2

adangel added this to the 7.24.0 milestone Apr 23, 2026

adangel approved these changes Apr 23, 2026

View reviewed changes

adangel added a commit to stokpop/pmd that referenced this pull request Apr 23, 2026

[doc] Update release notes (pmd#6003, pmd#6497)

460a6d5

[doc] Update release notes (pmd#6003, pmd#6497)

1328d18

adangel force-pushed the fix-kotlin-grammar-minimal branch from 460a6d5 to 1328d18 Compare April 23, 2026 18:45

adangel merged commit 1436f7d into pmd:main Apr 23, 2026
2 checks passed

stokpop deleted the fix-kotlin-grammar-minimal branch May 15, 2026 11:32

coderabbitai Bot mentioned this pull request Jun 4, 2026

Implement fast login presets for DEBUG build variant ADT-VOLUNTEERS-CASE/MOBILE#81

Merged

3 tasks

Uh oh!

Conversation

stokpop commented Mar 10, 2026 • edited by adangel Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fix kotlin grammar for parsing multidollar interpolation

Related issues

Ready?

Background and discussion

Uh oh!

pmd-actions-helper Bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stokpop commented Mar 11, 2026

Uh oh!

adangel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stokpop commented Mar 25, 2026

Uh oh!

stokpop commented Mar 25, 2026

Uh oh!

adangel commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codacy-production Bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Not up to standards ⛔

Uh oh!

stokpop commented Apr 3, 2026

Uh oh!

stokpop commented Apr 16, 2026

Uh oh!

adangel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stokpop commented Apr 20, 2026

Uh oh!

adangel commented Apr 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

stokpop commented Mar 10, 2026 •

edited by adangel

Loading

pmd-actions-helper Bot commented Mar 10, 2026 •

edited

Loading

adangel commented Mar 27, 2026 •

edited

Loading

codacy-production Bot commented Apr 3, 2026 •

edited

Loading