Skip to content

[swift] Parsing errors encountered with emoji #2698

@ghost

Description

Affects PMD Version:
6.26.0
[CPD]

Description:

I am using cpd to run a folder contains swift files. Following is the command I run, and errors encountered.

Exception Stacktrace:

net.sourceforge.pmd.lang.ast.TokenMgrError: Lexical error in file ServerUserService.swift at line 80, column 16.  Encountered: token recognition error at: '👤'
	at net.sourceforge.pmd.cpd.AntlrTokenizer.tokenize(AntlrTokenizer.java:44)
	at net.sourceforge.pmd.cpd.CPD.addAndThrowLexicalError(CPD.java:144)
	at net.sourceforge.pmd.cpd.CPD.add(CPD.java:139)
	at net.sourceforge.pmd.cpd.CPD.add(CPD.java:108)
	at net.sourceforge.pmd.cpd.CPD.add(CPD.java:70)
	at net.sourceforge.pmd.cpd.CPD.addDirectory(CPD.java:80)
	at net.sourceforge.pmd.cpd.CPD.addRecursively(CPD.java:65)
	at net.sourceforge.pmd.cpd.CPDCommandLineInterface.addSourcesFilesToCPD(CPDCommandLineInterface.java:134)
	at net.sourceforge.pmd.cpd.CPDCommandLineInterface.addSourceFilesToCPD(CPDCommandLineInterface.java:114)
	at net.sourceforge.pmd.cpd.CPDCommandLineInterface.main(CPDCommandLineInterface.java:87)
	at net.sourceforge.pmd.cpd.CPD.main(CPD.java:179)
Caused by: LexerNoViableAltException('👤')
	at org.antlr.v4.runtime.atn.LexerATNSimulator.failOrAccept(LexerATNSimulator.java:312)
	at org.antlr.v4.runtime.atn.LexerATNSimulator.execATN(LexerATNSimulator.java:233)
	at org.antlr.v4.runtime.atn.LexerATNSimulator.match(LexerATNSimulator.java:117)
	at org.antlr.v4.runtime.Lexer.nextToken(Lexer.java:141)
	at net.sourceforge.pmd.lang.antlr.AntlrTokenManager.getNextTokenFromAnyChannel(AntlrTokenManager.java:47)
	at net.sourceforge.pmd.lang.antlr.AntlrTokenManager.getNextToken(AntlrTokenManager.java:40)
	at net.sourceforge.pmd.cpd.token.internal.BaseTokenFilter.getNextToken(BaseTokenFilter.java:45)
	at net.sourceforge.pmd.cpd.AntlrTokenizer.tokenize(AntlrTokenizer.java:39)
	... 10 more
2020-08-05 18:45:06.480 CPDSwift[33372:1842414] Error occured on Parsing CPD XML output file : The operation couldn’t be completed. (com.google.GDataXML error -1.)
Command PhaseScriptExecution failed with a nonzero exit code

Code Sample demonstrating the issue:

if let userToken = userToken {
     print("👤 | User > Token: \(userToken)")
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    a:bugPMD crashes or fails to analyse a file.in:grammarAbout the grammar of a lexer or parser, eg, a parse/lex exception

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions