Refactor private name tokenizing#13256
Conversation
|
This pull request is automatically built and testable in CodeSandbox. To see build info of the built libraries, click here or the icon next to each commit SHA. Latest deployment of this branch, based on commit 52f533d:
|
|
Build successful! You can test your changes in the REPL here: https://babeljs.io/repl/build/45853/ |
806d076 to
97a2f60
Compare
| token.type = "Numeric"; | ||
| token.value = `${token.value}n`; | ||
| } else if (type === tt.privateName) { | ||
| token.type = "PrivateIdentifier"; |
There was a problem hiding this comment.
We convert tt.privateName to PrivateIdentifier, which aligns to eslint/js#486
Note that although this PR merges tt.hash and tt.name to tt.privateName, this behaviour will be not observed by @babel/eslint-parsers because of the compat layer. However if we run with BABEL_8_BREAKING=true, the eslint parser will see tt.privateName, instead of breaking tt.privateName, we align it to the new espree behaviour.
3f161e3 to
1f65f5e
Compare
| this.finishToken(tt.bracketHashL); | ||
| } | ||
| this.state.pos += 2; | ||
| } else if (isIdentifierStart(next) || next === charCodes.backslash) { |
There was a problem hiding this comment.
Can you explain the backslash? I'm confused why #\ would be valid.
There was a problem hiding this comment.
It's for the escaped private names: #\u0061.
|
Thank you |
Co-authored-by: Justin Ridgewell <justin@ridgewell.name>
This PR overhauls how Babel parser tokenizes the privateIdentifier
#name. Currently#nameis tokenized ashashandname, in this PR we merge these two tokens into a newprivateNametoken whosevalueholds the String value of private identifier (without hash). We observe performance gain up to 18%.This PR is based on the observation that Babel always does a lookahead when tokenizing
#, so we can determine early if an identifier start is following a#, and avoid extra read of the leading identifier character.By merging
#andnamewe also avoids the tokenizer context update hooks forname-type tokens. We don't need to checkof,functionsandclassfor private identifiers anyway.Since we expose
tokenswhenoptions.tokensistrue, we add a compat routine fortt.privateNamewhich essentially undo the merging, hopefully we can remove it in Babel 8.Note that Acorn adopts the same approach, which means it is likely that
@babel/eslint-parserwill have to merge#andnamefor older Babel versions. By merging tokens in@babel/parserwe also do a favour for the@babel/eslint-parser.