Merged
Conversation
Collaborator
|
Build successful! You can test your changes in the REPL here: https://babeljs.io/repl/build/48769/ |
|
This pull request is automatically built and testable in CodeSandbox. To see build info of the built libraries, click here or the icon next to each commit SHA. Latest deployment of this branch, based on commit d34fb4b:
|
| // For performance the token type helpers depend on the following declarations order. | ||
| // When adding new token types, please also check if the token helpers needs update. | ||
|
|
||
| export const types: { [name: string]: TokenType } = { |
Member
There was a problem hiding this comment.
Can we just export this as tt, since types won't be replaced by the plugin?
Contributor
Author
|
I mark this PR as |
Co-authored-by: Nicolò Ribaudo <nicolo.ribaudo@gmail.com>
7c9237d to
6a14732
Compare
6a14732 to
d34fb4b
Compare
nicolo-ribaudo
approved these changes
Sep 16, 2021
existentialism
approved these changes
Sep 17, 2021
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Token type is the output of tokenizer, a digested interpretation of the source code. A token type is an object including a few meta infos, e.g. whether a token is an assignment operator (
isAssign), or the precedence if it is a binary operator (binop). While accessing info from the token type directly is convenient, it introduces extra object access instructions.In this PR we store the token type as a number. By doing so we can eliminate all the
tt.*usage at build time, replacing them to a number, which is similar to how we handle thecharCodes.*usage: Thebabel-plugin-transform-charcodesreplaces them by the actual numeric values.Because the token types are now a continuum of integers, some queries can be simplified to a in-range call if we carefully align the declarations of token types. For example, in this PR,
token.isLoopis reimplemented as:the ordinality of integers allow us to reduce multiple branch condition to two branch condition*.
Since we export the tokens via
* Technically an in-range call can be further reduced to a single branch condition:tokTypes, an extra compatibility layer is introduced so the parser can work on internal number token types while emitting the object token types whentokensis enabled.But I didn't observe actual performance improvements. Since the JavaScript does not have an unsigned u32 type, the compiler can not actually use u32 here.