V8intrinsic syntax plugin by JLHwung · Pull Request #10148 · babel/babel

JLHwung · 2019-06-29T21:48:37Z

Q	A
Fixed Issues?	Fixes #10104
Patch: Bug Fix?
Major: Breaking Change?
Minor: New Feature?	✔️
Tests Added + Pass?	✔️
Documentation PR Link
Any Dependency Changes?	No
License	MIT

Authored a new syntax plugin v8intrinsic to support V8 CallRuntime syntax

CallRuntime ::
  '%' Identifier Arguments

The original implementation in V8 parser peek() twice to check whether a modulo token is immediately followed by an identifier and a parenL token.

As in Babel lookahead() does not support multiple steps, here we implicitly track the execution branch inside parseSubscript to see if parseCallExpressionArguments is immediately called in the first round of parseExprSubscripts. If so we conclude that the modulo token is a valid V8Intrinsic marker.

It is also interesting to note that a V8Intrinsic call inside a new expression does not raise a SyntaxError in V8. i.e.

node --allow-natives-syntax -e "new %DebugPrint(null)"
// throws TypeError: (intermediate value) is not a constructor

node -e "new %DebugPrint(null)"
// throws SyntaxError: Unexpected token %

@mathiasbynens I don't think there are any V8 intrinsics supposed to work with new expression, aren't there?

If so, it should be reasonable to prevent v8intrisic from new expression, as is the behavior of this plugin. Otherwise we still need to take care on the new expression.

Action Items

Add v8intrinsic to babel-parser typings after updated ParsePlugin type definition #10259 gets merged.
Document updates: feat: add v8intrinsic language extensions website#2074

packages/babel-parser/src/plugins/v8intrinsic.js

babel-bot · 2019-06-29T21:56:17Z

Build successful! You can test your changes in the REPL here: https://babeljs.io/repl/build/11486/

nicolo-ribaudo

I was thinking that maybe we can remove this.state.V8IntrinsicStart and put all the parsing logic in a single method: we can overwrite parseExprAtom and add something like this:

If eat %%,
i. Parse an identifier
ii. Set that identifier's type to V8IntrinsicIdentifier
iii. If the upcoming token isn't (, throw an error
Else, return super.parseExprAtom.

This is similar to what we do for super() and import().

packages/babel-parser/src/plugins/v8intrinsic.js

packages/babel-parser/test/fixtures/v8intrinsic/expression/await-expression/input.js

JLHwung · 2019-07-01T03:01:28Z

I was thinking that maybe we can remove this.state.V8IntrinsicStart and put all the parsing logic in a single method

I agree, actually my first attack is on rewriting parseIdentifier but I find out it would mess up with the token position internal state. Apparently overriding parseExprAtom is way better than my previous approach. I have pushed the rewritten version.

Can you also add a test for yield?

Great observation! The parseYield will check if a token can startsExpr, if not it ends up with an empty yield statement, which fails under this case where % can actually start an expression. I would try to figure out a way to do this in the plugin land.

mathiasbynens · 2019-07-01T07:30:08Z

Thanks for your work on this!

@mathiasbynens I don't think there are any V8 intrinsics supposed to work with new expression, aren't there?

That's correct. Here's how V8 implements parsing support based on the flag.

packages/babel-parser/src/plugin-utils.js

packages/babel-parser/test/fixtures/v8intrinsic/_errors/not-in-call-expression/input.js

packages/babel-parser/test/fixtures/v8intrinsic/options.json

JLHwung · 2019-07-01T18:56:01Z

That's correct. Here's how V8 implements parsing support based on the flag.

The implementation of V8 parsing V8Intrinsic will lead to the fact that new %StringParseInt("42", 10) will not throw a syntax error, instead it throws a runtime type error.

However as I have rewritten the plugin and now it closely follow the way how V8 parse. I have added this newExpression test case to the success test suites.

JLHwung · 2019-07-01T21:42:50Z

I've talked to Nicolo offline a bit. Here we add { startsExpr: true } configuration to modulo token in parser/types since

It is only used in parseYield. The impact is under control.
There have been some plugin-defined tokens inside this file, i.e. braceBarL, braceBarR. It is acceptable to place plugin-defined token configurations here.

jridgewell

Awesome!

existentialism

👍

jridgewell · 2019-07-06T18:48:16Z

@nicolo-ribaudo: Is this waiting for a release, or can it be merged?

nicolo-ribaudo · 2019-07-06T21:24:52Z

We just released v7.5 so this is waiting for the next minor, but if it is urgent we could do it earlier than the usual time between two minor releases.

nicolo-ribaudo · 2019-09-03T15:52:23Z

packages/babel-types/src/definitions/misc.js

  },
 });
+
+defineType("V8IntrinsicIdentifier", {


I just realized that we should add this to the allowed callees of CallExpression

Implement V8 Intrinsic Syntax Extension. Here we check the execution branch inside the parseSubscript to make sure the V8IntrinsicIdentifier is immediately followed by a call expression.

per babel#10104 (comment)

This approach is identical to V8’s implementation. Move the test cases as the behaviour changes.

JLHwung commented Jun 29, 2019

View reviewed changes

packages/babel-parser/src/plugins/v8intrinsic.js Outdated Show resolved Hide resolved

nicolo-ribaudo reviewed Jun 30, 2019

View reviewed changes

packages/babel-parser/src/plugins/v8intrinsic.js Outdated Show resolved Hide resolved

packages/babel-parser/test/fixtures/v8intrinsic/expression/await-expression/input.js Show resolved Hide resolved

nicolo-ribaudo added pkg: parser PR: New Feature 🚀 A type of pull request used for our changelog categories labels Jun 30, 2019

mathiasbynens reviewed Jul 1, 2019

View reviewed changes

packages/babel-parser/src/plugin-utils.js Outdated Show resolved Hide resolved

mathiasbynens reviewed Jul 1, 2019

View reviewed changes

packages/babel-parser/test/fixtures/v8intrinsic/_errors/not-in-call-expression/input.js Outdated Show resolved Hide resolved

mathiasbynens reviewed Jul 1, 2019

View reviewed changes

packages/babel-parser/test/fixtures/v8intrinsic/options.json Outdated Show resolved Hide resolved

nicolo-ribaudo approved these changes Jul 2, 2019

View reviewed changes

jridgewell approved these changes Jul 4, 2019

View reviewed changes

nicolo-ribaudo added the PR: Ready to be Merged A pull request with already two approvals, but waiting for the next minor release label Jul 4, 2019

nicolo-ribaudo added this to the v7.6.0 milestone Jul 4, 2019

existentialism approved these changes Jul 6, 2019

View reviewed changes

jridgewell mentioned this pull request Jul 24, 2019

Print V8IntrinsicIdentifier JLHwung/babel#1

Merged

jridgewell force-pushed the v8intrinsic-syntax-plugin branch from 096c337 to 6bba3c9 Compare July 24, 2019 04:53

jridgewell mentioned this pull request Jul 24, 2019

Add V8IntrinsicIdentifier printing JLHwung/babel#2

Merged

JLHwung mentioned this pull request Jul 25, 2019

feat: add v8intrinsic language extensions babel/website#2074

Merged

JLHwung force-pushed the v8intrinsic-syntax-plugin branch from 3c4fd6c to e16be2e Compare July 29, 2019 15:20

JLHwung force-pushed the v8intrinsic-syntax-plugin branch 2 times, most recently from 5c75a05 to a0dcb31 Compare August 21, 2019 02:25

nicolo-ribaudo reviewed Sep 3, 2019

View reviewed changes

JLHwung force-pushed the v8intrinsic-syntax-plugin branch from 3333b03 to 67d6c2a Compare September 3, 2019 21:28

JLHwung added 2 commits September 6, 2019 17:38

feat: v8intrinsic syntax plugin

f4de92e

Implement V8 Intrinsic Syntax Extension. Here we check the execution branch inside the parseSubscript to make sure the V8IntrinsicIdentifier is immediately followed by a call expression.

feat: disable combining placeholders and v8intrisic

ebebd5f

per babel#10104 (comment)

JLHwung and others added 10 commits September 6, 2019 17:38

refactor: parse v8 intrinsic in parseExprAtom

cb51068

This approach is identical to V8’s implementation. Move the test cases as the behaviour changes.

fix: plugin-name typo

ca1145d

test: add yield-expression test case

d3964df

feat: require startsExpr on modulo for v8intrinsic

c58e96d

perf: skip eof and braceR check as they must return false

2072607

feat: add v8intrinsic to parser typings

04a4ea2

Print V8IntrinsicIdentifier

aa81c50

Add generated type helpers

3f29611

fix: incorrect type definition

f2d1dca

fix: allow V8IntrinsicIdentifier to be callee

c1a3cbf

nicolo-ribaudo force-pushed the v8intrinsic-syntax-plugin branch from 67d6c2a to c1a3cbf Compare September 6, 2019 15:42

nicolo-ribaudo merged commit da0af5f into babel:master Sep 6, 2019

JLHwung deleted the v8intrinsic-syntax-plugin branch September 6, 2019 18:42

snyk-bot mentioned this pull request Oct 24, 2019

[Snyk] Upgrade @babel/runtime from 7.4.4 to 7.6.3 Matthelonianxl/drizzle#9

Open

snyk-bot mentioned this pull request Nov 1, 2019

[Snyk] Upgrade @babel/core from 7.2.2 to 7.6.4 nbv1982/material-ui#5

Open

snyk-bot mentioned this pull request Nov 12, 2019

[Snyk] Upgrade @babel/runtime from 7.1.5 to 7.6.3 ajesse11x/Parse-SDK-JS#1

Open

This was referenced Nov 28, 2019

[Snyk] Upgrade @babel/runtime from 7.0.0-beta.42 to 7.7.2 ali8889/mist#4

Open

[Snyk] Upgrade @babel/runtime from 7.5.5 to 7.7.4 dyna-dot/CyberChef#3

Open

snyk-bot mentioned this pull request Dec 5, 2019

[Snyk] Upgrade @babel/runtime from 7.3.1 to 7.7.4 dyna-dot/bit-javascript#5

Open

lock bot added the outdated A closed issue/PR that is archived due to age. Recommended to make a new issue label Dec 6, 2019

lock bot locked as resolved and limited conversation to collaborators Dec 6, 2019

Uh oh!

Conversation

JLHwung commented Jun 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Action Items

Uh oh!

Uh oh!

babel-bot commented Jun 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nicolo-ribaudo left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

JLHwung commented Jul 1, 2019

Uh oh!

mathiasbynens commented Jul 1, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JLHwung commented Jul 1, 2019

Uh oh!

JLHwung commented Jul 1, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jridgewell left a comment

Choose a reason for hiding this comment

Uh oh!

existentialism left a comment

Choose a reason for hiding this comment

Uh oh!

jridgewell commented Jul 6, 2019

Uh oh!

nicolo-ribaudo commented Jul 6, 2019

Uh oh!

nicolo-ribaudo Sep 3, 2019

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

JLHwung commented Jun 29, 2019 •

edited

Loading

babel-bot commented Jun 29, 2019 •

edited

Loading

nicolo-ribaudo left a comment •

edited

Loading

JLHwung commented Jul 1, 2019 •

edited

Loading