Bug Report

Current and Expected Behavior

Currently @babel/parser allows numeric separators (i.e. the _ in 123_456) inside unicode escape sequences, both in the \u{...} and \uXXXX forms. They should be disallowed.

Input Code

• REPL or Repo link if applicable:

"\u{1F_639}";
"\u12_34";
"\x1_2";

repl

Babel Configuration (.babelrc, package.json, cli command)

You may need to enable the numericSeparator parser plugin in order to debug this issue.

Environment

• Babel version(s): v7.6

Possible Solution

This is a bug in the Tokenizer, the part of @babel/parser which transforms the input source code into a lazy list of tokens (e.g. strings, numbers, identifiers, punctuators, ...).

The problem is that readInt, when the numericSeparator plugin is enabled, always allows numeric separators.
When readInt is called by readHexChar, if it finds _ it should throw something like Numeric separators are not allowed inside unicode escape sequences.

If anyone wants to work on this issue, please leave a comment below! If you need any help, plase ask me (or to any other Babel team member) on Slack 🙂

If you don't know how to clone Babel, follow these steps: (you need to have make and yarn available on your machine).

1. Fork the repo
2. Run git clone https://github.com/<YOUR_USERNAME>/babel.git && cd babel
3. Run yarn && make bootstrap
4. Wait ⏳
5. Run make watch (or make build whenever you change a file)
6. Add some tests (at least one for \u{...}, \uXXXX and one for \xXX) in packages/babel-parser/test/fixtures/experimental/numeric-separator. Please don't call it "invalid-*", but use a descriptive name 😛
   • If you want an example of a test for an error, you can check one of the invalid-* tests in that folder.
7. Update the code!
8. yarn jest parser to run the tests
9. If it is working, run make test to run all the tests
10. Note: this change will likely also impact the results of running the Test262 suite on Babel.
    • Before running those tests you need to run make bootstrap-test262 to download them
    • Use make test-test262 to run them
    • If some tests which were previously failing now pass, you can run make test-test262-update-whitelist
11. Run git push and open a PR!