Skip to content

perf(parser): peek instead of lookahead for yield disambiguation#23071

Merged
graphite-app[bot] merged 1 commit into
mainfrom
perf/parser-yield-peek
Jun 8, 2026
Merged

perf(parser): peek instead of lookahead for yield disambiguation#23071
graphite-app[bot] merged 1 commit into
mainfrom
perf/parser-yield-peek

Conversation

@Boshen

@Boshen Boshen commented Jun 7, 2026

Copy link
Copy Markdown
Member

Outside a generator, yield is an IdentifierReference unless the next token (on the same line) can start a YieldExpression operand — a single-token decision. is_yield_expression ran a full lookahead (checkpoint + speculative sub-parse + rewind) for it.

With is_await = false, the worker reduces to exactly bump; !next.is_on_new_line() && next.kind().is_after_await_or_yield(), so peek the one token instead — mirroring the existing is_unambiguous_await.

The worker next_token_is_identifier_or_keyword_or_literal_on_same_line was only reachable from here (the await path uses is_unambiguous_await, and its is_await = true branch was already dead), so it is removed.

Behavior-preserving:

  • cargo coverage -- parser — Test262 / Babel / TypeScript / ESTree snapshots byte-identical (no .snap changes).
  • just allocs — allocation snapshot unchanged.
  • Full parser output byte-identical to baseline on a yield edge-case fixture (generator/non-generator, newline-after-yield, yield /re/).

🤖 Generated with Claude Code

@github-actions github-actions Bot added the A-parser Area - Parser label Jun 7, 2026
@codspeed-hq

codspeed-hq Bot commented Jun 7, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 52 untouched benchmarks
⏩ 14 skipped benchmarks1


Comparing perf/parser-yield-peek (8199b98) with main (883dfc1)

Open in CodSpeed

Footnotes

  1. 14 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@Boshen Boshen self-assigned this Jun 7, 2026
@Boshen Boshen marked this pull request as draft June 7, 2026 13:57
@Boshen Boshen marked this pull request as ready for review June 8, 2026 12:57
@Boshen Boshen added the 0-merge Merge with Graphite Merge Queue label Jun 8, 2026

Boshen commented Jun 8, 2026

Copy link
Copy Markdown
Member Author

Merge activity

)

Outside a generator, `yield` is an `IdentifierReference` unless the next token (on the same line) can start a `YieldExpression` operand — a single-token decision. `is_yield_expression` ran a full `lookahead` (checkpoint + speculative sub-parse + rewind) for it.

With `is_await = false`, the worker reduces to exactly `bump; !next.is_on_new_line() && next.kind().is_after_await_or_yield()`, so peek the one token instead — mirroring the existing `is_unambiguous_await`.

The worker `next_token_is_identifier_or_keyword_or_literal_on_same_line` was only reachable from here (the `await` path uses `is_unambiguous_await`, and its `is_await = true` branch was already dead), so it is removed.

Behavior-preserving:
- `cargo coverage -- parser` — Test262 / Babel / TypeScript / ESTree snapshots byte-identical (no `.snap` changes).
- `just allocs` — allocation snapshot unchanged.
- Full parser output byte-identical to baseline on a `yield` edge-case fixture (generator/non-generator, newline-after-`yield`, `yield /re/`).

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@graphite-app graphite-app Bot force-pushed the perf/parser-yield-peek branch from 8199b98 to 7d89909 Compare June 8, 2026 12:58
@graphite-app graphite-app Bot merged commit 7d89909 into main Jun 8, 2026
30 checks passed
@graphite-app graphite-app Bot removed the 0-merge Merge with Graphite Merge Queue label Jun 8, 2026
@graphite-app graphite-app Bot deleted the perf/parser-yield-peek branch June 8, 2026 13:02
Boshen added a commit that referenced this pull request Jun 8, 2026
### 💥 BREAKING CHANGES

- ee4dc73 ast: [**BREAKING**] Add `#[non_exhaustive]` to AST nodes
(#23046) (overlookmotel)
- 4c35362 ast: [**BREAKING**] Add
`AstBuilder::template_element_escape_raw` and
`template_element_escape_raw_with_lone_surrogates` methods (#23047)
(overlookmotel)

### 🚀 Features

- b846ab2 react_compiler: Integrate the Rust port of the React Compiler
(#22942) (Boshen)
- 5b8dd68 parser: Report TS1255 for invalid class definite assertions
(#22917) (camc314)
- 85efabf semantic: Make building the class table optional, off by
default (#22862) (Boshen)

### 🐛 Bug Fixes

- 556acdc codegen: Parenthesize TS-cast assignment targets (#23112)
(Boshen)
- 37169ff codegen: Don't emit space between postfix `--` and `>` when
minifying (#23036) (Boshen)
- a4b1bf7 codegen: Drop redundant whitespace in minified TypeScript
output (#23038) (Boshen)
- cf53285 parser: Report reserved type-declaration names in the parser
(#23035) (Boshen)
- 4e44969 ast: Fix UB in `escape_template_element_raw` (#23052)
(overlookmotel)
- c543154 parser: Report comma operator in JSX expression in the parser
(#23030) (Boshen)
- 325c94f codegen: Tighten conditional-type and constructor-type
whitespace when minifying (#23033) (Boshen)
- 95dd3a2 parser: Report `import type` alias to a non-external reference
in the parser (#23032) (Boshen)
- 90180b8 codegen: Drop space after `:` in function return type when
minifying (#23028) (Boshen)
- 6da876e parser: Report `abstract` private class field in the parser
(#23029) (Boshen)
- 28467ce codegen: Don't emit space before a postfix update operand when
minifying (#23027) (Boshen)
- cb29926 codegen: Drop redundant space after `export default` when
minifying (#23024) (Boshen)
- 62965ae codegen: Drop redundant space after `else` when minifying
(#23025) (Boshen)
- 989230a parser: Report compound assignment to non-simple target in the
parser (#23022) (Boshen)
- 06f367c parser: Report `super.#field` private access in the parser
(#23014) (Boshen)
- 184edef codegen: Print space before `const`/`declare` enum modifier
(#23013) (Boshen)
- 4d722e0 parser: Report duplicate switch `default` clause in the parser
(#23012) (Boshen)
- 597ed85 codegen: Parenthesize `let`/`async` for-of head target
(#23008) (Boshen)
- 8b631bf codegen: Remove stray space before mapped type value colon
(#23010) (Boshen)
- c08407e codegen: Don't over-parenthesize `in` inside an arrow in a
for-init (#23009) (Boshen)
- 600cd6f codegen: Parenthesize lower-precedence
`TSInstantiationExpression` operand (#23007) (Boshen)
- 187e1a5 codegen: Don't leak space after comment-only JSX expression
container (#23006) (Boshen)
- 294c473 codegen: Don't over-parenthesize `TSTypeAssertion` operand
(#23004) (Boshen)
- 786d96f codegen: Give `TSTypeAssertion` unary precedence (#23002)
(Boshen)
- 1295882 parser: Report `new.target` and `import.meta` syntax errors in
the parser (#23003) (Boshen)
- d727b6b codegen: Parenthesize `await` expression as base of `**`
(#23001) (Boshen)
- 67dfa08 codegen: Keep parentheses around `new` callees containing a
call (#22997) (Boshen)
- 17e7cf3 parser: Disallow unerasable `as`/`satisfies` assertions
(#22986) (Boshen)
- beb46d3 parser: Commit to module goal on decorated exports (#22941)
(Boshen)
- 49e63f7 isolated-declarations: Require annotations for satisfies
initializers (#22898) (camc314)
- 8c93601 isolated-declarations: Allow unknown enum initializer in
non-const enum (#22900) (camc314)

### ⚡ Performance

- 7d89909 parser: Peek instead of lookahead for yield disambiguation
(#23071) (Boshen)
- bf872f0 parser: Skip arrow lookahead for a parenthesized literal
(#23070) (Boshen)
- d19fc54 parser: Guard type-argument speculation behind an angle-token
check (#23069) (Boshen)
- 8eb5507 parser: Skip redundant member-rest re-scan on call entry
(#23068) (Boshen)
- 883dfc1 parser: Skip parse_call_expression_rest when no call follows
(#23063) (Boshen)
- b171153 parser: Peek before the await-using lookahead (#23059)
(Boshen)
- 56f21bd parser: Use peek_token for the TS `asserts` type predicate
(#23058) (Boshen)
- 68805ac parser: Use peek_token instead of checkpoint/rewind for
single-token decisions (#23056) (Boshen)
- 1f9d8eb ast: `AstBuilder::template_element_escape_raw` avoid
allocation if no escape required (#23053) (overlookmotel)
- 502b04d semantic: Move cold function redeclaration handling into
`#[cold]` function (#22973) (overlookmotel)

### 📚 Documentation

- 275d318 napi/minifier: Point `target` to oxc docs (#23102) (camc314)

Co-authored-by: Boshen <1430279+Boshen@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-parser Area - Parser

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant