perf(parser): use peek_token instead of lookahead on the modifier path#22842
Merged
Conversation
Merging this PR will not alter performance
Comparing 
|
Member
Author
Merge activity
|
#22842) ## What The modifier-disambiguation helpers (`get_modifier`, `next_token_can_follow_modifier`, `next_token_is_on_same_line_and_can_follow_modifier`) used the parser-level `lookahead`, which performs a full `checkpoint` — lexer checkpoint **plus** saving `cur_token`, `prev_token_end`, `errors.len()` and taking `fatal_error` — then a `bump_any`, then a `rewind`, just to inspect the single token after a modifier keyword. This replaces them with the lexer-level `peek_token`, which does the same single re-lex without the parser-field save/restore. - `can_follow_modifier` / `can_follow_get_or_set_keyword` now take a `Kind`, so the peeked token's kind is passed directly (no duplicated helpers to keep in sync). - `get_modifier_worker` is removed, folded into `get_modifier`. Net result is **less code** as well as fewer instructions per modifier check. ## Correctness Semantics-preserving: - The speculative `bump_any` in the old form could push an escaped-keyword error, but `lookahead`'s rewind always discarded it; the real error still fires on the non-speculative bump the caller performs once the check returns `true`. - The `is_on_new_line` flag read from the peeked token is identical to the post-bump `cur_token().is_on_new_line()` the old code read — both refer to the same lexed token (`peek_token` = checkpoint + `next_token` + rewind, and the lexer sets the newline flag while lexing that token). Verified: - `cargo test -p oxc_parser`: 69/69 pass - `cargo coverage -- parser`: zero conformance snapshot diff (Test262 / TypeScript / Babel / estree) - `just fmt` + `cargo clippy -p oxc_parser --all-features --all-targets`: clean ## Performance Measured via an interleaved A/B (pre-built binaries, cooldowns) validated against a parse-invariant control (`estree/checker.ts`, which times serialization only and stays flat). The changed build was faster in every measured round on declaration/member-dense input — conservatively **~0.5–1% on `binder.ts`** — and noise-level on type-dominated files (`cal.com.tsx`, `kitchen-sink.tsx`), as expected since those are dominated by TS type parsing rather than modifier scanning. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
9426980 to
9e741c2
Compare
camc314
pushed a commit
that referenced
this pull request
Jun 1, 2026
### 🚀 Features - 9c71f2e ast, codegen, formatter: Add `WithClauseKeyword::as_str` helper and use it (#22791) (camc314) ### 🐛 Bug Fixes - cf5769c parser: Reject TypeScript declarations in single-statement context (#22827) (Boshen) - c360fb6 parser: Reject generators in ambient contexts and overload signatures (TS1221/TS1222) (#22848) (Boshen) - cc60d8d parser: Reject invalid index signature parameter types (TS1268/TS1337) (#22852) (Boshen) - 3d13e29 parser: Reject `declare` in an already-ambient context (TS1038) (#22850) (Boshen) - 5152854 parser: Reject statements in ambient contexts (TS1036) (#22849) (Boshen) - 4f9afc5 parser: Reject `export as namespace` inside a namespace body (TS1316) (#22846) (Boshen) - 2eafea6 parser: Reject function implementations in ambient contexts (TS1183) (#22845) (Boshen) - c645615 parser: Reject incompatible class member modifiers (#22843) (Boshen) - 276b78b parser: Reject module-referencing imports/exports in a namespace body (#22836) (Boshen) - 842ed1c parser: Parse `class implements` with `implements` as the class name (#22801) (Boshen) ### ⚡ Performance - 7e3a567 parser: Reuse cached token kind in delimited-list loops (#22841) (Boshen) - 9e741c2 parser: Use peek_token instead of lookahead on the modifier path (#22842) (Boshen) - 9e496a7 semantic: Defer declare lookup for empty accessors (#22810) (camc314)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The modifier-disambiguation helpers (
get_modifier,next_token_can_follow_modifier,next_token_is_on_same_line_and_can_follow_modifier) used the parser-levellookahead, which performs a fullcheckpoint— lexer checkpoint plus savingcur_token,prev_token_end,errors.len()and takingfatal_error— then abump_any, then arewind, just to inspect the single token after a modifier keyword.This replaces them with the lexer-level
peek_token, which does the same single re-lex without the parser-field save/restore.can_follow_modifier/can_follow_get_or_set_keywordnow take aKind, so the peeked token's kind is passed directly (no duplicated helpers to keep in sync).get_modifier_workeris removed, folded intoget_modifier.Net result is less code as well as fewer instructions per modifier check.
Correctness
Semantics-preserving:
bump_anyin the old form could push an escaped-keyword error, butlookahead's rewind always discarded it; the real error still fires on the non-speculative bump the caller performs once the check returnstrue.is_on_new_lineflag read from the peeked token is identical to the post-bumpcur_token().is_on_new_line()the old code read — both refer to the same lexed token (peek_token= checkpoint +next_token+ rewind, and the lexer sets the newline flag while lexing that token).Verified:
cargo test -p oxc_parser: 69/69 passcargo coverage -- parser: zero conformance snapshot diff (Test262 / TypeScript / Babel / estree)just fmt+cargo clippy -p oxc_parser --all-features --all-targets: cleanPerformance
Measured via an interleaved A/B (pre-built binaries, cooldowns) validated against a parse-invariant control (
estree/checker.ts, which times serialization only and stays flat). The changed build was faster in every measured round on declaration/member-dense input — conservatively ~0.5–1% onbinder.ts— and noise-level on type-dominated files (cal.com.tsx,kitchen-sink.tsx), as expected since those are dominated by TS type parsing rather than modifier scanning.🤖 Generated with Claude Code