perf(linter/plugins): use array buffer views for tokens#20301
Merged
graphite-app[bot] merged 1 commit intomainfrom Mar 12, 2026
Merged
Conversation
Member
Author
How to use the Graphite Merge QueueAdd either label to this PR to merge it via the merge queue:
You must have a Graphite account in order to use the merge queue. Sign up using this link. An organization admin has enabled the Graphite Merge Queue in this repository. Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue. This stack of pull requests is managed by Graphite. Learn more about stacking. |
Contributor
There was a problem hiding this comment.
Pull request overview
Optimizes token deserialization in apps/oxlint’s JS plugin layer by creating typed-array views that cover only the tokens region of the raw transfer buffer, reducing per-token offset arithmetic and simplifying the deserialization logic.
Changes:
- Introduces
Uint8Array/Uint32Arrayviews scoped to the tokens region of the transfer buffer. - Replaces
baseOffset + tokenIndex * TOKEN_SIZEaddressing with index-based addressing into the new views. - Refactors
deserializeTokenIntoto take a token index rather than an absolute byte position.
Member
Author
Merge activity
|
Small perf optimization to deserializing tokens. Instead of calculating position in the buffer of each token with `baseOffset + tokenIndex * TOKEN_SIZE`, create typed arrays which cover just to region of the buffer containing tokens data. Then it can be indexed into with just `tokenIndex * TOKEN_SIZE`. More importantly, this simplifies the code, which will make feasible another much larger optimization - all tokens methods can search for tokens directly in the buffer, without having to deserialize all the tokens to JS objects eagerly, as they do now.
2eacc60 to
77cdacc
Compare
camc314
pushed a commit
that referenced
this pull request
Mar 16, 2026
# Oxlint ### 🚀 Features - c95951f linter/plugins: Implement `sourceCode.markVariableAsUsed` (#20357) (overlookmotel) - 7a2a7d0 linter: Implement `n/handle-callback-err` rule (#19616) (Mikhail Baev) ### 🐛 Bug Fixes - f8fbd6e linter/plugins: Remove `hashbang` property from AST (#20365) (overlookmotel) - 6eb5b01 linter/prefer-await-to-then: Ignore Promise static methods (#20347) (camc314) - a4b61f7 linter: Remove `defineConfig` check (#20308) (camc314) - 3ad7f53 linter/explicit-module-boundary-types: False positive with satisfies expr (#20309) (camc314) - f547401 linter/no-unused-private-class-members: Treat switch discriminants as read (#20307) (camc314) - 1c07b3b diagnostics: Handle `WouldBlock` in stdout writes to prevent panic (#20295) (Boshen) ### ⚡ Performance - e4f7248 linter: Remove unnecessary clone of owned String in drain loop (#20388) (Boshen) - 4a67f1d linter: Eliminate Vec allocation in disable directive matching (#20387) (Boshen) - 618a598 linter/plugins: Add fast path for files with no comments (#20366) (overlookmotel) - b0125c5 linter/plugins: Deserialize comments without AST (#20364) (overlookmotel) - 9cd612f linter/plugins: Recycle comment objects (#20362) (overlookmotel) - bf442f8 linter/plugins: Cheaper `Token` creation (#20360) (overlookmotel) - 5474d0a semantic: V8-style walk-up reference resolution (#20292) (Boshen) - 7946eba linter/plugins: Avoid arguments spread and temp array when merging (#20318) (overlookmotel) - fc7cf8a linter/plugins: Pre-define less CFG merger functions (#20317) (overlookmotel) - 3b9eb28 linter/plugins: Streamline getting/creating visit fn mergers (#20319) (overlookmotel) - f04e850 linter/plugins: Inline binary search functions into call sites (#20312) (overlookmotel) - fe24afe linter/plugins: Apply replace globals TSDown plugin to JS files (#20305) (overlookmotel) - 77cdacc linter/plugins: Use array buffer views for tokens (#20301) (overlookmotel) - 910c941 linter/plugins: Reorder branches in `getTokenByRangeStart` (#20296) (overlookmotel) - af7674c linter/tokens: Avoid extra token value allocation (#20013) (camc314) ### 📚 Documentation - 24490b5 linter: Improve formatting for 80ish rules' docs. (#20411) (connorshea) - 3383523 linter: Improve `--tsconfig` flag docs (#20342) (camc314) # Oxfmt ### 🚀 Features - d22c443 oxfmt: Export `OxfmtConfig` type (#20275) (leaysgur) - a11ecff oxfmt/lsp: Respect `angular` language id as `.component.html` file (#20242) (Sysix) ### 🐛 Bug Fixes - ce65099 formatter: Preserve parentheses around as expression before private field access (#20419) (bab) - f908742 oxfmt: Revert #20326 partially (#20413) (leaysgur) - 4ef93ea formatter: Honor trailing ignore comments after list separators (#19925) (Andreas Lubbe) - 68fb0d0 oxfmt: Skip vite.config.ts which fails to import (#20326) (leaysgur) - 88ee826 oxfmt: Handle literalline for script-in-vue (#20130) (leaysgur) - 1c07b3b diagnostics: Handle `WouldBlock` in stdout writes to prevent panic (#20295) (Boshen) Co-authored-by: Boshen <1430279+Boshen@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Small perf optimization to deserializing tokens. Instead of calculating position in the buffer of each token with
baseOffset + tokenIndex * TOKEN_SIZE, create typed arrays which cover just to region of the buffer containing tokens data. Then it can be indexed into with justtokenIndex * TOKEN_SIZE.More importantly, this simplifies the code, which will make feasible another much larger optimization - all tokens methods can search for tokens directly in the buffer, without having to deserialize all the tokens to JS objects eagerly, as they do now.