perf(linter/plugins): reduce allocations for regex tokens#20479
Merged
graphite-app[bot] merged 1 commit intomainfrom Mar 21, 2026
Merged
Conversation
This was referenced Mar 17, 2026
Member
Author
This was referenced Mar 17, 2026
66d4271 to
208bee0
Compare
9e16a5a to
9254068
Compare
This was referenced Mar 18, 2026
208bee0 to
5374681
Compare
9254068 to
1c54851
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes regex-token bookkeeping in apps/oxlint/src-js/plugins/tokens.ts by replacing an Array of token references with a reusable Uint32Array of token indexes, aiming to reduce per-file allocations and GC overhead during linting.
Changes:
- Track regex tokens via
tokensWithRegexIndexes: Uint32Array+activeTokensWithRegexCountinstead ofToken[]. - Reuse
regexObjectsentries keyed byactiveTokensWithRegexCount, and cleartoken.regexduringresetTokens()using the stored indexes.
1c54851 to
7c469ec
Compare
This was referenced Mar 18, 2026
7c469ec to
3426e69
Compare
5374681 to
2090432
Compare
3426e69 to
dfd5e3f
Compare
2090432 to
94eb885
Compare
Contributor
Merge activity
|
Optimize storing regex tokens so they can have their `regex` property set back to `undefined` at the end of linting a file. Previously we pushed the `Token`s into an `Array` and then reset the array with `tokensWithRegex.length = 0;` at the end. This had a problem: When you set an array's length to 0, V8 releases the whole backing allocation, which means that on the next file, it has to allocate again. Worse, that new allocation is made in "new space", and if there's a lot of object creation in rules while linting the file, it will be copied during minor GC, and may even get graduated to "old space". All of that is expensive. Instead, use a `Uint32Array` to store indexes, for the same reasons as in #20477.
94eb885 to
4ee80ac
Compare
dfd5e3f to
9c7a267
Compare
graphite-app bot
pushed a commit
that referenced
this pull request
Mar 21, 2026
… accessed `loc` (#20480) Similar to #20479. `tokensWithLoc` and `commentsWithLoc` contain tokens/comments on which `loc` property has been accessed. Previously these arrays were grown with `.push(token)`, and shrunk at end of linting file with `tokensWithLoc.length = 0;`. Problem with that is that setting length to 0 frees the array's backing allocation, which means it has to reallocate again on next file when a token's `loc` is accessed. Instead, never shrink these arrays, and track the active length in separate variables. After warm-up over first batch of files, these arrays will graduate to "old space" and just sit there without further allocations. Unlike #20479, we don't use `Uint32Array`s as `loc` is calculated lazily, and `Token`'s / `Comment`s don't know what their index is in the `cachedTokens` / `cachedComments` arrays. Adding an `#index` field to `Token` and `Comment` would bloat every instance of these classes by 8 bytes.
Base automatically changed from
om/03-17-perf_linter_plugins_remove_bounds_checks_on_regex_tokens
to
main
March 21, 2026 12:51
This was referenced Mar 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Optimize storing regex tokens so they can have their
regexproperty set back toundefinedat the end of linting a file. Previously we pushed theTokens into anArrayand then reset the array withtokensWithRegex.length = 0;at the end.This had a problem: When you set an array's length to 0, V8 releases the whole backing allocation, which means that on the next file, it has to allocate again. Worse, that new allocation is made in "new space", and if there's a lot of object creation in rules while linting the file, it will be copied during minor GC, and may even get graduated to "old space". All of that is expensive.
Instead, use a
Uint32Arrayto store indexes, for the same reasons as in #20477.