Skip to content

perf(napi/parser, linter/plugins): use Int32Array in raw transfer deserializer#21132

Merged
graphite-app[bot] merged 1 commit intomainfrom
om/04-07-perf_napi_parser_linter_plugins_use_int32array_in_raw_transfer_deserializer
Apr 7, 2026
Merged

perf(napi/parser, linter/plugins): use Int32Array in raw transfer deserializer#21132
graphite-app[bot] merged 1 commit intomainfrom
om/04-07-perf_napi_parser_linter_plugins_use_int32array_in_raw_transfer_deserializer

Conversation

@overlookmotel
Copy link
Copy Markdown
Member

@overlookmotel overlookmotel commented Apr 7, 2026

Use int32 (Int32Array) instead of uint32(Uint32Array) for getting data from buffer in raw transfer deserializer. The buffer is 2 GiB in size, so all offsets within the buffer can be represented as a u31 which can be stored in an i32 without any loss, and with no risk of being interpreted as negative numbers.

Same as in #21129, the advantage of fetching offsets from an Int32Array is that V8 statically knows the value can be stored in an SMI, its native integer type, without any range checks.

Copy link
Copy Markdown
Member Author

overlookmotel commented Apr 7, 2026


How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • 0-merge - adds this PR to the back of the merge queue
  • hotfix - for urgent changes, fast-track this PR to the front of the merge queue

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Apr 7, 2026

Merging this PR will not alter performance

✅ 48 untouched benchmarks
⏩ 3 skipped benchmarks1


Comparing om/04-07-perf_napi_parser_linter_plugins_use_int32array_in_raw_transfer_deserializer (56d49d6) with main (c5c8c03)2

Open in CodSpeed

Footnotes

  1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on om/04-07-refactor_napi_parser_linter_plugins_add_int32_buffer_view (73f8ddd) during the generation of this report, so main (c5c8c03) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@overlookmotel overlookmotel marked this pull request as ready for review April 7, 2026 15:54
@overlookmotel overlookmotel requested a review from camc314 as a code owner April 7, 2026 15:54
Copilot AI review requested due to automatic review settings April 7, 2026 15:54
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the raw-transfer JS deserialization path (N-API parser + oxlint JS side) to read buffer offsets/lengths via Int32Array (int32) instead of Uint32Array (uint32), aiming to improve V8 performance by keeping values in SMI form.

Changes:

  • Update the raw-transfer deserializer generator to emit int32[...] reads for offsets/lengths (and adjust related docs/constants).
  • Regenerate N-API parser deserializers to use int32 and clear int32 references in resetBuffer.
  • Update oxlint’s JS parsing/source initialization code to use buffer.int32 for program/source offsets.

Reviewed changes

Copilot reviewed 5 out of 16 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tasks/ast_tools/src/generators/raw_transfer.rs Generator now emits int32 reads for data pointer, strings, vectors, boxes, and some Option niche checks.
napi/parser/src-js/generated/deserialize/js.js Switch raw-transfer deserializer to int32 for pointer/length reads and resetBuffer cleanup.
napi/parser/src-js/generated/deserialize/js_range.js Same int32 switch for range-enabled variant.
napi/parser/src-js/generated/deserialize/js_range_parent.js Same int32 switch for range+parent variant.
napi/parser/src-js/generated/constants.js Doc comment updated to reflect Int32Array indexing for DATA_POINTER_POS_32.
napi/parser/bench.bench.js Bench now reads programPos using buffer.int32.
crates/oxc_ast/src/serialize/js.rs Embedded raw deserializer snippets updated to use int32 for sentinel/pointer checks.
apps/oxlint/src-js/plugins/source_code.ts Source offsets/length read from buffer.int32 during initialization.
apps/oxlint/src-js/package/parse.ts Parse success sentinel (programOffset) read via buffer.int32.
apps/oxlint/src-js/generated/deserialize.js Generated oxlint deserializer updated to use int32 (and cleanup in resetBuffer).
apps/oxlint/src-js/generated/constants.ts Doc comment updated to reflect Int32Array indexing for DATA_POINTER_POS_32.
Comments suppressed due to low confidence (1)

tasks/ast_tools/src/generators/raw_transfer.rs:1070

  • int32[...] reads signed 32-bit values, but niche.value() is formatted as an unsigned literal (u128). For niches that come from the end of the value range (i.e. count_start == 0), Niche::value() can be u32::MAX/u64::MAX or other values > i32::MAX, which will be read as negative via Int32Array and make these === checks fail. Consider either keeping uint32[...] for niche comparisons, or formatting the niche constant(s) as signed i32 (two’s complement) when generating the JS for int32[...] (including both halves in the size == 8 case).
        let none_condition = match niche.size {
            1 => format!("uint8[{}] === {}", pos_offset(niche_offset), niche.value()),
            // 2 => format!("uint16[{}] === {}", pos_offset_shift(offset, 1), niche.value()),
            4 => format!("int32[{}] === {}", pos_offset_shift(niche_offset, 2), niche.value()),
            8 => {
                // TODO: Use `float64[pos >> 3] === 0` instead of
                // `int32[pos >> 2] === 0 && int32[(pos + 4) >> 2] === 0`?
                let value = niche.value();
                format!(
                    "int32[{}] === {} && int32[{}] === {}",
                    pos_offset_shift(niche_offset, 2),
                    value & u128::from(u32::MAX),
                    pos_offset_shift(niche_offset + 4, 2),
                    value >> 32,
                )

@overlookmotel overlookmotel self-assigned this Apr 7, 2026
@overlookmotel overlookmotel force-pushed the om/04-07-refactor_napi_parser_linter_plugins_add_int32_buffer_view branch from 35ececf to 73f8ddd Compare April 7, 2026 16:27
@overlookmotel overlookmotel force-pushed the om/04-07-perf_napi_parser_linter_plugins_use_int32array_in_raw_transfer_deserializer branch 2 times, most recently from 3c40fb4 to 56d49d6 Compare April 7, 2026 16:39
@graphite-app graphite-app Bot added the 0-merge Merge with Graphite Merge Queue label Apr 7, 2026
@graphite-app graphite-app Bot force-pushed the om/04-07-refactor_napi_parser_linter_plugins_add_int32_buffer_view branch from 73f8ddd to ca49dcb Compare April 7, 2026 17:17
@graphite-app graphite-app Bot force-pushed the om/04-07-perf_napi_parser_linter_plugins_use_int32array_in_raw_transfer_deserializer branch from 56d49d6 to f5552d3 Compare April 7, 2026 17:17
@graphite-app
Copy link
Copy Markdown
Contributor

graphite-app Bot commented Apr 7, 2026

Merge activity

…eserializer (#21132)

Use `int32` (`Int32Array`) instead of `uint32`(`Uint32Array`) for getting data from buffer in raw transfer deserializer. The buffer is 2 GiB in size, so all offsets within the buffer can be represented as a `u31` which can be stored in an `i32` without any loss, and with no risk of being interpreted as negative numbers.

Same as in #21129, the advantage of fetching offsets from an `Int32Array` is that V8 statically knows the value can be stored in an SMI, its native integer type, without any range checks.
@graphite-app graphite-app Bot force-pushed the om/04-07-refactor_napi_parser_linter_plugins_add_int32_buffer_view branch from ca49dcb to c70a8e9 Compare April 7, 2026 17:26
@graphite-app graphite-app Bot force-pushed the om/04-07-perf_napi_parser_linter_plugins_use_int32array_in_raw_transfer_deserializer branch from f5552d3 to c0278ab Compare April 7, 2026 17:26
graphite-app Bot pushed a commit that referenced this pull request Apr 7, 2026
…elds as `i32`s (#21135)

Continuation of #21132. Get span `start` and `end` from the `Int32Array` view of the buffer, instead of the `Uint32Array`view. This allows V8 to statically see that they are SMIs, and skip type checks.
@graphite-app graphite-app Bot removed the 0-merge Merge with Graphite Merge Queue label Apr 7, 2026
Base automatically changed from om/04-07-refactor_napi_parser_linter_plugins_add_int32_buffer_view to main April 7, 2026 17:31
@graphite-app graphite-app Bot merged commit c0278ab into main Apr 7, 2026
26 of 27 checks passed
@graphite-app graphite-app Bot deleted the om/04-07-perf_napi_parser_linter_plugins_use_int32array_in_raw_transfer_deserializer branch April 7, 2026 17:32
graphite-app Bot pushed a commit that referenced this pull request Apr 7, 2026
…rs (#21136)

Continuation of #21132. Use `Int32Array` view of the buffer for all operations related to tokens and comments.
graphite-app Bot pushed a commit that referenced this pull request Apr 7, 2026
Continuation of #21132. Use `Int32Array` view of the buffer for tokens deserialization in `oxc-parser`.
graphite-app Bot pushed a commit that referenced this pull request Apr 7, 2026
Continuation of #21132. Use `Int32Array` view of the buffer for all code in lazy deserialization. Lazy deserialization is not currently in use, but keeping it in line with the main raw transfer deserializer allows removing the `Unint32Array` view of the buffer entirely in #21140 without breaking the lazy deserializer tests.
graphite-app Bot pushed a commit that referenced this pull request Apr 7, 2026
Continuation of #21132. All uses of the `Uint32Array` view of the buffer in raw transfer code have been removed in preceding PRs. So we can now remove it entirely.
camc314 pushed a commit that referenced this pull request Apr 13, 2026
### 💥 BREAKING CHANGES

- 36cdc31 str: [**BREAKING**] Remove identity `FromIn` impl for `Ident`
(#21251) (overlookmotel)
- 382958a span: [**BREAKING**] Remove re-exports of string types from
`oxc_span` crate (#21246) (overlookmotel)
- c4aedfa str: [**BREAKING**] Add `static_ident!` macro (#21245)
(overlookmotel)

### 🚀 Features

- e7e1aea transformer/typescript: Add `optimize_enums` option for
regular enum inlining (#20539) (Dunqing)
- 679f57f transformer/typescript: Implement const enum inlining and
declaration removal (#20508) (Dunqing)
- 6dd061c semantic: Extend `MemberWriteTarget` to cover all property
modification patterns (#21205) (Dunqing)
- f134e24 minifier: Support `property_write_side_effects` option to drop
unused property assignments (#20773) (Dunqing)
- 75663c0 semantic: Add enum member value evaluation for const enum
support (#20602) (Dunqing)
- 3cfe8ed semantic: Add `MemberWriteTarget` flag to `ReferenceFlags`
(#20772) (Dunqing)

### 🐛 Bug Fixes

- af1a586 transformer/class-properties: Use correct property name when
converting parameter properties (#21268) (Amal Jossy)
- b43250a allocator: Move allocation tracking into `Bump` (#21342)
(overlookmotel)
- 36f505f allocator: `StringBuilder` use `Allocator::alloc_layout`
(#21340) (overlookmotel)
- 7a08a6f allocator: Fix allocation counting in
`Allocator::alloc_concat_strs_array` (#21336) (overlookmotel)
- 2338e28 ecmascript: Treat `this` as potentially having side effects
(#21297) (sapphi-red)
- bd8bd39 allocator: Remove unsafe hacks from `from_raw_parts` methods
(#21283) (overlookmotel)
- 8f4c340 allocator: Remove dangerous pointer const to mut cast (#21279)
(overlookmotel)
- aa9259f parser: Add missing error code for optional param diagnostic
(#21258) (camc314)
- 04b3c2f str: Fix unsound casting const pointers to mut pointers
(#21242) (overlookmotel)
- ceadf6c str: Make `Ident::from_raw` an unsafe function (#21241)
(overlookmotel)
- eab13b3 transformer/decorators: Avoid accessor storage name collisions
(#21106) (Dunqing)
- 07e8a30 transformer/react-refresh: Handle parenthesized variable
initializers (#21047) (camc314)

### ⚡ Performance

- c3ca6f6 allocator: `StringBuilder::from_strs_array_in` check for 0
length earlier (#21338) (overlookmotel)
- c2422bb allocator: `Allocator::alloc_concat_strs_array` check for 0
length earlier (#21337) (overlookmotel)
- 04b0fdc allocator: Mark `Allocator::alloc_layout` as
`#[inline(always)]` (#21335) (overlookmotel)
- 17aee9e allocator: Use `offset_from_unsigned` in
`ChunkFooter::as_raw_parts` (#21280) (overlookmotel)
- 61adedd minifier: Fix O(n²) perf on very many var decls (#21062)
(Gunnlaugur Thor Briem)
- addcd02 napi/parser, linter/plugins: Raw transfer deserializer for
`Vec`s use shift instead of multiply where possible (#21142)
(overlookmotel)
- 3068ded napi/parser, linter/plugins: Shift before add when calculating
positions in raw transfer deserializer (#21141) (overlookmotel)
- eb400b8 napi/parser, linter/plugins: Remove `uint32` buffer view
(#21140) (overlookmotel)
- 2675085 napi/parser: Lazy deserialization use only `Int32Array`
(#21139) (overlookmotel)
- 5b35a53 napi/parser: Deserializing tokens use only `int32` array
(#21138) (overlookmotel)
- f163d10 parser: Tokens raw deserialization use `Int32Array` (#21137)
(overlookmotel)
- 7a86613 linter/plugins: Use `Int32Array`s for tokens and comments
buffers (#21136) (overlookmotel)
- 8c51121 napi/parser, linter/plugins: Raw transfer deserialize `Span`
fields as `i32`s (#21135) (overlookmotel)
- bc1bcdd napi/parser, linter/plugins: Inline trivial raw transfer field
deserializers into node object definitions (#21134) (overlookmotel)
- c0278ab napi/parser, linter/plugins: Use `Int32Array` in raw transfer
deserializer (#21132) (overlookmotel)
- 43482c7 linter/plugins: Use `>>` not `>>>` in binary search loops
(#21129) (overlookmotel)

### 📚 Documentation

- f5e1845 allocator: Upgrade headers in doc comments for `Bump` (#21263)
(overlookmotel)
- 2870174 allocator: Upper case `SAFETY` in comments (#21253)
(overlookmotel)
- 01bc269 str: Reformat `Ident` doc comments (#21240) (overlookmotel)
- dd47359 allocator: Add doc comments for panics and errors (#21230)
(overlookmotel)
camc314 pushed a commit that referenced this pull request Apr 13, 2026
# Oxlint
### 💥 BREAKING CHANGES

- 382958a span: [**BREAKING**] Remove re-exports of string types from
`oxc_span` crate (#21246) (overlookmotel)
- c4aedfa str: [**BREAKING**] Add `static_ident!` macro (#21245)
(overlookmotel)
- 7354f3c linter: [**BREAKING**] Error on no matched files (#21144)
(camc314)

### 🚀 Features

- 91f2c79 linter/eslint-jest-plugin: Implemented
`prefer-importing-jest-globals` rule (#21303) (Said Atrahouch)
- a02f32c linter: Add release version for existing rules (#21363)
(camchenry)
- b9e93da linter: Allow tagging rules with release version (#21362)
(camchenry)
- f99ecda oxlint: Gate `vite.config.ts` recognition behind `VP_VERSION`
env var (#21298) (leaysgur)
- cf459d3 linter: Implement suggestion for `no-empty-function` rule
(#21347) (Mikhail Baev)
- 7213d61 linter: Adding pending suggestions fix to `valid_expect`
rules. (#21249) (Said Atrahouch)
- ae45312 linter: Introduce `--type-check-only` flag (#21184) (camc314)
- 1ce8b90 linter: Implemented `valid-expect-in-promise` vitest and jest
rule (#21170) (Said Atrahouch)
- 39f7fda linter: Add auto-fix to `unicorn/prefer-default-parameters`
(#21166) (yefan)
- 15574bc linter/unicorn: Implement consistent-template-literal-escape
(#21126) (AliceLanniste)
- c5c8c03 linter/prefer-readonly-parameter-types: Move rule from nursery
to pedantic (#21114) (camc314)
- 1893be1 linter/no-useless-default-assignment: Move rule from nursery
to correctness (#21113) (camc314)
- 5462ff9 linter/strict-void-return: Move rule from nursery to pedantic
(#21115) (camc314)
- c2989bd linter/no-unnecessary-type-parameters: Move rule from nursery
to suspicious (#21112) (camc314)
- 79d339a linter/no-unnecessary-qualifier: Move rule from nursery to
style (#21111) (camc314)

### 🐛 Bug Fixes

- b577efc linter/unicorn: Handle optional chaining in
`prefer-array-flat` and `no-invalid-remove-event-listener` (#21299)
(Mikhail Baev)
- 5e55735 oxlint/lsp: Skip .git directories in LSP walkers (#21316)
(camc314)
- ec7f6ed oxlint, oxfmt: Apply `check_for_writer_error` to `.flush()`
(#21343) (Craig Morrison)
- a17a08a linter/no-useless-assignment: Handle continue edges in loop
analysis (#21358) (camc314)
- a0eac12 linter/array-type: Move match to first stmt (#21357) (camc314)
- 1b3abc3 linter: Exclude boundary tokens from JSXText whitespace check
in isSpaceBetweenTokens (#21313) (bab)
- ecbcf5e linter: More info to summary output for GitHub formatter
(#21330) (Théo LUDWIG)
- a0a8c62 linter/no-fallthrough: Check from start of switch case for
empty lines (#21324) (Josh Cartmell)
- 36f0bc4 linter/no-cycle: Report all cyclic dependencies inside a file
(#21259) (camc314)
- 3f80536 linter: Ignore regex flags other than `g`/`u`/`v` in
`prefer-string-replace-all` (#21203) (bab)
- f21d3aa linter/unicorn: Report on optional in
`require-number-to-fixed-digits-argument` rule (#21207) (Mikhail Baev)
- af8e122 linter: Render each config error as a separate diagnostic
(#21120) (bab)
- a950f55 linter/unicorn: Do not report on optionals in
`no-single-promise-in-promise-methods` (#21157) (Mikhail Baev)
- 472f8ee linter: Mark complete comment for unused disable directives +
lsp fix (#21092) (copilot-swe-agent)
- edd0865 linter/no-array-index-key: False positive when index is inside
an expression within a template literal (#21123) (bab)
- 7e8d520 linter/unicorn: Report on optional `foo?.postMessage` in
`require-post-message-target-origin` rule (#21104) (Mikhail Baev)

### ⚡ Performance

- addcd02 napi/parser, linter/plugins: Raw transfer deserializer for
`Vec`s use shift instead of multiply where possible (#21142)
(overlookmotel)
- 3068ded napi/parser, linter/plugins: Shift before add when calculating
positions in raw transfer deserializer (#21141) (overlookmotel)
- eb400b8 napi/parser, linter/plugins: Remove `uint32` buffer view
(#21140) (overlookmotel)
- 7a86613 linter/plugins: Use `Int32Array`s for tokens and comments
buffers (#21136) (overlookmotel)
- 8c51121 napi/parser, linter/plugins: Raw transfer deserialize `Span`
fields as `i32`s (#21135) (overlookmotel)
- bc1bcdd napi/parser, linter/plugins: Inline trivial raw transfer field
deserializers into node object definitions (#21134) (overlookmotel)
- c0278ab napi/parser, linter/plugins: Use `Int32Array` in raw transfer
deserializer (#21132) (overlookmotel)
- 43482c7 linter/plugins: Use `>>` not `>>>` in binary search loops
(#21129) (overlookmotel)

### 📚 Documentation

- 7888280 linter: Move config docs for `no-restricted-exports` (#21360)
(camchenry)
- 162d26c linter: Improve docs for `typescript/array-type` (#21356)
(camchenry)
- a2dbaec linter: Add missing docs for options for
`typescript/class-literal-property-style` (#21355) (camchenry)
- 79593eb linter: Improve docs for
`typescript/consistent-type-assertions` (#21353) (camchenry)
- f9d20d2 linter: Move config option docs for
`typescript/no-empty-object-type` (#21352) (camchenry)
- a8f650d linter: Add missing config option docs for
`prefer-string-start-ends-with` (#21332) (camchenry)
- cfd8a4f linter: Don't rely on old eslint doc for available globals
(#21334) (Nicolas Le Cam)
- 03865fa linter: Jest/prefer-snapshot-hint: add doc comment for
snapshot hint mode (#21290) (camchenry)
- a6fe09b linter: Add missing docs for config options in `react` plugin
(#21289) (camchenry)
- 60eaf47 linter: Add missing docs for config options in unicorn plugin
(#21288) (camchenry)
- c3c2055 linter: `jsx-a11y/label-has-associated-control`: document the
`assert` options (#21287) (camchenry)
- a928ed9 linter: Add missing config docs for vitest plugin rules
(#21285) (camchenry)
- 7e07c7c linter: `id-length`: move enum docs to doc comments (#21281)
(camchenry)
- 9746bdf linter: Add missing docs for `class-methods-use-this` config
(#21278) (camchenry)
- 6ffe7a5 linter: Move docs for `Target` variant onto enum (#21277)
(camchenry)
- 305350d linter/plugins: Correct comments (#21130) (overlookmotel)
# Oxfmt
### 💥 BREAKING CHANGES

- 382958a span: [**BREAKING**] Remove re-exports of string types from
`oxc_span` crate (#21246) (overlookmotel)

### 🚀 Features

- e3081e1 oxfmt: Gate `vite.config.ts` recognition behind `VP_VERSION`
env var (#21295) (leaysgur)
- 5b0b573 oxfmt: Update prettier to 3.8.2 (#21294) (leaysgur)
- 0d67834 oxfmt: Show hint for all files are ignored case (#21154)
(leaysgur)

### 🐛 Bug Fixes

- 2871fc2 oxfmt: Non idempotent formatting on comments in TS (#20449)
(Cat Chen)
- ec7f6ed oxlint, oxfmt: Apply `check_for_writer_error` to `.flush()`
(#21343) (Craig Morrison)
- 1a8c225 formatter: Preserve newline between self-closing JSX element
and single-char text (#21149) (Justin Mecham)
- 407b725 oxfmt: Indent dangling comments in empty enum with block
indent (#21163) (Leonabcd123)
- d13fd37 formatter: Remove extra outer parentheses on return with JSDoc
type cast (#21109) (bab)
- 22babde oxfmt: Fix unicode char escaping (#21162) (leaysgur)
- 4da53e5 formatter: Preserve trailing comma in TSX arrow functions with
default type params (#21151) (Justin Mecham)
- 94fe774 oxfmt: Handle paths with consecutive leading slashes (#21155)
(leaysgur)
- 50c389b oxfmt: Support `.editorconfig` `quote_type` (#20989)
(leaysgur)

### ⚡ Performance

- 0ce619f formatter: Use `Allocator::alloc_concat_strs_array` instead of
`StringBuilder::from_strs_array_in` (#21339) (overlookmotel)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ast Area - AST A-ast-tools Area - AST tools A-cli Area - CLI A-linter Area - Linter A-linter-plugins Area - Linter JS plugins A-parser Area - Parser C-performance Category - Solution not expected to change functional behavior, only performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants