refactor(napi/parser): raw transfer store source text at end of buffer by overlookmotel · Pull Request #22392 · oxc-project/oxc

overlookmotel · 2026-05-13T23:38:12Z

Oxlint's raw transfer implementation stores the source text at end of the buffer. Prior to this PR napi/parser stored source text at the start of the buffer.

Bring napi/parser into line with Oxlint, by also storing source text at end of the buffer.

Making all versions of raw transfer that we use align with each other removes complication (all versions now have the same implementation of deserializeStr) and also removes some fragile code where it'd be easy to accidentally trigger UB (see the updated comments in arena/fixed_size/windows.rs and arena/alloc_impl.rs).

Unfortunately, storing source at end of buffer is slightly less efficient than storing it at the start. We'll switch all implementations to storing source at start of buffer when we move to storing all strings (source text and all other strings) together in a single contiguous block. But in meantime, this slight inefficiency is outweighed by the gain in safety.

overlookmotel · 2026-05-13T23:38:27Z

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

0-merge - adds this PR to the back of the merge queue
hotfix - for urgent changes, fast-track this PR to the front of the merge queue

You must have a Graphite account in order to use the merge queue. Sign up using this link.

_{An organization admin has enabled the Graphite Merge Queue in this repository.} _{Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.}

This stack of pull requests is managed by Graphite. Learn more about stacking.

codspeed-hq · 2026-05-13T23:46:04Z

Merging this PR will not alter performance

✅ 48 untouched benchmarks
⏩ 3 skipped benchmarks¹

_{Comparing om/05-13-refactor_napi_parser_raw_transfer_store_source_text_at_end_of_buffer (c2c8f80) with main (6d42395)²}

3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
No successful run was found on main (4ab57eb) during the generation of this report, so 6d42395 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

Copilot

Pull request overview

This PR aligns napi/parser’s raw-transfer buffer layout with Oxlint by placing the UTF-8 source text at the end of the active buffer region and passing an explicit sourceStartPos through the JS/Rust deserialization pipeline, simplifying the shared deserializeStr logic and reducing fragile/UB-prone layout assumptions.

Changes:

Update raw-transfer JS deserializers (parser + Oxlint) to accept sourceStartPos and treat source-backed string slices as buffer offsets relative to that start.
Update napi/parser Rust raw-transfer entrypoints to accept source_start and initialize the arena cursor before the source region so AST data is written “above” it.
Refresh allocator safety commentary related to fixed-size arenas and externally owned buffers.

Reviewed changes

Copilot reviewed 8 out of 18 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tasks/ast_tools/src/generators/raw_transfer.rs	Updates eager deserializer generator to use `sourceStartPos` and source-at-end string slicing logic.
tasks/ast_tools/src/generators/raw_transfer_lazy.rs	Updates lazy deserializer generator for the new source positioning.
napi/parser/src/raw_transfer.rs	Extends N-API raw-transfer parse functions with `source_start` and adjusts arena cursor/source slicing accordingly.
napi/parser/src-js/raw-transfer/common.js	Writes source bytes near the end of the active region and threads `sourceStartPos` through parse/convert calls.
napi/parser/src-js/raw-transfer/eager.js	Updates eager raw-transfer deserializer invocation signature to include `sourceStartPos`.
napi/parser/src-js/raw-transfer/lazy.js	Stores `sourceStartPos` on the lazy AST wrapper object and updates construction signature.
napi/parser/src-js/generated/lazy/constructors.js	Regenerates lazy constructors with updated string/source handling.
napi/parser/src-js/generated/deserialize/js.js	Regenerates JS eager deserializer with `sourceStartPos`-based source string slicing.
napi/parser/src-js/generated/deserialize/js_parent.js	Same as above for parent-enabled variant.
napi/parser/src-js/generated/deserialize/js_range.js	Same as above for range-enabled variant.
napi/parser/src-js/generated/deserialize/js_range_parent.js	Same as above for range+parent variant.
napi/parser/src-js/generated/deserialize/ts.js	Regenerates TS eager deserializer with `sourceStartPos`-based source string slicing.
napi/parser/src-js/generated/deserialize/ts_parent.js	Same as above for parent-enabled variant.
napi/parser/src-js/generated/deserialize/ts_range.js	Same as above for range-enabled variant.
napi/parser/src-js/generated/deserialize/ts_range_parent.js	Same as above for range+parent variant.
crates/oxc_allocator/src/arena/fixed_size/windows.rs	Updates comments about fixed-size arena growth safety with externally owned buffers.
crates/oxc_allocator/src/arena/alloc_impl.rs	Updates comments describing when `is_fixed_size` can be true and expected behavior.
apps/oxlint/src-js/generated/deserialize.js	Updates Oxlint JS deserializer signature/plumbing to match the new `sourceStartPos` flow.

graphite-app · 2026-05-14T22:26:09Z

Merge activity

May 14, 10:26 PM UTC: overlookmotel added this pull request to the Graphite merge queue.
May 14, 10:36 PM UTC: Merged by the Graphite merge queue.

#22392) Oxlint's raw transfer implementation stores the source text at end of the buffer. Prior to this PR `napi/parser` stored source text at the start of the buffer. Bring `napi/parser` into line with Oxlint, by also storing source text at end of the buffer. Making all versions of raw transfer that we use align with each other removes complication (all versions now have the same implementation of `deserializeStr`) and also removes some fragile code where it'd be easy to accidentally trigger UB (see the updated comments in `arena/fixed_size/windows.rs` and `arena/alloc_impl.rs`). Unfortunately, storing source at end of buffer is slightly less efficient than storing it at the start. We'll switch all implementations to storing source at start of buffer when we move to storing all strings (source text and all other strings) together in a single contiguous block. But in meantime, this slight inefficiency is outweighed by the gain in safety.

overlookmotel mentioned this pull request May 13, 2026

refactor(allocator): add Arena::grow_fixed_size_chunk method #22123

Merged

github-actions Bot added A-linter Area - Linter A-parser Area - Parser A-cli Area - CLI A-ast-tools Area - AST tools A-linter-plugins Area - Linter JS plugins A-allocator Area - Allocator labels May 13, 2026

This was referenced May 13, 2026

ci(miri): run oxc_allocator tests with Miri on Windows #22121

Merged

refactor(allocator): per-platform implementation of freeing fixed-size chunks #22122

Merged

This was referenced May 13, 2026

fix(allocator): fixed-size allocators use VirtualAlloc on Windows #22124

Merged

fix(allocator): fix segfault on Linux MUSL with fixed-size allocators #22388

Merged

overlookmotel marked this pull request as ready for review May 13, 2026 23:52

overlookmotel requested a review from camc314 as a code owner May 13, 2026 23:52

Copilot AI review requested due to automatic review settings May 13, 2026 23:52

Copilot started reviewing on behalf of overlookmotel May 13, 2026 23:53 View session

overlookmotel self-assigned this May 13, 2026

Copilot AI reviewed May 13, 2026

View reviewed changes

Comment thread tasks/ast_tools/src/generators/raw_transfer_lazy.rs Outdated

Comment thread napi/parser/src/raw_transfer.rs

overlookmotel force-pushed the om/05-13-refactor_napi_parser_raw_transfer_store_source_text_at_end_of_buffer branch from fc55390 to 62e44cc Compare May 14, 2026 14:28

overlookmotel mentioned this pull request May 14, 2026

refactor(napi/parser, linter/plugins): improve safety of raw transfer interface #22424

Merged

overlookmotel force-pushed the om/05-13-refactor_napi_parser_raw_transfer_store_source_text_at_end_of_buffer branch from 62e44cc to 7e21af0 Compare May 14, 2026 18:15

overlookmotel force-pushed the om/05-03-fix_allocator_fixed-size_allocators_use_virtualalloc_on_windows branch 2 times, most recently from 053c1c6 to 74a046f Compare May 14, 2026 18:28

overlookmotel force-pushed the om/05-13-refactor_napi_parser_raw_transfer_store_source_text_at_end_of_buffer branch 2 times, most recently from 0f1fb87 to eebf4df Compare May 14, 2026 21:15

overlookmotel force-pushed the om/05-03-fix_allocator_fixed-size_allocators_use_virtualalloc_on_windows branch from 74a046f to 60033f6 Compare May 14, 2026 21:15

graphite-app Bot force-pushed the om/05-03-fix_allocator_fixed-size_allocators_use_virtualalloc_on_windows branch from 60033f6 to 4ab57eb Compare May 14, 2026 22:29

graphite-app Bot force-pushed the om/05-13-refactor_napi_parser_raw_transfer_store_source_text_at_end_of_buffer branch from eebf4df to c2c8f80 Compare May 14, 2026 22:29

Base automatically changed from om/05-03-fix_allocator_fixed-size_allocators_use_virtualalloc_on_windows to main May 14, 2026 22:35

graphite-app Bot merged commit c2c8f80 into main May 14, 2026
41 checks passed

graphite-app Bot deleted the om/05-13-refactor_napi_parser_raw_transfer_store_source_text_at_end_of_buffer branch May 14, 2026 22:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor(napi/parser): raw transfer store source text at end of buffer#22392

refactor(napi/parser): raw transfer store source text at end of buffer#22392
graphite-app[bot] merged 1 commit into
mainfrom
om/05-13-refactor_napi_parser_raw_transfer_store_source_text_at_end_of_buffer

overlookmotel commented May 13, 2026 •

edited

Loading

Uh oh!

overlookmotel commented May 13, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented May 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

graphite-app Bot commented May 14, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

overlookmotel commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

overlookmotel commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How to use the Graphite Merge Queue

Uh oh!

codspeed-hq Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

graphite-app Bot commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

overlookmotel commented May 13, 2026 •

edited

Loading

overlookmotel commented May 13, 2026 •

edited

Loading

codspeed-hq Bot commented May 13, 2026 •

edited

Loading

graphite-app Bot commented May 14, 2026 •

edited

Loading