Skip to content

refactor(parser): Token::set_* method visible only within lexer#19852

Merged
graphite-app[bot] merged 1 commit intomainfrom
om/02-24-refactor_parser_kind_set__method_visible_only_within_lexer
Feb 28, 2026
Merged

refactor(parser): Token::set_* method visible only within lexer#19852
graphite-app[bot] merged 1 commit intomainfrom
om/02-24-refactor_parser_kind_set__method_visible_only_within_lexer

Conversation

@overlookmotel
Copy link
Member

@overlookmotel overlookmotel commented Feb 28, 2026

Token::set_* methods should not be accessible outside of the lexer. Only the lexer creates tokens, and it should not be possible to alter them, to ensure we can rely on them being accurate throughout the parser.

Make Token::set_start etc pub(super) instead of pub(crate).

Copy link
Member Author

overlookmotel commented Feb 28, 2026


How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • 0-merge - adds this PR to the back of the merge queue
  • hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@codspeed-hq
Copy link

codspeed-hq bot commented Feb 28, 2026

Merging this PR will not alter performance

✅ 52 untouched benchmarks
⏩ 3 skipped benchmarks1


Comparing om/02-24-refactor_parser_kind_set__method_visible_only_within_lexer (29a5a68) with main (c1bfdcf)

Open in CodSpeed

Footnotes

  1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@overlookmotel overlookmotel changed the title refactor(parser): Kind::set_* method visible only within lexer refactor(parser): Token::set_* method visible only within lexer Feb 28, 2026
@overlookmotel overlookmotel marked this pull request as ready for review February 28, 2026 15:41
Copilot AI review requested due to automatic review settings February 28, 2026 15:41
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Restricts Token mutation APIs to the lexer module so tokens can’t be modified elsewhere in the parser crate, strengthening the invariant that only the lexer constructs and mutates token metadata.

Changes:

  • Change Token::set_start, set_end, set_kind, and flag setters from pub(crate) to pub(super) to limit visibility to lexer and its submodules.

@overlookmotel overlookmotel self-assigned this Feb 28, 2026
@overlookmotel overlookmotel force-pushed the om/02-28-perf_estree_tokens_preallocate_sufficient_space_for_tokens_json branch from 0d8f3e4 to e50063c Compare February 28, 2026 16:06
@overlookmotel overlookmotel force-pushed the om/02-24-refactor_parser_kind_set__method_visible_only_within_lexer branch from 697c7be to c32a896 Compare February 28, 2026 16:06
@graphite-app graphite-app bot added the 0-merge Merge with Graphite Merge Queue label Feb 28, 2026
@graphite-app
Copy link
Contributor

graphite-app bot commented Feb 28, 2026

Merge activity

@graphite-app graphite-app bot removed the 0-merge Merge with Graphite Merge Queue label Feb 28, 2026
@overlookmotel overlookmotel force-pushed the om/02-24-refactor_parser_kind_set__method_visible_only_within_lexer branch from c32a896 to 834c7d1 Compare February 28, 2026 16:46
@overlookmotel overlookmotel force-pushed the om/02-28-perf_estree_tokens_preallocate_sufficient_space_for_tokens_json branch from e50063c to 7eec16b Compare February 28, 2026 16:46
@overlookmotel overlookmotel added the 0-merge Merge with Graphite Merge Queue label Feb 28, 2026
@graphite-app graphite-app bot changed the base branch from om/02-28-perf_estree_tokens_preallocate_sufficient_space_for_tokens_json to graphite-base/19852 February 28, 2026 16:53
@graphite-app graphite-app bot removed the 0-merge Merge with Graphite Merge Queue label Feb 28, 2026
@graphite-app graphite-app bot changed the base branch from graphite-base/19852 to main February 28, 2026 17:00
…9852)

`Token::set_*` methods should not be accessible outside of the lexer. Only the lexer creates tokens, and it should not be possible to alter them, to ensure we can rely on them being accurate throughout the parser.

Make `Token::set_start` etc `pub(super)` instead of `pub(crate)`.
@graphite-app graphite-app bot force-pushed the om/02-24-refactor_parser_kind_set__method_visible_only_within_lexer branch from 834c7d1 to 29a5a68 Compare February 28, 2026 17:01
graphite-app bot pushed a commit that referenced this pull request Feb 28, 2026
#19852 made `Token`s impossible to mutate outside of the lexer.

Add an escape hatch to provide methods to mutate them, for use in ESTree conversion, behind a `mutate_tokens` Cargo feature.

Also add a `Token::set_span` method, to complement the other setter methods.
@graphite-app graphite-app bot merged commit 29a5a68 into main Feb 28, 2026
31 checks passed
@graphite-app graphite-app bot deleted the om/02-24-refactor_parser_kind_set__method_visible_only_within_lexer branch February 28, 2026 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-parser Area - Parser C-cleanup Category - technical debt or refactoring. Solution not expected to change behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants