Skip to content

Name the CST nodes#710

Merged
Xanewok merged 42 commits intoNomicFoundation:mainfrom
Xanewok:named-nodes
Dec 19, 2023
Merged

Name the CST nodes#710
Xanewok merged 42 commits intoNomicFoundation:mainfrom
Xanewok:named-nodes

Conversation

@Xanewok
Copy link
Copy Markdown
Contributor

@Xanewok Xanewok commented Dec 14, 2023

Closes #703

I think this branch has reached a critical mass where the core functionality is there and this needs tweaks/polishes that could probably be done in subsequent PRs.

The gist of the change is that now the RuleNode has children of type NamedNode = (String, Node). The parsers now thread the expected node/field names into the PG, which adds a with_name transformation, that assigns the relevant names for significant nodes in the intermediate parse results, which is folded to the CST using the existing mechanism.

Trivia scanners are unnamed. The cursor still internally uses an unnamed node not to incur cost when it's not used but a new CursorWithNames wrapper is introduced, that wraps the underlying cursor and returns (String, Node) items.

Quite a few things have been fixed since the PoC, mostly around delimited, flattened fields and now the recovered nodes also have names. Dummy names were removed and only the desired "auto-generated" ones remain, i.e. item, separator, variant.

There's still an open issue how best to name/propagate the names to the unreduced, recovered from precedence parse results (e.g. 2 * new - the MultiplicativeExpression is not yet named) but since it's only related to the recovery and in the precedence case, we can punt it for now (especially since recovery isn't ideal yet around the precedence parser).

Other than that, the next thing will be collecting the field names into an enum to cut down the memory footprint of the CST.

Specifically, don't overwrite the field names for the operator
expressions and also don't name the trivia items (insignificant).
Copy link
Copy Markdown
Contributor

@OmarTawfik OmarTawfik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Left a few suggestions, but nothing blocking.

@Xanewok Xanewok enabled auto-merge December 19, 2023 18:34
@Xanewok Xanewok added this pull request to the merge queue Dec 19, 2023
Merged via the queue into NomicFoundation:main with commit 2025b6c Dec 19, 2023
@Xanewok Xanewok deleted the named-nodes branch December 19, 2023 18:48
@github-actions github-actions bot mentioned this pull request Dec 19, 2023
github-merge-queue bot pushed a commit that referenced this pull request Jan 3, 2024
Ref #703

Builds on and should be merged after #710

Separated it out since it probably should warrant another review pass
and is mostly an optimization.
github-merge-queue bot pushed a commit that referenced this pull request Jan 9, 2024
Part of #710 #713

Forgot to expose the relevant function in NAPI.
github-merge-queue bot pushed a commit that referenced this pull request Jan 11, 2024
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @nomicfoundation/slang@0.13.0

### Minor Changes

- [#710](#710)
[`2025b6cb`](2025b6c)
Thanks [@Xanewok](https://github.com/Xanewok)! - CST children nodes are
now named

- [#723](#723)
[`b3dc6bcd`](b3dc6bc)
Thanks [@Xanewok](https://github.com/Xanewok)! - Properly parse
unreserved keywords in an identifier position, i.e. `from`, `emit`,
`global` etc.

- [#728](#728)
[`662a672c`](662a672)
Thanks [@Xanewok](https://github.com/Xanewok)! - Remove Language#scan
API; use the parser API instead

- [#719](#719)
[`1ad6bb37`](1ad6bb3)
Thanks [@OmarTawfik](https://github.com/OmarTawfik)! - introduce strong
types for all Solidity non terminals in the TypeScript API.

### Patch Changes

- [#719](#719)
[`1ad6bb37`](1ad6bb3)
Thanks [@OmarTawfik](https://github.com/OmarTawfik)! - unify
Rust/TypeScript node helpers: `*_with_kind()`, `*_with_kinds()`,
`*_is_kind()`), ...

- [#731](#731)
[`3deaea2e`](3deaea2)
Thanks [@OmarTawfik](https://github.com/OmarTawfik)! - add
`RuleNode.unparse()` to the TypeScript API

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
rollup-smithbm0p added a commit to rollup-smithbm0p/slang that referenced this pull request Dec 26, 2025
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @nomicfoundation/slang@0.13.0

### Minor Changes

- [#710](NomicFoundation/slang#710)
[`2025b6cb`](NomicFoundation/slang@2025b6c)
Thanks [@Xanewok](https://github.com/Xanewok)! - CST children nodes are
now named

- [#723](NomicFoundation/slang#723)
[`b3dc6bcd`](NomicFoundation/slang@b3dc6bc)
Thanks [@Xanewok](https://github.com/Xanewok)! - Properly parse
unreserved keywords in an identifier position, i.e. `from`, `emit`,
`global` etc.

- [#728](NomicFoundation/slang#728)
[`662a672c`](NomicFoundation/slang@662a672)
Thanks [@Xanewok](https://github.com/Xanewok)! - Remove Language#scan
API; use the parser API instead

- [#719](NomicFoundation/slang#719)
[`1ad6bb37`](NomicFoundation/slang@1ad6bb3)
Thanks [@OmarTawfik](https://github.com/OmarTawfik)! - introduce strong
types for all Solidity non terminals in the TypeScript API.

### Patch Changes

- [#719](NomicFoundation/slang#719)
[`1ad6bb37`](NomicFoundation/slang@1ad6bb3)
Thanks [@OmarTawfik](https://github.com/OmarTawfik)! - unify
Rust/TypeScript node helpers: `*_with_kind()`, `*_with_kinds()`,
`*_is_kind()`), ...

- [#731](NomicFoundation/slang#731)
[`3deaea2e`](NomicFoundation/slang@3deaea2)
Thanks [@OmarTawfik](https://github.com/OmarTawfik)! - add
`RuleNode.unparse()` to the TypeScript API

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow CST node children to be named

3 participants