Skip to content

Stop leaking identifiers in parser generator and use more DSLv2 types#977

Merged
Xanewok merged 8 commits intoNomicFoundation:mainfrom
Xanewok:dont-leak-in-pg-and-cleanup
May 24, 2024
Merged

Stop leaking identifiers in parser generator and use more DSLv2 types#977
Xanewok merged 8 commits intoNomicFoundation:mainfrom
Xanewok:dont-leak-in-pg-and-cleanup

Conversation

@Xanewok
Copy link
Copy Markdown
Contributor

@Xanewok Xanewok commented May 23, 2024

Part of #638, ticks the "stop leaking identifiers" box

  • We leverage the existing codegen_language_definition::model::Identifier in PG rather than rely on &'static strs from DSL v1, so we don't leak anymore and we start using the DSLv2 type in PG
  • Similarly, we move some of the old DSLv1 utilities in favor of directly using the DSLv2 counterparts
  • Version-dependent codegen is now extracted into a dedicated module rather than split ad-hoc in trie.rs and parser_definition.rs
  • We separate the accumulator-specific state from the model itself to better convey what is required and necessary during the accumulation and what data is finally being derived from the accumulation and stored in the resulting ParserModel used in the MVC fashion for PG codegen

The remaining work is to stop using the DSLv1 Grammar(Visitor) but rather directly collect items from and use the DSLv2 model to construct the final ParserModel for codegen.

I wouldn't look too much into details like duplicated doc comments for the state accumulator/model because it's going away in the next PR, similarly naming for stuff like Versioned(Quote) that's also renamed temporarily and will be changed/removed since we will move off the DSLv1 model.

@Xanewok Xanewok requested a review from a team as a code owner May 23, 2024 08:53
@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented May 23, 2024

⚠️ No Changeset found

Latest commit: b408825

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@Xanewok Xanewok changed the title Stop leak identifiers in parser generator and use more DSLv2 types Stop leaking identifiers in parser generator and use more DSLv2 types May 23, 2024
Labeled, ParserDefinitionNode, ParserDefinitionRef, TriviaParserDefinitionRef,
};
use crate::parser::versioned::{Versioned as _, VersionedQuote as _};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest removing the as _ from import statements, to enable unused warnings when it is no longer needed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not true: after removing the usages I still get correct unused import: `Versioned as _` rustc diagnostic; like I mentioned, this is somewhat special-cased by rustc.

Copy link
Copy Markdown
Contributor

@OmarTawfik OmarTawfik May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is somewhat special-cased by rustc

hmmm, this is surprising. it goes against how underscore behaves in other places! thanks for pointing out 🤔
I wonder what is the benefit of that syntax then, rather than just use Versioned;? in this case, the underscore is meaningless to rustc?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It exists when you want to import a trait but also use a type with the same name - in this case, using the trait in scope allows you to use the trait methods but you can still use the other data type with a given name.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@Xanewok Xanewok added this pull request to the merge queue May 24, 2024
Merged via the queue into NomicFoundation:main with commit 099f2d5 May 24, 2024
@Xanewok Xanewok deleted the dont-leak-in-pg-and-cleanup branch May 24, 2024 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants