Skip to content

Simplify the code around parsing interpolated string literals.#57945

Merged
CyrusNajmabadi merged 6 commits intodotnet:mainfrom
CyrusNajmabadi:simplifyInterpolationPArsing
Nov 24, 2021
Merged

Simplify the code around parsing interpolated string literals.#57945
CyrusNajmabadi merged 6 commits intodotnet:mainfrom
CyrusNajmabadi:simplifyInterpolationPArsing

Conversation

@CyrusNajmabadi
Copy link
Contributor

Extracted from the raw-string-literals work (esp. ongoing work to get raw-interpolated-string-literals working).

@CyrusNajmabadi CyrusNajmabadi requested a review from a team as a code owner November 23, 2021 18:55
@ghost ghost added the Area-Compilers label Nov 23, 2021
// lexical errors
var info = default(Lexer.TokenInfo);
tempLexer.ScanInterpolatedStringLiteralTop(interpolations, isVerbatim, ref info, ref error, out closeQuoteMissing);
tempLexer.ScanInterpolatedStringLiteralTop(interpolations, isVerbatim, ref info, out error, out closeQuoteMissing);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ref was bogus here. we never passed data in.

{
internal partial class Lexer
{
internal readonly struct Interpolation
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a pure move (and the type was made 'readonly').

}

private class InterpolatedStringScanner
private struct InterpolatedStringScanner
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was made a struct as it is not copied and only sits on one stack frame.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, what enforces "NonCopyable"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have an analyzer that does it :)

{
private readonly Lexer _lexer;
private bool _isVerbatim;
private readonly bool _isVerbatim;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for this to mutate as we can just make a fresh scanner in place.

try
{
_isVerbatim = isVerbatimSubstring;
ScanInterpolatedStringLiteralTop(interpolations, ref info, closeQuoteMissing: out _);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the act of recursing directly into sub functions was adding a lot of complexity in teh raw-string-literal side of things. It's much simpler for the interpolation-sub-scanner to just recurse directly into the lexer so that all state is fresh and appropriate to each sub-item scanned.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: we have a lot of tests for this area (several of my previous PRs beefed this up a ton).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So was the old way trying to reuse the same InterpolatedStringScanner instance between the parent and nested interpolated strings? I don't see why else it would work the way it did..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. But i posit there is anti-value in doing that, esp. given how recursive the processing is of nested interpolations is.

case '*':
// check for and scan /* comment */
ScanInterpolatedStringLiteralNestedComment();
_lexer.ScanMultiLineComment(out _);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can (and should) just reuse the actual helpers we have here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: we have a lot of tests for this area (several of my previous PRs beefed this up a ton).

@CyrusNajmabadi CyrusNajmabadi merged commit 8c43f2e into dotnet:main Nov 24, 2021
@ghost ghost added this to the Next milestone Nov 24, 2021
@CyrusNajmabadi CyrusNajmabadi deleted the simplifyInterpolationPArsing branch November 24, 2021 19:07
@jcouv
Copy link
Member

jcouv commented Nov 29, 2021

Please squash compiler changes

@CyrusNajmabadi
Copy link
Contributor Author

So, squashing is effectively impossible with these PRs. I have currently 8 separate branches where these chnages are flowing. If i squash, then i always conflict flowing into the other branches. Whereas, if hte other branches are built off of commits forked from these commits, then git understands how to flow things in properly (i.e. knowing what is already in the dest branch, and only applying commits not there). This makes for trivial branch consolidation. At the cost of a true branching structure in the commits :(

@jcouv
Copy link
Member

jcouv commented Nov 30, 2021

So, squashing is effectively impossible with these PRs.

I don't think so. Before opening the next PR for review, just rebase it to erase commits from previous PR.

@allisonchou allisonchou modified the milestones: Next, 17.1.P2 Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants