Skip to content

Merge master to features/compiler#29207

Merged
brettfo merged 35 commits intofeatures/compilerfrom
merges/master-to-features/compiler
Aug 20, 2018
Merged

Merge master to features/compiler#29207
brettfo merged 35 commits intofeatures/compilerfrom
merges/master-to-features/compiler

Conversation

@dotnet-bot
Copy link
Copy Markdown
Collaborator

This is an automatically generated pull request from master into features/compiler.

git fetch --all
git checkout merges/master-to-features/compiler
git reset --hard upstream/features/compiler
git merge upstream/master
# Fix merge conflicts
git commit
git push merges/master-to-features/compiler --force

Once all conflicts are resolved and all the tests pass, you are free to merge the pull request.

CyrusNajmabadi and others added 30 commits August 1, 2018 11:39
…rs in a string-literal token

From the #23984 PR:

The first subsystem is called the VirtualCharService deals with the following issue. To the final .net regex, the following snippets of code appear completely identical to it:

"\\1"       // In a normal string, we have to escape the \
@"\1"       // But not in a verbatim string
"\\\u0031"  // The '1' could be escaped
"\\u005c1"  // Even the backslash *itself* may be escaped

These are all ways of writing the \1 regex.

In other words, C# allows a wide variety of input strings to all compile down to the same final 'value' (or 'ValueText') that the regex engine will finally see. This is a major issue as it means that any data reported by the regex engine must be accurate with respect to the text as the user wrote it. For example, in all of the equivalent cases above, there is the same error "Reference to undefined group number 1". However, for each form the user wrote, it's necessary to understand what the right value is to highlight as the problem. i.e.

https://user-images.githubusercontent.com/4564579/34459671-5bb785b2-edab-11e7-8413-79c331ef373f.png

and

https://user-images.githubusercontent.com/4564579/34459672-6deb88dc-edab-11e7-8236-7ba7cd331247.png

So, the purpose of the VirtualCharService is to translate all of the above pieces of user code to the same final set of characters the regex engine will see (specifically \ and 1) while also maintaining the knowledge of where those characters came from (for example, that 1 came from \u0031 in the last example). In essence, the VirtualCharService is able to produce the ValueText for any string literal, while having a mapping back from each character in the ValueText back to the original source span of the document that formed this.

With the VirtualCharService user code can be translated into a common format that then can be processed uniformly. This means that the part of the system that actually tries to understand the regex does not need to know about the differences between @"" and "" strings, or the differences between C# and VB. It also means that it can be used by any roslyn language (for example, F#) if that is so desired.
* Watson fix

* try null

* Add IsNullOrWhitespace check

* Add unit test

* respond to feedback

* respond to feedback
* fix race by making it thread-safe

* PR feedbacks

* more PR feedbacks
* Do not override assembly version in non-official builds

The toolset sets the assembly version to 42.42.42.42 if not set explicitly.

* Fix property name

* Fix build of the day number

* Set OfficialBuildId
Add support for colorizing escape sequences in string literals.
@brettfo brettfo merged commit 7800b90 into features/compiler Aug 20, 2018
@brettfo brettfo deleted the merges/master-to-features/compiler branch August 20, 2018 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants