Conversation
|
💭 If we used https://github.com/AArnott/Xunit.SkippableFact, you could skip the test without failing if the current culture is not en-US. 💭 Should |
| result.Diagnostics.Verify( | ||
| Diagnostic(ERRID.ERR_InvalidDebugInformationFormat).WithArguments("-1"), | ||
| Diagnostic(ERRID.ERR_InvalidOutputName).WithArguments(CodeAnalysisResources.NameCannotStartWithWhitespace), | ||
| Diagnostic(ERRID.ERR_InvalidOutputName).WithArguments(nameCannotStartWithWhitespace), |
There was a problem hiding this comment.
The Microsoft.CodeAnalysis.Test.Utilities.DiagnosticDescription.Equals method calls GetArgumentsAsStrings which looks up the argument via diagnosticInfo.GetMessage(EnsureEnglishUICulture.PreferredOrNull). nameCannotStartWithWhitespace therefore needs also be looked up with EnsureEnglishUICulture.PreferredOrNull. The opposite behavior (message in english and argument in german) can be seen in GetDiagnosticsTests.vb below.
| EnsureEnglishUICulture.PreferredOrNull) | ||
| CompilationUtils.AssertTheseDiagnostics(compilation, <errors> | ||
| BC31030: Conditional compilation constant '1' is not valid: Identifier expected. | ||
| BC31030: <%= String.Format(conditionalCompilationConstantNotValidFormat, VBResources.ERR_ExpectedIdentifier, "1") %> |
There was a problem hiding this comment.
CompilationUtils.AssertTheseDiagnostics calls CompilationUtils.ErrorText which uses Diagnostic.GetMessage(EnsureEnglishUICulture.PreferredOrNull) to get the error text. GetMessage can translate the message but can't translate the argument. Therefore the error text ends up being a mixture of an English message and a German argument:
BC31030: Conditional compilation constant '1' is not valid: Bezeichner erwartet.
There are several options to solve this problem:
- Build the same mixed english/german message as the assert expects (as I did).
- Use the [UseCulture("en-US")] attribute to force the test to run in the en-US locale.
- Don't use
EnsureEnglishUICulture.PreferredOrNullinCompilationUtils.ErrorText
While I personally would prefer the third option it is also the one with the biggest impact. Changing CompilationUtils.ErrorText might cause other tests to fail (I haven't tried it, so the actual impact is unknown). To see the impact it would be useful to have Test.cmd passing without failures on a non-english machine as a bottom line. Therefore I would propose to merge the changes with either the first or the second solution and to follow up with another issue that removes the usages of EnsureEnglishUICulture.PreferredOrNull once all the PRs related to the localization problems are merged.
That is what I was thinking about to. The CultureInfo.DefaultThreadCurrentUICulture static property should do the trick:
I'm not certain how I would like to mention that this problem should not be blocking the merging of this PR. The purpose of this and the related PRs is to get as much failures resolved as possible. It's more important to have a good bottom line in the |
| EnsureEnglishUICulture.PreferredOrNull) | ||
| CompilationUtils.AssertTheseDiagnostics(compilation, <errors> | ||
| BC31030: Conditional compilation constant '1' is not valid: Identifier expected. | ||
| BC31030: <%= String.Format(conditionalCompilationConstantNotValidFormat, VBResources.ERR_ExpectedIdentifier, "1") %> |
There was a problem hiding this comment.
<%= String.Format(conditionalCompilationConstantNotValidFormat, VBResources.ERR_ExpectedIdentifier, "1") %> [](start = 9, length = 107)
I don't think this is the right approach. When we are requesting error message for a specific culture, the culture should be propagated to the message arguments as well, but for some reason it is not done here. This looks like a real product issue, not a test issue because I believe our test helper is forcing english culture regardless of default culture on the machine.
There was a problem hiding this comment.
The diagnostic reported by the compiler is in German. But the assertion logic takes this diagnostic and recreates it enforcing English. But this does not succeeds for the argument leaving the resulting message as a mixture of German (the argument) and English (the message format string). See also my comment above.
There was a problem hiding this comment.
The diagnostic reported by the compiler is in German. But the assertion logic takes this diagnostic and recreates it enforcing English.
Compiler never reports a diagnostic in any specific language. Language only comes into play when a message is formatted. If Diagnostic.GetMessage is called with specific culture, everything in that message should be in that culture, the template and the arguments.
But this does not succeeds for the argument leaving the resulting message as a mixture of German (the argument) and English (the message format string).
This should never happen. The only exception is for arguments that compiler gets from some other APIs as strings. For example, when an exception message is used as an argument for a diagnostic. It doesn't look like any exceptions should be applied for this particular error. So, if you observe that different parts of the message use different language, I would say there is a real bug in implementation of Diagnostic.GetMessage API or in compiler, which we should fix.
There was a problem hiding this comment.
The diagnostic under test is created here like so:
builder.Add(Diagnostic.Create(MessageProvider.Instance, ERRID.ERR_ConditionalCompilationConstantNotValid, VBResources.ERR_ExpectedIdentifier, symbol.Key))
If I got you you right it should be created like this one:
diagnostics.Add(
messageProvider.CreateDiagnostic(code, Location.None,
new CodeAnalysisResourcesLocalizableErrorArgument(errorArgumentResourceId)));
It seems there isn't a VBResourcesLocalizableErrorArgument class (at least I couldn't find anything suitable).
I see this options on how to proceed:
- Fix how the argument is passed to the diagnostic as I did in commit 220c349 for a
CodeAnalysisResourcesLocalizableErrorArgument. I need some advice how you want this to be done (like Create a VBResourcesLocalizableErrorArgument class in this folder). - Revert my changes and let the test fail on non english machines for now. The test needs to be fixed by the compiler team later.
There was a problem hiding this comment.
There is Microsoft.CodeAnalysis.VisualBasic.LocalizableErrorArgument type.
Alternatively, for consistency with other places where the same diagnostics is created (that is the approach I would prefer), we should be able to call Diagnostic.Create overload that takes DiagnosticInfo. Like this:
Diagnostic.Create(ErrorFactory.ErrorInfo(ERRID.ERR_ConditionalCompilationConstantNotValid,
ErrorFactory.ErrorInfo(ERRID.ERR_ExpectedIdentifier),
symbol.Key))
| // (16,16): error CS1503: Argument 1: cannot convert from '(<null>, int)' to 'in (int arg1, int arg2)' | ||
| // Method((null, 1)); | ||
| Diagnostic(ErrorCode.ERR_BadArgType, "(null, 1)").WithArguments("1", "(<null>, int)", "in (int arg1, int arg2)").WithLocation(16, 16), | ||
| Diagnostic(ErrorCode.ERR_BadArgType, "(null, 1)").WithArguments("1", $"({CSharpResources.IDS_NULL}, int)", "in (int arg1, int arg2)").WithLocation(16, 16), |
There was a problem hiding this comment.
{CSharpResources.IDS_NULL} [](start = 88, length = 26)
I think it would be better to force english culture when we get the arguments.
There was a problem hiding this comment.
@sharwell provided a [UseCulture] attribute for those cases. It's a matter of preference. You might discuss this with the other teams to keep it consistent across the code base. Up until now we have not used the attribute but made the tests working in other cultures.
There was a problem hiding this comment.
@AlekseyTs The general preference has been to use localized resources "to the extent reasonably possible". Typically the point where we start looking for alternatives is when simple use of string.Format + a resource is not enough to get the test to pass consistently.
There was a problem hiding this comment.
Up until now we have not used the attribute but made the tests working in other cultures.
This is not what I am suggesting. I am suggesting that the test helper that compares the diagnostics should request arguments for specific culture. All arguments for diagnostics should be localizable. No strings should be pulled from resources until someone asked for a message and then the requested culture should apply to everything in it.
There was a problem hiding this comment.
The general preference has been to use localized resources "to the extent reasonably possible".
That is not the case for compiler tests. Compiler tests do not test localizability. Our approach is to make them culture independent and baselines human readable. The culture independence is achieved by explicitly requesting error messages for english culture. If that doesn't work, then this is an indication that the test infrastructure is broken, or the product is broken. The fallback to accessing resources explicitly is the last resort approach and I am not convinced it should be used here.
There was a problem hiding this comment.
The diagnostic is created here and the argument is created via argument.Display
The problem here is that the BoundTupleExpression.Display uses a stringbuilder. So the argument is a text as opposed to a LocalizableErrorArgument. I don't see any straight forward way on how to resolve this. @AlekseyTs I would propose to revert my changes here and let the test fail on non english machines for now.
There was a problem hiding this comment.
I would propose to revert my changes here and let the test fail on non english machines for now
It would be great if you could fix it because you can easily verify the effect, but it is up to you.
I think there is a simple fix for BoundTupleExpression.Display. Instead of building complete string, we can build a format string with argument markers and collect arguments in an array. Then we call to System.Runtime.CompilerServices.FormattableStringFactory.Create and return its result.
There is also an issue with
internal partial class BoundStackAllocArrayCreation
{
public override object Display
{
get { return string.Format(MessageID.IDS_StackAllocExpression.Localize().ToString(), ElementType, Count.Syntax); }
}
}
This one can be fixed by creating an object that is very similar to LocalizableErrorArgument, that calls Format after getting the string for requested culture.
…diagnostic instead of a string.
| Dim nameCannotStartWithWhitespace = New CodeAnalysisResourcesLocalizableErrorArgument(NameOf(CodeAnalysisResources.NameCannotStartWithWhitespace)) | ||
| result.Diagnostics.Verify( | ||
| Diagnostic(ERRID.ERR_InvalidDebugInformationFormat).WithArguments("-1"), | ||
| Diagnostic(ERRID.ERR_InvalidOutputName).WithArguments(CodeAnalysisResources.NameCannotStartWithWhitespace), |
There was a problem hiding this comment.
CodeAnalysisResources.NameCannotStartWithWhitespace [](start = 70, length = 51)
I think we should simply hardcode "Name cannot start with whitespace." string here.
| // (8,17): error CS0023: Operator 'is' cannot be applied to operand of type '(int, method group)' | ||
| // if ((1, object.Equals) is()) {} | ||
| Diagnostic(ErrorCode.ERR_BadUnaryOp, "(1, object.Equals) is()").WithArguments("is", "(int, method group)").WithLocation(8, 17) | ||
| Diagnostic(ErrorCode.ERR_BadUnaryOp, "(1, object.Equals) is()").WithArguments("is", $"(int, {CSharpResources.IDS_MethodGroup})").WithLocation(8, 17) |
There was a problem hiding this comment.
$"(int, {CSharpResources.IDS_MethodGroup})") [](start = 100, length = 44)
This looks like the same issue with BoundTupleExpression.Display.
| // (9,13): error CS0023: Operator 'is' cannot be applied to operand of type '(int, <null>)' | ||
| // if ((1, null) is 4) {} | ||
| Diagnostic(ErrorCode.ERR_BadUnaryOp, "(1, null) is 4").WithArguments("is", "(int, <null>)").WithLocation(9, 13), | ||
| Diagnostic(ErrorCode.ERR_BadUnaryOp, "(1, null) is 4").WithArguments("is", $"(int, {CSharpResources.IDS_NULL})").WithLocation(9, 13), |
There was a problem hiding this comment.
$"(int, {CSharpResources.IDS_NULL}) [](start = 91, length = 35)
This looks like the same issue with BoundTupleExpression.Display.
| // (13,13): error CS0023: Operator 'is' cannot be applied to operand of type '(int, <null>)' | ||
| // if ((1, null) is var x4) {} | ||
| Diagnostic(ErrorCode.ERR_BadUnaryOp, "(1, null) is var x4").WithArguments("is", "(int, <null>)").WithLocation(13, 13) | ||
| Diagnostic(ErrorCode.ERR_BadUnaryOp, "(1, null) is var x4").WithArguments("is", $"(int, {CSharpResources.IDS_NULL})").WithLocation(13, 13) |
There was a problem hiding this comment.
$"(int, {CSharpResources.IDS_NULL})") [](start = 96, length = 37)
This looks like the same issue with BoundTupleExpression.Display.
| // (6,13): error CS0023: Operator 'is' cannot be applied to operand of type '(int, <null>)' | ||
| // if ((1, null) is Program) {} | ||
| Diagnostic(ErrorCode.ERR_BadUnaryOp, "(1, null) is Program").WithArguments("is", "(int, <null>)").WithLocation(6, 13) | ||
| Diagnostic(ErrorCode.ERR_BadUnaryOp, "(1, null) is Program").WithArguments("is", $"(int, {CSharpResources.IDS_NULL})").WithLocation(6, 13) |
There was a problem hiding this comment.
$"(int, {CSharpResources.IDS_NULL})" [](start = 97, length = 36)
This looks like the same issue with BoundTupleExpression.Display.
| var parsedTree = ParseWithRoundTripCheck(source.ToString()); | ||
| IEnumerable<Diagnostic> actualErrors = parsedTree.GetDiagnostics(); | ||
| Assert.Equal("202", actualErrors.Count().ToString()); | ||
| Assert.Equal("(1,1201): error CS1056: Unexpected character '\\u003C'", actualErrors.ElementAt(200).ToString()); |
There was a problem hiding this comment.
ToString [](start = 111, length = 8)
Rather than calling Object.ToString(), we should call IFormattable.ToString and passing EnsureEnglishUICulture.PreferredOrNull as the culture
| Assert.Equal("202", actualErrors.Count().ToString()); | ||
| Assert.Equal("(1,2001): error CS1056: Unexpected character '\\U0000003C'", actualErrors.ElementAt(200).ToString()); | ||
| Assert.Equal("(1,2011): error CS1056: Unexpected character '\\u003E\\u003E\\u003E\\u003E'", actualErrors.ElementAt(201).ToString()); | ||
| Assert.Equal($"(1,2001): error CS1056: {string.Format(CSharpResources.ERR_UnexpectedCharacter, "\\U0000003C")}", actualErrors.ElementAt(200).ToString()); |
There was a problem hiding this comment.
ToString() [](start = 153, length = 10)
Same as above, call IFormattable.ToString passing EnsureEnglishUICulture.PreferredOrNull as the culture, instead.
| ' This test failes on non english machines. | ||
| ' The expected output is generated by the runtime and is localized with the culture of the current machine. | ||
| ' Localized StringRessources of the runtime are not part of the Roslyn solution. | ||
| ' Setting DefaultThreadCurrentCulture to en-US does not help because CompileAndVerify starts an AppDomain. |
There was a problem hiding this comment.
Setting DefaultThreadCurrentCulture to en-US does not help because CompileAndVerify starts an AppDomain. [](start = 14, length = 104)
Modify code in the Main function above to change to invariant culture and restore at the end. The same way as it is done in Spilling_ExceptionInArrayAccess unit test for example.
| Dim compilation = CreateCompilationWithMscorlib({syntaxTree1, syntaxTree2, syntaxTree3}, options:=options) | ||
| Dim diagnostics = compilation.GetDiagnostics() | ||
|
|
||
| Dim conditionalCompilationConstantNotValidFormat = VBResources.ResourceManager.GetString( |
There was a problem hiding this comment.
Dim conditionalCompilationConstantNotValidFormat = VBResources.ResourceManager.GetString( [](start = 12, length = 89)
Looks like the same bug in VisualBasicParseOptions.ValidateOptions
| Dim compilation = CreateCompilationWithMscorlib({syntaxTree1, syntaxTree2, syntaxTree3}, options:=options) | ||
| Dim diagnostics = compilation.GetDiagnostics() | ||
|
|
||
| Dim conditionalCompilationConstantNotValidFormat = VBResources.ResourceManager.GetString( |
There was a problem hiding this comment.
conditionalCompilationConstantNotValidFormat [](start = 16, length = 44)
Looks like the same bug in VisualBasicParseOptions.ValidateOptions
| Dim compilation = CreateCompilationWithMscorlibAndVBRuntime({syntaxTree1, syntaxTree2}, options:=options) | ||
| Dim diagnostics = compilation.GetDiagnostics() | ||
|
|
||
| Dim conditionalCompilationConstantNotValidFormat = VBResources.ResourceManager.GetString( |
There was a problem hiding this comment.
conditionalCompilationConstantNotValidFormat [](start = 16, length = 44)
Looks like the same bug in VisualBasicParseOptions.ValidateOptions
|
|
||
| Dim options = New VisualBasicCompilationOptions(OutputKind.DynamicallyLinkedLibrary, parseOptions:=parseOptions1) | ||
|
|
||
| Dim conditionalCompilationConstantNotValidFormat = VBResources.ResourceManager.GetString( |
There was a problem hiding this comment.
conditionalCompilationConstantNotValidFormat [](start = 16, length = 44)
Looks like the same bug in VisualBasicParseOptions.ValidateOptions
| Dim options = New VisualBasicParseOptions(preprocessorSymbols:=symbols) | ||
|
|
||
| options.Errors.Verify(Diagnostic(ERRID.ERR_ConditionalCompilationConstantNotValid).WithArguments("Identifier expected.", "1").WithLocation(1, 1)) | ||
| options.Errors.Verify(Diagnostic(ERRID.ERR_ConditionalCompilationConstantNotValid).WithArguments(VBResources.ERR_ExpectedIdentifier, "1").WithLocation(1, 1)) |
There was a problem hiding this comment.
VBResources.ERR_ExpectedIdentifier [](start = 105, length = 34)
Looks like the same bug in VisualBasicParseOptions.ValidateOptions. Applies to all changes in this file.
|
Done with review pass (iteration 7) |
|
I'm going to close this PR as it tries to solve to many divergent problems at once1. I will create separate issues for each problem and copy the relevant comments from this PR. I will open a new PR resolving one of the new issues. [1]: Furthermore there are some more personal reasons:
|
Fix for #23837.
Follow up for #24407,#24424, #24426 and #24460.
This is the fifth of a series of PRs meant to resolve the unit test failures caused by missing localizations.
This PR includes test related to the compiler and fixes the remaining errors of #24460
Roslyn.Compilers.CSharp.Semantic.UnitTests
Done. Tested local (de-DE) and CI (en-US).
Roslyn.Compilers.CSharp.Syntax.UnitTests
Done. Tested local (de-DE) and CI (en-US).
Roslyn.Compilers.VisualBasic.Emit.UnitTests
Unresolved. See below.
Roslyn.Compilers.VisualBasic.Semantic.UnitTests
Done. Tested local (de-DE) and CI (en-US).
Roslyn.Compilers.VisualBasic.Symbol.UnitTests
Done. Tested local (de-DE) and CI (en-US).
Roslyn.Compilers.VisualBasic.Syntax.UnitTests
Done. Tested local (de-DE) and CI (en-US).
Remarks for Roslyn.Compilers.VisualBasic.Emit.UnitTests
There is only one failing test here but I couldn't resolve it.
The localization error could not be fixed, because the localized text is not part of Roslyn. The error message is produced by the runtime. There are options to circumvent the problem but the solutions are not straight forward, because there is an AppDomain created by the test. I just added a comment for the moment and awaiting further advices from the code review.