Description

As part of dotnet/sdk#29755, if DOTNET_CLI_UI_LANGUAGE or VSLANG is set, then the console output and input encoding will change to UTF 8, so that the code page can change to be UTF 8 as well. This means that characters from languages set by those environment variable(s) can be rendered correctly.

This only applies on Windows Machines only (encoding was ok on other platforms.) And it also only applies to windows 10+ machines. This encoding change only happens if the UI Culture set by the user is not some form of english as well.

Version

.NET 8 Preview 1

Note: We are considering this for 7.0.3xx.

Previous behavior

Characters in certain languages but not limited to Chinese, German, Japanese, and or Russian, would sometimes display as garbled characters, or as ?.

New behavior

Characters will render correctly. The encoding will change as well as the code page.

How this is breaking:
Versions of Windows older than Windows 10 (November 2019 update) may not fully support UTF 8. These versions may experience issues.

In addition, there is an existing bug where the SDK leaves its encoding behind and can affect the encoding of other commands/programs called in the same command prompt after it's finished execution. (dotnet/sdk#30170). Now that the SDK more frequently changes the encoding, the impact of this may have increased.

In addition, some legacy consoles may not support UTF 8.

Type of breaking change

• Binary incompatible: Existing binaries may encounter a breaking change in behavior, such as failure to load or execute, and if so, require recompilation.
• Source incompatible: When recompiled using the new SDK or component or to target the new runtime, existing source code may require source changes to compile successfully.
• Behavioral change: Existing binaries may behave differently at run time.

Reason for change

Using the SDK CLI in other languages provided a poor experience before this change.
Example:
before:

after:

Recommended action

For those on an older version of windows 10, upgrade to November 2019 update or higher.

For those who want to use a legacy console or are facing build issues or others due to the encoding change, they should unset VSLANG and or DOTNET_CLI_UI_LANGUAGE to disable this change. We expect minimal impact as this language setting wouldn't have worked well in the first place due to garbled characters. Anyone who was not using these already will be unimpacted, and only those on windows 10+ will be impacted, most of which we think will be on November 2019 update or higher. The legacy scenarios are less likely to support the broken languages either, so it is unlikely the user would want to use another language to cause this break anyway, as well.

Feature area

SDK

Affected APIs

No response

Associated WorkItem - 67117