Skip to content

{Core} Always use UTF-8 for command metadata log file encoding#17996

Merged
jiasli merged 2 commits intoAzure:devfrom
jiasli:encoding
May 8, 2021
Merged

{Core} Always use UTF-8 for command metadata log file encoding#17996
jiasli merged 2 commits intoAzure:devfrom
jiasli:encoding

Conversation

@jiasli
Copy link
Copy Markdown
Member

@jiasli jiasli commented May 7, 2021

Fix #17994
Adopt microsoft/knack#247

Context

On Windows with English as the system language (without UTC-8 enabled), the system encoding by default is cp1252 (Western Europe), and Python will use cp1252 as the file encoding by default.

Writing Unicode characters like 汉字 to log file results in error:

Traceback (most recent call last):
  File "C:\Users\xxx\AppData\Local\Programs\Python\Python38\lib\logging\__init__.py", line 1084, in emit
    stream.write(msg + self.terminator)
  File "C:\Users\xxx\AppData\Local\Programs\Python\Python38\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 87-88: character maps to <undefined>

Change

This PR forces command metadata log files to use UTF-8 so that az feedback works on non-UTF-8 systems as well.

@jiasli jiasli mentioned this pull request May 7, 2021
Closed
@yonzhan yonzhan added this to the S187 milestone May 7, 2021
@yonzhan
Copy link
Copy Markdown
Collaborator

yonzhan commented May 7, 2021

Core

@jiasli jiasli requested a review from yonzhan May 7, 2021 04:54
@jiasli jiasli requested a review from avanigupta May 7, 2021 04:56
yonzhan
yonzhan approved these changes May 7, 2021
View reviewed changes
Copy link
Copy Markdown
Collaborator

@yonzhan yonzhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jiasli jiasli requested a review from zhoxing-ms May 7, 2021 07:19
@jiasli
Copy link
Copy Markdown
Member Author

jiasli commented May 8, 2021

/azp run

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines bot commented May 8, 2021

Azure Pipelines successfully started running 2 pipeline(s).

@jiasli jiasli merged commit 0914bac into Azure:dev May 8, 2021
@jiasli jiasli deleted the encoding branch May 8, 2021 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@evelyn-ys evelyn-ys evelyn-ys approved these changes

@yonzhan yonzhan yonzhan approved these changes

@fengzhou-msft fengzhou-msft Awaiting requested review from fengzhou-msft

@houk-ms houk-ms Awaiting requested review from houk-ms

@jsntcy jsntcy Awaiting requested review from jsntcy

@Juliehzl Juliehzl Awaiting requested review from Juliehzl

@avanigupta avanigupta Awaiting requested review from avanigupta

@zhoxing-ms zhoxing-ms Awaiting requested review from zhoxing-ms

Assignees

@jiasli jiasli

Labels

None yet

Projects

None yet

Milestone

S187

Development

Successfully merging this pull request may close these issues.

Unable to print special characters to console output

3 participants

@jiasli @yonzhan @evelyn-ys