Change Summary

Removes unneeded .decode().

I couldn't find any justification for decoding the bytes. The code was introduced in this commit in #5674 and I don't see any PR comments explaining it. @adriangb any idea?

There are some encodings where a non-empty byte string can decode to any empty string:

>>> ''.encode('utf16')
b'\xff\xfe'
>>> ''.encode('utf16').decode('utf16')
''
>>> ''.encode('utf16').decode('utf16') == ''
True

but .decode() without arguments should always use UTF8 in Python 3 regardless of locale settings.

I've checked that any non-empty byte string with up to 3 bytes decodes to a non-empty string if it can be decoded at all:

import itertools

count = 0
total = 0
for length in [1, 2, 3]:
    for b in itertools.product(range(256), repeat=length):
        total += 1
        b = bytes(b)
        assert b
        assert len(b) == length > 0
        try:
            s = b.decode()
        except UnicodeDecodeError:
            continue
        assert s
        assert len(s) > 0
        assert b == s.encode()
        count += 1

print(count)  # 2668544
print(total)  # 16843008

I also think that any non-empty byte string should be treated as non-empty when displaying even if it can be decoded to an empty unicode string. Although I'm also not sure about revealing whether any secret value is empty.

Related issue number

Closes #7971

Checklist

• The pull request title is a good summary of the changes - it will be used in the changelog
• Unit tests for the changes exist
• Tests pass on CI
• Documentation reflects the changes where applicable
• My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Selected Reviewer: @lig