Skip to content

UnicodeDecodeError when setup.cfg contains non-ASCII and LC_ALL=C #1062

@benoit-pierre

Description

@benoit-pierre
> cat >setup.py <<\EOF
# -*- coding: utf-8 -*-
from setuptools import setup
setup(
name='test',
version='1.0',
description='éàïñ',
)
EOF
> env LC_ALL=C python3 ./setup.py --description
éàïñ
> cat >setup.cfg <<\EOF
[metadata]
name = test
version = 1.0
description = éàïñ
EOF
> cat >setup.py <<\EOF
from setuptools import setup
setup()
EOF
> env LC_ALL=C python3 ./setup.py --description
Traceback (most recent call last):
  File "./setup.py", line 2, in <module>
    setup()
  File "/usr/lib/python3.6/distutils/core.py", line 121, in setup
    dist.parse_config_files()
  File "/usr/lib/python3.6/site-packages/setuptools/dist.py", line 355, in parse_config_files
    _Distribution.parse_config_files(self, filenames=filenames)
  File "/usr/lib/python3.6/distutils/dist.py", line 395, in parse_config_files
    parser.read(filename)
  File "/usr/lib/python3.6/configparser.py", line 697, in read
    self._read(fp, filename)
  File "/usr/lib/python3.6/configparser.py", line 1015, in _read
    for lineno, line in enumerate(fp, start=1):
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 75: ordinal not in range(128)

With PKG-INFO always written as UTF-8, I think it would make sense to load setup.cfg as UTF-8 too.

I see in the code that there's a setuptools/py36compat.py providing a patched parse_config_files but as seen in the traceback, it's not called. Is that normal?

According to the log it was added to fix #889, but adding a test for that fails:

diff --git i/setuptools/tests/test_config.py w/setuptools/tests/test_config.py
index 8bd2a494..2b118b70 100644
--- i/setuptools/tests/test_config.py
+++ w/setuptools/tests/test_config.py
@@ -288,6 +288,15 @@ class TestMetadata:
         with get_dist(tmpdir) as dist:
             assert set(dist.metadata.classifiers) == expected

+    def test_no_interpolation(self, tmpdir):
+        fake_env(
+            tmpdir,
+            '[metadata]\n'
+            'description = %(message)s\n'
+        )
+        with get_dist(tmpdir) as dist:
+            assert dist.metadata.description == '%(message)s'
+

 class TestOptions:

Anyway, here is my attempt at fixing: master...benoit-pierre:fix_889_and_non-ascii_in_setup.cfg

I can make proper PR(s) for those changes that are OK.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Needs TriageIssues that need to be evaluated for severity and status.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions