Skip to content

bpo-32304: Fix distutils upload for tar files ending with b'\r'#5264

Merged
merwok merged 7 commits into
python:masterfrom
bbayles:distutils-upload-cr
Jan 26, 2018
Merged

bpo-32304: Fix distutils upload for tar files ending with b'\r'#5264
merwok merged 7 commits into
python:masterfrom
bbayles:distutils-upload-cr

Conversation

@bbayles

@bbayles bbayles commented Jan 21, 2018

Copy link
Copy Markdown
Contributor

This PR fixes issue 32304, in which distutils's upload command corrupts tar files that end with b'\r' by appending b'\n' to them (here).

The original report describes a simple fix, which is endorsed. I've provided that fix here, plus a test that ensures that (1) the "content" doesn't get the newline appended, (2) the other keys do.

Of course, one wonders if fixing '\r' without '\n' is so important now that no major OSs use it?

https://bugs.python.org/issue32304

Comment thread Lib/distutils/command/upload.py Outdated
body.write(value)
if value and value[-1:] == b'\r':
if value and (value[-1:] == b'\r') and (key != 'content'):
body.write(b'\n') # write an extra newline (lurve Macs)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m wondering if this if block is even needed. Only Mac OS 9 used \n as end of line character, right?
We could delete both of these lines.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I agree! (I alluded to this above)

Shall I change this PR to do that? It would save having to add the test case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep the test case!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I've zapped the newline correction. The test is still there, but a little odd - "make sure we don't append '\n' to anything," so I added an explanatory comment.

@merwok merwok self-assigned this Jan 23, 2018
@merwok merwok added type-bug An unexpected behavior, bug, or error needs backport to 3.6 and removed CLA not signed labels Jan 23, 2018
Comment thread Lib/distutils/tests/test_upload.py Outdated
cmd.run()

# an extra character should have been added to the description,
# but not to the content

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove obsolete comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops, good catch.

Comment thread Lib/distutils/tests/test_upload.py Outdated
dist_files = [(command, pyversion, filename)]
self.write_file(self.rc, PYPIRC_LONG_PASSWORD)

# other fields that end with \r should not be modified.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update comment: other fields that ended with \r used to be modified, now are preserved

@merwok

merwok commented Jan 25, 2018

Copy link
Copy Markdown
Member

Thanks a lot! I will merge tomorrow and this will be in the next beta.

Comment thread Doc/whatsnew/3.7.rst
The ``upload`` command now longer tries to change CR end-of-line characters
to CRLF. This fixes a corruption issue with sdists that ended with a byte
equivalent to CR.
(Contributed by Bo Bayles in :issue:`32304`.)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bbayles I used the name found in your github profile, is that ok?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's fine. Thanks!

@merwok merwok merged commit 2fc98ae into python:master Jan 26, 2018
@miss-islington

Copy link
Copy Markdown
Contributor

Thanks @bbayles for the PR, and @merwok for merging it 🌮🎉.. I'm working now to backport this PR to: 2.7, 3.6.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@miss-islington

Copy link
Copy Markdown
Contributor

Sorry, @bbayles and @merwok, I could not cleanly backport this to 3.6 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 2fc98ae115e2a2095a0bcf388c27a878aafdb454 3.6

@miss-islington

Copy link
Copy Markdown
Contributor

Sorry, @bbayles and @merwok, I could not cleanly backport this to 2.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 2fc98ae115e2a2095a0bcf388c27a878aafdb454 2.7

bbayles added a commit to bbayles/cpython that referenced this pull request Jan 26, 2018
@bedevere-bot

Copy link
Copy Markdown

GH-5330 is a backport of this pull request to the 3.6 branch.

@bedevere-bot

Copy link
Copy Markdown

GH-5331 is a backport of this pull request to the 2.7 branch.

merwok pushed a commit that referenced this pull request Jan 27, 2018
merwok pushed a commit that referenced this pull request Jan 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type-bug An unexpected behavior, bug, or error

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants