Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tarfile.data_filter wrongly rejects some tarballs with symlinks #107845

Closed
encukou opened this issue Aug 10, 2023 · 3 comments
Closed

tarfile.data_filter wrongly rejects some tarballs with symlinks #107845

encukou opened this issue Aug 10, 2023 · 3 comments
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes 3.11 bug and security fixes 3.12 bugs and security fixes 3.13 new features, bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@encukou
Copy link
Member

encukou commented Aug 10, 2023 •
edited by bedevere-bot

My implementation of PEP-706 has a bug: it wrongly determines the target of symlinks, and thus wrongly raises LinkOutsideDestinationError on some valid tarballs.

I didn't pay enough attention to this quirk of the format (which I'd like to add to TarInfolinkname docs):

For symbolic links (SYMTYPE), the linkname is relative to the directory that contains the link.
For hard links (LNKTYPE), the linkname is relative to the root of the archive.

So, in a tarball that contains the following, the links point to dir/target:

  • dir/target
  • other_dir/symlink -> ../dir/target
  • other_dir/hardlink -> dir/target

But data_filter thinks that other_dir/symlink will point to ../dir target outside the destination directory.

I have a fix but would like to test it more next week, before merging.
Sorry for the extra work this'll cause for a lot of people :(

Linked PRs
@encukou encukou added the type-bug An unexpected behavior, bug, or error label Aug 10, 2023
@bedevere-bot bedevere-bot mentioned this issue Aug 10, 2023
Merged
@sunmy2019 sunmy2019 added the stdlib Python modules in the Lib dir label Aug 10, 2023
@encukou encukou added 3.11 bug and security fixes 3.10 only security fixes 3.9 only security fixes 3.8 only security fixes 3.12 bugs and security fixes 3.13 new features, bugs and security fixes labels Aug 14, 2023
encukou added a commit that referenced this issue Aug 21, 2023
@encukou @vstinner @frenzymadness
gh-107845: Fix symlink handling for tarfile.data_filter (GH-107846)
acbd3f9 
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
@bedevere-bot bedevere-bot mentioned this issue Aug 21, 2023
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Aug 21, 2023
@encukou @vstinner
@frenzymadness @miss-islington
pythongh-107845: Fix symlink handling for tarfile.data_filter (python…
e79384c 
…GH-107846)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
@bedevere-bot bedevere-bot mentioned this issue Aug 21, 2023
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Aug 21, 2023
@encukou @vstinner
@frenzymadness @miss-islington
pythongh-107845: Fix symlink handling for tarfile.data_filter (python…
f227b1c 
…GH-107846)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
@bedevere-bot bedevere-bot mentioned this issue Aug 21, 2023
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Aug 21, 2023
@encukou @vstinner
@frenzymadness @miss-islington
pythongh-107845: Fix symlink handling for tarfile.data_filter (python…
3d1bcb8 
…GH-107846)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
Yhg1s pushed a commit that referenced this issue Aug 21, 2023
@miss-islington @encukou
@vstinner @frenzymadness
[3.12] gh-107845: Fix symlink handling for tarfile.data_filter (GH-10…
a909ec3 
…7846) (#108211)

gh-107845: Fix symlink handling for tarfile.data_filter (GH-107846)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
@encukou
Copy link
Member Author

encukou commented Aug 21, 2023

3.13 PR is merged.
3.12 PR is merged (thanks Thomas)!
Buildbots are running on the 3.11 PR: #108209
If the bots don't find any issues I'll continue with the older releases (3.10 needs adjusting tests, 3.9-3.8 doesn't apply cleanly in Git).

encukou added a commit that referenced this issue Aug 22, 2023
@miss-islington @encukou
@vstinner @frenzymadness
[3.11] gh-107845: Fix symlink handling for tarfile.data_filter (GH-10…
8e83737 
…7846) (GH-108209)

gh-107845: Fix symlink handling for tarfile.data_filter (GH-107846)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
encukou added a commit to encukou/cpython that referenced this issue Aug 22, 2023
@miss-islington @encukou
@vstinner @frenzymadness
[3.11] pythongh-107845: Fix symlink handling for tarfile.data_filter (p…
ac19f5c 
…ythonGH-107846) (pythonGH-108209)

pythongh-107845: Fix symlink handling for tarfile.data_filter (pythonGH-107846)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
@bedevere-bot bedevere-bot mentioned this issue Aug 22, 2023
Merged
encukou added a commit to encukou/cpython that referenced this issue Aug 22, 2023
@miss-islington @encukou
@vstinner @frenzymadness
[3.11] pythongh-107845: Fix symlink handling for tarfile.data_filter (p…
741330e 
…ythonGH-107846) (pythonGH-108209)

pythongh-107845: Fix symlink handling for tarfile.data_filter (pythonGH-107846)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
@bedevere-bot bedevere-bot mentioned this issue Aug 22, 2023
Merged
encukou added a commit to encukou/cpython that referenced this issue Aug 22, 2023
@miss-islington @encukou
@vstinner @frenzymadness
[3.11] pythongh-107845: Fix symlink handling for tarfile.data_filter (p…
4a91f9c 
…ythonGH-107846) (pythonGH-108209)

pythongh-107845: Fix symlink handling for tarfile.data_filter (pythonGH-107846)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
encukou added a commit to encukou/cpython that referenced this issue Aug 22, 2023
@miss-islington @encukou
@vstinner @frenzymadness
[3.11] pythongh-107845: Fix symlink handling for tarfile.data_filter (p…
63556bc 
…ythonGH-107846) (pythonGH-108209)

pythongh-107845: Fix symlink handling for tarfile.data_filter (pythonGH-107846)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
pablogsal pushed a commit that referenced this issue Aug 22, 2023
@miss-islington
[3.10] gh-107845: Fix symlink handling for tarfile.data_filter (GH-10…
7d44551 
…7846) (#108210)
ambv pushed a commit that referenced this issue Aug 22, 2023
@encukou @vstinner @frenzymadness
[3.9] gh-107845: Fix symlink handling for tarfile.data_filter (GH-107846
42deeab 
) (#108274)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
ambv pushed a commit that referenced this issue Aug 22, 2023
@encukou @vstinner @frenzymadness
[3.8] gh-107845: Fix symlink handling for tarfile.data_filter (GH-107846
b4368b5 
) (#108279)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
@LecrisUT LecrisUT mentioned this issue Sep 4, 2023
Merged
@henryiii henryiii mentioned this issue Sep 5, 2023
Closed
@LecrisUT LecrisUT mentioned this issue Sep 12, 2023
Closed
carlosroman pushed a commit to DataDog/cpython that referenced this issue Oct 11, 2023
@encukou @vstinner
@frenzymadness @carlosroman
[3.9] pythongh-107845: Fix symlink handling for tarfile.data_filter (p…
7c75c7e 
…ythonGH-107846) (python#108274)

(cherry picked from commit acbd3f9)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
@hugovk
Copy link
Member

hugovk commented Nov 9, 2023

Ready to close this issue?

@dimbleby dimbleby mentioned this issue Nov 10, 2023
Closed
4 tasks
@encukou
Copy link
Member Author

encukou commented Nov 13, 2023

Yes, thank you for the ping.
For reference, the affected versions, which can raise unwanted tarfile.LinkOutsideDestinationError on valid input, are:

sys.version_info[:3] in {(3, 8, 17), (3, 9, 17), (3, 10, 12), (3, 11, 4)}

@encukou encukou closed this as completed Nov 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes 3.11 bug and security fixes 3.12 bugs and security fixes 3.13 new features, bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
Tarfile issues
Status: Done
Milestone
No milestone
Development

No branches or pull requests
3 participants
@encukou @hugovk @sunmy2019