Validate VCS urls in hash-checking mode using their commit hashes by sbidoul · Pull Request #11968 · pypa/pip

sbidoul · 2023-04-16T14:13:20Z

The default is potentially dangerous.

sbidoul · 2023-04-16T14:24:19Z

+                raise DependencyVcsHashNotSupported()
+            return VcsHashes()
        if req.link.is_existing_dir():
            raise DirectoryUrlHashUnsupported()


Note to self: when req.is_wheel_from_cache is True, link is the cached wheel and not the link to the artifact that was downloaded to build the wheel. So this function may not necessarily do what one expects when reasoning on req.link.

uranusjr · 2023-04-17T09:11:22Z

+            elif isinstance(req.download_info.info, VcsInfo):
+                if not req.user_supplied:
+                    raise DependencyVcsHashNotSupported()


I wonder if this would break anything. How can this branch be reached?

This would raise in require-hashes mode only, if you have a dependency as a git direct URL that is obtained from cache.

I think these should not be considered hash-valid unless the commit hash has been provided by the user, either as a requirement file or a constraint.

This PR still needs work. Next step is adding a lot of test coverage. Then possibly some refactoring.

If an immutable VCS URL is both user-supplied and a dependency, would this cause the latter specifier break installation?

It should not, because when the dependency is examined by the resolver it is "merged" with the user-supplied one? That's definitely a test case to consider.

uranusjr · 2023-04-17T09:11:47Z

+
+    pass


Suggested change

pass

sbidoul · 2023-06-25T11:21:23Z

This is still on my list but unfortunately I won't have time to finish it in time for 23.2.

pfmoore · 2023-06-25T12:49:30Z

I removed #6469 from the 23.2 milestone for consistency.

bcail · 2023-07-13T20:00:05Z

This PR seemed to work for me, for installing a package from a git repo.

adamzap · 2023-09-05T18:24:22Z

Is there anything we can do to move this forward? It seems to work. Thanks!

Co-authored-by: Ed Morley <501702+edmorley@users.noreply.github.com>

sbidoul · 2023-10-01T14:51:45Z

Tests to do

test git url + --hash errors out (with and without --require-hashes, with and without cache)
test git url + not commit hash errors out with --require-hashes
test git url + not commit hash don't error out without --require-hashes
test that an indirect dependency specified as a git url fails in hash-checking mode

bcail · 2023-10-30T14:58:52Z

@sbidoul Here's a test for the second and third points you mentioned. Is that helpful? Should I work on expanding it more?

diff --git a/tests/unit/test_req.py b/tests/unit/test_req.py
index d70fa9da5..22cc7256c 100644
--- a/tests/unit/test_req.py
+++ b/tests/unit/test_req.py
@@ -241,6 +241,35 @@ class TestRequirementSet:
             ):
                 resolver.resolve(reqset.all_requirements, True)
 
+    def test_vcs_url_without_commit_hash(self, data: TestData) -> None:
+        """VCS URLs should raise errors when --require-hashes is on, if
+        they don't have a commit hash.
+        """
+        reqset = RequirementSet()
+        reqset.add_unnamed_requirement(
+            get_processed_req_from_line(
+                "git+https://github.com/pypa/pip-test-package",
+                lineno=1,
+            )
+        )
+
+        finder = make_test_finder(find_links=[data.find_links])
+
+        with self._basic_resolver(finder, require_hashes=True) as resolver:
+            with pytest.raises(
+                HashErrors,
+                match=(
+                    r"Can't verify hashes for these VCS requirements because their ref is "
+                    r"not immutable:\n"
+                    r"    git\+https://github\.com/pypa/pip-test-package \(from -r "
+                    r"file \(line 1\)\)"
+                ),
+            ):
+                resolver.resolve(reqset.all_requirements, True)
+
+        with self._basic_resolver(finder, require_hashes=False) as resolver:
+            resolver.resolve(reqset.all_requirements, True)
+

FrancoisPerez · 2024-05-06T08:55:55Z

Any way we can help this PR move forward @sbidoul ?

ichard26 · 2025-11-06T05:34:41Z

@sbidoul do you wish to repick up this PR at some point?

sbidoul · 2025-11-06T05:50:15Z

Possibly. I'd need to delve into this again but I suspect it will be necessary in some form or another to support installation from pylock.

sbidoul · 2026-03-28T18:04:07Z

Finishing this is necessary to support pylock vcs entries.

Don't set a default value for ireq.hashes() function

061d3e9

The default is potentially dangerous.

sbidoul added the C: vcs pip's interaction with version control systems like git, svn and bzr label Apr 16, 2023

sbidoul added this to the 23.2 milestone Apr 16, 2023

sbidoul mentioned this pull request Apr 16, 2023

Validate VCS urls in hash-checking mode using their commit hashes #6469

Open

sbidoul commented Apr 16, 2023

View reviewed changes

This was referenced Apr 16, 2023

Only use hashes in requirements.txt if there are no VCS repositories python-poetry/poetry-plugin-export#69

Open

Enable hashes in generated requirements.txt file moneymeets/python-poetry-buildpack#19

Open

sbidoul force-pushed the vcs-require-hashes branch from 51173ff to b5608e7 Compare April 16, 2023 15:27

psf-chronographer Bot added the bot:chronographer:provided label Apr 16, 2023

sbidoul added 2 commits April 16, 2023 17:34

Don't use deprecated git protocol in test

88ac144

Support immutable VCS references in hash-checking mode.

a1cd61b

sbidoul force-pushed the vcs-require-hashes branch from b5608e7 to a1cd61b Compare April 16, 2023 15:35

uranusjr reviewed Apr 17, 2023

View reviewed changes

Comment thread src/pip/_internal/utils/hashes.py

Comment on lines +160 to +161

pass

uranusjr Apr 17, 2023

Copy link
Copy Markdown

Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

pass

sbidoul mentioned this pull request May 29, 2023

editable cannot be installed when requiring hashes #4995

Open

sbidoul removed this from the 23.2 milestone Jun 25, 2023

atugushev mentioned this pull request Jul 9, 2023

Ignore hashes for version control repositories jazzband/pip-tools#994

Open

edmorley reviewed Sep 5, 2023

View reviewed changes

Comment thread src/pip/_internal/exceptions.py Outdated

Comment thread docs/html/topics/secure-installs.md Outdated

Comment thread src/pip/_internal/operations/prepare.py Outdated

sbidoul and others added 3 commits October 1, 2023 16:50

Update src/pip/_internal/operations/prepare.py

7aab44c

Co-authored-by: Ed Morley <501702+edmorley@users.noreply.github.com>

Update docs/html/topics/secure-installs.md

c59af71

Co-authored-by: Ed Morley <501702+edmorley@users.noreply.github.com>

Update src/pip/_internal/exceptions.py

8003490

Co-authored-by: Ed Morley <501702+edmorley@users.noreply.github.com>

pradyunsg removed the bot:chronographer:provided label Dec 20, 2023

psf-chronographer Bot added the bot:chronographer:provided label Dec 20, 2023

sbidoul mentioned this pull request Mar 10, 2024

Generate hashes in requirements.txt sbidoul/pip-deepfreeze#42

Open

synapticarbors mentioned this pull request Sep 13, 2024

feat(export): pypi pip requirements export prefix-dev/pixi#2049

Draft

sbidoul mentioned this pull request Mar 28, 2026

Add initial support for -r pylock.toml (experimental) #13876

Merged

7 tasks

sbidoul mentioned this pull request Apr 24, 2026

Support VCS entries from a pylock.toml which also has other entries with hashes #13944

Open

1 task

coderabbitai Bot mentioned this pull request May 8, 2026

Prepare revrem for public launch GitCmurf/revrem#6

Merged

Conversation

sbidoul commented Apr 16, 2023

Uh oh!

sbidoul Apr 16, 2023

Choose a reason for hiding this comment

Uh oh!

uranusjr Apr 17, 2023

Choose a reason for hiding this comment

Uh oh!

sbidoul Apr 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

uranusjr Apr 17, 2023

Choose a reason for hiding this comment

Uh oh!

sbidoul Apr 17, 2023

Choose a reason for hiding this comment

Uh oh!

uranusjr Apr 17, 2023

Choose a reason for hiding this comment

Uh oh!

sbidoul commented Jun 25, 2023

Uh oh!

pfmoore commented Jun 25, 2023

Uh oh!

bcail commented Jul 13, 2023

Uh oh!

adamzap commented Sep 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sbidoul commented Oct 1, 2023

Uh oh!

bcail commented Oct 30, 2023

Uh oh!

FrancoisPerez commented May 6, 2024

Uh oh!

ichard26 commented Nov 6, 2025

Uh oh!

sbidoul commented Nov 6, 2025

Uh oh!

sbidoul commented Mar 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

sbidoul Apr 17, 2023 •

edited

Loading

adamzap commented Sep 5, 2023 •

edited

Loading