feat: Add the ability to skip install of local source dependencies by olivier-lacroix · Pull Request #3092 · prefix-dev/pixi

olivier-lacroix · 2025-02-09T07:48:16Z

This PR implements an additional --skip argument for pixi install, that will skip installation of packages

As this argument is intended to be used in cases where these local dependencies may be absent, we do not want to update the lockfile, and the use of --skip requires the "--frozen" argument.

With this PR, a dockerfile using pixi may look like

FROM python:3.12.2-slim

WORKDIR /app

COPY pixi.lock pixi.toml .

RUN pixi install --frozen --skip package-name

COPY sample-package . 

RUN pixi install --frozen

ruben-arts

Thanks @olivier-lacroix, would you be able to add a test to validate that this works for both conda and PyPI path dependencies. And add the CLI documentation?

src/cli/install.rs

olivier-lacroix · 2025-02-10T08:20:30Z

Thanks @olivier-lacroix, would you be able to add a test to validate that this works for both conda and PyPI path dependencies. And add the CLI documentation?

Can do yes :-)

olivier-lacroix · 2025-02-11T23:03:51Z

@ruben-arts what sort of test(s) do you think would be best? unit test(s) and/or integration tests?

ruben-arts · 2025-02-12T07:46:05Z

For Pypi you might use the src/install_pypi/plan/test code. But for the full picture I would write an integration test. Please try to keep these fast, small packages from the local test data channels.

src/lib.rs

olivier-lacroix · 2025-02-15T11:10:46Z

@ruben-arts integration test added. Let me know what you think.

src/lock_file/update.rs

tdejager · 2025-02-19T05:27:20Z

Yeah you are right that should be fine :) only skipping the folders then, I suppose. You would still want to install sdists from paths and direct urls right? Or do you want to skip those as well with the flag set?

…

On Tue, 18 Feb 2025 at 22:30, Olivier Lacroix ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In src/lock_file/update.rs <#3092 (comment)>: > @@ -483,6 +498,10 @@ impl<'p> LockFileDerivedData<'p> { Ok(locked_env .conda_packages(platform) + .map(|packages| { + packages + .filter(|p| !no_path_dependencies || matches!(p.location(), UrlOrPath::Url(_))) @tdejager <https://github.com/tdejager> would detecting if the path is a folder or not work? — Reply to this email directly, view it on GitHub <#3092 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADF4XRTROOCA7B45NFCVQD2QORADAVCNFSM6AAAAABWYRQH5OVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDMMRVGA3TMMZXGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

olivier-lacroix · 2025-02-19T11:26:31Z

Yeah you are right that should be fine :) only skipping the folders then, I suppose. You would still want to install sdists from paths and direct urls right? Or do you want to skip those as well with the flag set?

I guess so. the intent is really to skip packages that are being developed.

Quick one, is there a straightforward way to know if a Utf8TypedPathBuf is a directory? not too sure how to convert this to a type that can tell me if its a directory, without going through strings, which, I assume, could be pretty brittle.

tdejager · 2025-02-21T14:27:39Z

Utf8TypedPathBuf

I think as_std_path().is_dir(), does that work? Also, this does pivot the PR a bit but we could also have a --skip-install <pkg> that could skip the installation of any package, do you think that would make sense?

olivier-lacroix · 2025-02-21T23:36:03Z

I think as_std_path().is_dir(), does that work?

That'd be nice, but this is available in the camino crate, not the typed-path crate that is in use here . I have based the logic on file extension: A location is deemed a 'local source' if it is a path that is NOT a wheel or an egg file

olivier-lacroix · 2025-02-21T23:46:41Z

Alright, @tdejager @ruben-arts this PR is about ready to be merged I think.

@tdejager , a --skip flag could definitely be added if there is demand for it. Not sure what use cases there would be for ignoring a specific package though. I propose to defer adding this until some users ask for it? I created #3182 to track

I was not 100% happy with the flag name --no-path-dependencies, as local built artifact are not skipped, and switched to --skip-local-sources. this would make it consistent with a potential future --skip flag as well 😃 . Happy with that?

tdejager · 2025-02-27T11:03:29Z

Hey @olivier-lacroix, @ruben-arts and myself have been discussing this PR offline just now, and we wanted to report back. So while we do think the idea is good there are a couple of things, also because of my suggestion that do not work out. So supposing you still want to skip development dependencies:

I wrongly suggested as_source as that also includes git dependencies that you do want to include in the initial layering.
I think you also still want to include the sdists that would not be excluded, e.g: .zip, .tar.gz and maybe other extensions as well: https://github.com/astral-sh/uv/blob/f9c1684b96d1b3a88c4d1f4adb4db5e018a7d54f/crates/uv-distribution-filename/src/extension.rs#L29

Additionally, @ruben-arts and I discussed that we think that a general --skip makes more sense, as it would simplify some of the logic and you would need to be very explicit what you skip.

We are really sorry about the tardiness regarding this PR, and totally understand that it would be to much work to change this so happy to take this over from you if you do not want to make the change.

olivier-lacroix · 2025-07-30T04:44:54Z

Alright, I

removed the --skip for reinstall, this can be added later if needed.
added the conda test

this PR should be good to go I think @tdejager

tdejager

Nice! User tested it a bit and it all seems to work. One final thing I would like to see, if thats possible is that we give a warning and do not change the installation method if a non-existing name has been added.

Now currently if we make a typo - z in clickz:

 pixi i --frozen --skip clickz                                                                                                                                                                                                    
✔ The default environment has been installed excluding clickz.

Would be better if a tracing warn was printed and the excluding is removed.

Thanks!

olivier-lacroix · 2025-07-31T08:52:19Z

Ok @tdejager . will do!

olivier-lacroix · 2025-08-01T01:40:37Z

@tdejager hum, this isn't as straightforward as I thought 😓. My plan was to bubble up the call stack the packages that were actually skipped and report on that; but, if/when the cache is hit, we can't know this... any idea?

tdejager · 2025-08-01T09:07:03Z

Would you be able to check the package names that are locked, even though that does not change we should have access about that. And I guess we don't even need to bubble up anymore.

olivier-lacroix · 2025-08-04T09:45:59Z

Alright, hopefully getting there @tdejager 😅 .

I have:

Implemented a public method get_skipped_package_names that can test which packages in the lock file will be matched against a list of names, covering the cache hit case. Thanks for the suggestion!
Moved the filtering logic to the LockedPackageRef level (i.e. before specification into pypi or conda packages/records), which simplifies things a fair bit (I think).
Added a test and modified stdout / stderr output to cover your typo case.

Let me know what you think!

tdejager

Looks good to me @olivier-lacroix, let's merge when the CI passes. As always, great work!

tdejager · 2025-08-04T09:56:37Z

@pavelzw might be worth it to update the docker documentation, this might be useful for setting up correct layering.

olivier-lacroix · 2025-08-04T21:27:27Z

@ruben-arts this PR requires your approval to be merged.

ruben-arts

Thanks @olivier-lacroix !

Work around the circular dependency between pixi lock files and dynamic git-based versioning (versioningit). The issue: - Lock file contains version computed from git state - Committing lock file changes git state → version mismatch - CI with --locked fails because versions don't match Solution: Use pixi's --skip flag (added in v0.51.0) to skip the local editable package during locked install, then install it separately with pip. This preserves lock file integrity for all external deps while allowing the local package version to float. Also added documentation in README explaining the issue and solution for future reference. See: prefix-dev/pixi#3092 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add the ability to skip install of path dependencies

8ca2c1e

olivier-lacroix changed the title ~~Add the ability to skip install of local path dependencies~~ feat: Add the ability to skip install of local path dependencies Feb 9, 2025

Improve argument documentation

f3b1c15

ruben-arts requested changes Feb 10, 2025

View reviewed changes

src/cli/install.rs Show resolved Hide resolved

olivier-lacroix added 2 commits February 15, 2025 21:47

Add integration test and doc

43476d9

Merge branch 'main' into feat/no-path-dep

0f4dd2b

olivier-lacroix commented Feb 15, 2025

View reviewed changes

src/lib.rs Show resolved Hide resolved

olivier-lacroix commented Feb 15, 2025

View reviewed changes

src/lock_file/update.rs Outdated Show resolved Hide resolved

fixes

9a7c9df

olivier-lacroix force-pushed the feat/no-path-dep branch from 66caae2 to 9a7c9df Compare February 15, 2025 11:20

Test if pixi record is source

5965f12

olivier-lacroix force-pushed the feat/no-path-dep branch from d0391b1 to 5965f12 Compare February 18, 2025 09:38

olivier-lacroix force-pushed the feat/no-path-dep branch from 0d5875e to 38fdf94 Compare February 21, 2025 23:00

Limit to pypi source

98bc642

olivier-lacroix force-pushed the feat/no-path-dep branch from 38fdf94 to 98bc642 Compare February 21, 2025 23:34

Switch to --skip-local-sources

d5a45fa

olivier-lacroix requested a review from ruben-arts February 21, 2025 23:46

olivier-lacroix mentioned this pull request Feb 21, 2025

Consider allow skipping install of specific package(s) #3182

Closed

Update strings / doc

65c2ced

olivier-lacroix changed the title ~~feat: Add the ability to skip install of local path dependencies~~ feat: Add the ability to skip install of local source dependencies Feb 22, 2025

Re-add conda test

f4722e5

olivier-lacroix force-pushed the feat/no-path-dep branch from bf06ab9 to f4722e5 Compare July 30, 2025 04:14

Remove --skip for reinstall

28743e0

Merge branch 'main' into feat/no-path-dep

efb4558

tdejager reviewed Jul 31, 2025

View reviewed changes

olivier-lacroix added 5 commits August 4, 2025 10:08

Add function to get skipped packages

440c736

Merge branch 'main' into feat/no-path-dep

c3ae3a1

report

6569e6d

Add test on non-existent package being skipped

e563416

Filter at LockedPackageRef level

0f1d7fc

olivier-lacroix added 2 commits August 4, 2025 19:47

improve warning

13db8a4

improve message

4ddef55

tdejager approved these changes Aug 4, 2025

View reviewed changes

tdejager enabled auto-merge (squash) August 4, 2025 11:10

tdejager approved these changes Aug 4, 2025

View reviewed changes

Merge branch 'main' into feat/no-path-dep

00c2c21

ruben-arts approved these changes Aug 5, 2025

View reviewed changes

tdejager merged commit 1720903 into prefix-dev:main Aug 5, 2025
41 checks passed

BrewTestBot mentioned this pull request Aug 12, 2025

pixi 0.51.0 Homebrew/homebrew-core#233187

Merged

tdejager mentioned this pull request Aug 26, 2025

feat: ability to install package subsets #4404

Merged

KedoKudo mentioned this pull request Jan 1, 2026

fix(ci): use --skip flag for dynamic versioning workaround neutrons/python_project_template#133

Merged

2 tasks

peterfpeterson mentioned this pull request Jan 27, 2026

Ability to skip editable install prefix-dev/setup-pixi#246

Open

Conversation

olivier-lacroix commented Feb 9, 2025 • edited by ruben-arts Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ruben-arts left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

olivier-lacroix commented Feb 10, 2025

Uh oh!

olivier-lacroix commented Feb 11, 2025

Uh oh!

ruben-arts commented Feb 12, 2025

Uh oh!

Uh oh!

olivier-lacroix commented Feb 15, 2025

Uh oh!

Uh oh!

tdejager commented Feb 19, 2025 via email

Uh oh!

olivier-lacroix commented Feb 19, 2025

Uh oh!

tdejager commented Feb 21, 2025

Uh oh!

olivier-lacroix commented Feb 21, 2025

Uh oh!

olivier-lacroix commented Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tdejager commented Feb 27, 2025

Uh oh!

olivier-lacroix commented Jul 30, 2025

Uh oh!

tdejager left a comment

Choose a reason for hiding this comment

Uh oh!

olivier-lacroix commented Jul 31, 2025

Uh oh!

olivier-lacroix commented Aug 1, 2025

Uh oh!

tdejager commented Aug 1, 2025

Uh oh!

olivier-lacroix commented Aug 4, 2025

Uh oh!

tdejager left a comment

Choose a reason for hiding this comment

Uh oh!

tdejager commented Aug 4, 2025

Uh oh!

olivier-lacroix commented Aug 4, 2025

Uh oh!

ruben-arts left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

olivier-lacroix commented Feb 9, 2025 •

edited by ruben-arts

Loading

olivier-lacroix commented Feb 21, 2025 •

edited

Loading