Skip to content
This repository was archived by the owner on May 14, 2026. It is now read-only.

feat(git-fetcher): install git-hosted packages via git CLI + preparePackage (#436 §B+D)#446

Merged
zkochan merged 4 commits into
mainfrom
feat/436-git-fetcher
May 13, 2026
Merged

feat(git-fetcher): install git-hosted packages via git CLI + preparePackage (#436 §B+D)#446
zkochan merged 4 commits into
mainfrom
feat/436-git-fetcher

Conversation

@zkochan

@zkochan zkochan commented May 13, 2026

Copy link
Copy Markdown
Member

Summary

Builds on §A (#440). Adds a new pacquet-git-fetcher crate that handles LockfileResolution::Git snapshots during frozen-lockfile installs, plus the supporting preparePackage port. The install dispatcher now routes git resolutions through it instead of returning UnsupportedResolution { resolution_kind: "git" }.

New crate pacquet-git-fetcher

  • GitFetcher shells out to the system git (clone, or init + fetch --depth 1 on hosts in git_shallow_hosts), checks out the pinned commit, verifies via rev-parse HEAD, runs preparePackage, removes .git, computes a packlist, and imports the resulting file set into the CAS. Surfaces a friendly GitNotFound when git isn't on PATH — pacquet does not bundle it. Mirrors fetching/git-fetcher/src/index.ts.
  • prepare_package ports exec/prepare-package/src/index.ts. Reads the manifest, decides via package_should_be_built, honors an AllowBuildFn (the dispatcher adapts pacquet's AllowBuildPolicy into this shape), runs <pm>-install plus any prepublish / prepack / publish scripts via the existing pacquet_executor::run_lifecycle_hook, then deletes node_modules. Emits the upstream error codes GIT_DEP_PREPARE_NOT_ALLOWED, ERR_PNPM_PREPARE_PACKAGE, INVALID_PATH.
  • detect_preferred_pm sniffs the cloned tree for a lockfile to pick the install pm. Workspace-root walking deferred.
  • Minimal packlist honors the manifest's files field with a tiny * / ** / ? glob matcher (? and single * reject /; ** matches across directories), always-includes the README / LICENSE / package.json set, and always-excludes .git, node_modules, lockfiles for sibling pms, and common cruft. Full .npmignore / .gitignore semantics and bundleDependencies walking are deferred and tracked in module docs.

Wiring in pacquet-package-manager

  • InstallPackageBySnapshot now matches LockfileResolution::Git and runs the fetcher in place of DownloadTarballToStore.
  • &AllowBuildPolicy is computed once per install in InstallFrozenLockfile and threaded through CreateVirtualStoreInstallPackageBySnapshot.
  • snapshot_cache_key returns Ok(None) for Git resolutions so they cold-batch. Warm prefetch for git-hosted slots is a follow-up alongside §C.

Supporting changes

  • GitResolution.path: Option<String> so a lockfile pinning a sub-directory of a git-hosted monorepo deserializes without tripping deny_unknown_fields.
  • Config::git_shallow_hosts with pnpm's default list + pnpm-workspace.yaml override.
  • pacquet_executor::run_lifecycle_hook is now pub so the preparePackage port can dispatch by arbitrary stage name.

Out of scope (follow-ups for #436)

  • §C — git-hosted tarball fetcher (the gitHosted: true shape).
  • §E — full integration-test matrix (plans/TEST_PORTING.md 563-572). This PR ships crate-level tests against a local bare repo that prove the path works end-to-end.
  • Warm prefetch for git resolutions (re-clones on every install today — slow but correct).
  • Full npm-packlist semantics (.npmignore, gitignore layering, bundleDependencies walking).

Test plan

  • cargo nextest run -p pacquet-git-fetcher — 31 tests pass:
    preferred-pm sniffer (7), packlist MVP (7 — including a regression test pinning a?b/index.js does NOT match a/b/index.js), preparePackage manifest paths (10), and 7 integration tests using a local bare git repo (clone → checkout → import; commit mismatch; GIT_DEP_PREPARE_NOT_ALLOWED; shallow-host predicate; URL host parsing).
  • cargo nextest run -p pacquet-lockfileGitResolution { path } round-trip.
  • cargo nextest run -p pacquet-configpnpm-workspace.yaml's gitShallowHosts overrides the default wholesale (not merged).
  • cargo nextest run -p pacquet-package-manager — full suite (218 tests) still green after dispatcher wiring.
  • just ready — 685 tests run, 685 passed, 2 skipped. Clippy clean.
  • cargo doc --no-deps --workspace --document-private-items with RUSTDOCFLAGS=-D warnings — clean (Doc CI job green).
  • taplo format --check
  • just dylint

Written by an agent (Claude Code, claude-opus-4-7).

Copilot AI review requested due to automatic review settings May 13, 2026 11:14
@coderabbitai

coderabbitai Bot commented May 13, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Adds a new pacquet-git-fetcher crate for git-hosted packages (shallow/full), implements prepare-package and packlist logic, exposes Config.git_shallow_hosts and lockfile GitResolution.path, and threads allow_build_policy into the install pipeline to run GitFetcher for git resolutions.

Changes

Git-backed package resolution and shallow-clone support

Layer / File(s) Summary
Workspace dependency and Config git_shallow_hosts
Cargo.toml, crates/config/src/defaults.rs, crates/config/src/lib.rs, crates/config/src/workspace_yaml.rs, crates/config/src/workspace_yaml/tests.rs
Adds workspace dependency pacquet-git-fetcher; introduces default_git_shallow_hosts(); adds Config::git_shallow_hosts: Vec<String> and WorkspaceSettings::git_shallow_hosts: Option<Vec<String>> with a test verifying apply semantics.
Executor lifecycle API
crates/executor/src/lib.rs, crates/executor/src/lifecycle.rs
Reformats lifecycle re-export and exports run_lifecycle_hook as public with a doc comment about parent_env snapshot semantics.
Git fetcher crate manifest and root
crates/git-fetcher/Cargo.toml, crates/git-fetcher/src/lib.rs
Adds new pacquet-git-fetcher crate manifest and crate root exposing internal modules and public re-exports.
Errors and public types
crates/git-fetcher/src/error.rs
Adds PreparePackageError, PacklistError, and GitFetcherError public enums for the crate's error surface.
Prepare-package, packlist, preferred-PM detection
crates/git-fetcher/src/prepare_package.rs, crates/git-fetcher/src/prepare_package/tests.rs, crates/git-fetcher/src/packlist.rs, crates/git-fetcher/src/packlist/tests.rs, crates/git-fetcher/src/preferred_pm.rs, crates/git-fetcher/src/preferred_pm/tests.rs
Implements prepare_package (build gating, safe subdir resolution, lifecycle execution and cleanup), packlist (inclusion/exclusion rules and glob matcher), and detect_preferred_pm (lockfile precedence), with unit tests.
GitFetcher core workflow and helpers
crates/git-fetcher/src/fetcher.rs, crates/git-fetcher/src/fetcher/tests.rs
Adds GitFetcher orchestration: shallow vs clone path, git exec helpers (Windows long-path), commit verification, run prepare_package, delete .git, compute packlist, import files to CAS (preserve exec bit on Unix), and return CAS mapping; tests cover parsing, shallow logic, commit mismatch, CAS import, and build gating.
Lockfile GitResolution.path and install integration
crates/lockfile/src/resolution.rs, crates/lockfile/src/resolution/tests.rs, crates/package-manager/Cargo.toml, crates/package-manager/src/create_virtual_store.rs, crates/package-manager/src/install_frozen_lockfile.rs, crates/package-manager/src/install_package_by_snapshot.rs, crates/package-manager/src/install_package_from_registry/tests.rs
Adds optional GitResolution.path: Option<String> for sub-directory packaging and updates lockfile tests; threads allow_build_policy through CreateVirtualStore and InstallPackageBySnapshot, routes Git snapshots to the cold batch, invokes GitFetcher for Git resolutions (new GitFetch error variant), factors tarball integrity helper, and updates package-manager dependencies and tests.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related issues

Possibly related PRs

  • pnpm/pacquet#442: Overlaps CreateVirtualStore warm-vs-cold snapshot handling and related refactors.

  • pnpm/pacquet#391: Related to making lifecycle APIs public and lifecycle execution plumbing.

  • pnpm/pacquet#372: Related InstallPackageBySnapshot changes and requester/progress plumbing.

  • Poem

🐰 I hopped to fetch a Git-hosted prize,
Shallow clones where hostname wisdom lies.
I ran the scripts and packed files with care,
Wrote to CAS, left no loose hair.
Hooray for tidy fetches—hop, hop, hooray!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: adding a git-fetcher crate to install git-hosted packages and porting preparePackage functionality.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The pull request provides a comprehensive description covering the new crate, wiring, supporting changes, out-of-scope work, and test results. All major sections align with the template requirements.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/436-git-fetcher

Comment @coderabbitai help to get the list of available commands and usage tips.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for installing LockfileResolution::Git snapshots during frozen-lockfile installs by introducing a new git fetcher crate, wiring it into the install dispatcher, and adding supporting config + lockfile type updates.

Changes:

  • Introduces pacquet-git-fetcher to clone/fetch via system git, checkout pinned commit, run preparePackage, compute a packlist, and import results into the CAS.
  • Wires git resolutions through InstallPackageBySnapshot and threads AllowBuildPolicy through the install pipeline.
  • Extends config/workspace YAML with git_shallow_hosts defaults + override behavior, and extends lockfile git resolution with optional path.

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
crates/package-manager/src/install_package_from_registry/tests.rs Updates test config construction for new git_shallow_hosts field.
crates/package-manager/src/install_package_by_snapshot.rs Routes LockfileResolution::Git through GitFetcher; refactors tarball URL/integrity resolution.
crates/package-manager/src/install_frozen_lockfile.rs Computes AllowBuildPolicy once per install and threads it through.
crates/package-manager/src/create_virtual_store.rs Threads AllowBuildPolicy; routes git resolutions to cold-batch path (Ok(None)).
crates/package-manager/Cargo.toml Adds pacquet-git-fetcher and ssri dependency.
crates/lockfile/src/resolution/tests.rs Adds git resolution { path } round-trip coverage.
crates/lockfile/src/resolution.rs Adds GitResolution.path: Option<String>.
crates/git-fetcher/src/prepare_package/tests.rs Adds unit tests for prepare_package behavior and path safety.
crates/git-fetcher/src/prepare_package.rs Implements preparePackage port including allow-build gating and safe subdir handling.
crates/git-fetcher/src/preferred_pm/tests.rs Adds tests for lockfile-based preferred-PM detection.
crates/git-fetcher/src/preferred_pm.rs Implements preferred package manager detection via lockfile sniffing.
crates/git-fetcher/src/packlist/tests.rs Adds packlist MVP tests (files field, exclusions, always-included files, basic globs).
crates/git-fetcher/src/packlist.rs Implements packlist MVP + minimal glob matcher.
crates/git-fetcher/src/lib.rs New crate module wiring + re-exports.
crates/git-fetcher/src/fetcher/tests.rs Adds integration tests using a local bare git repo and git presence skip.
crates/git-fetcher/src/fetcher.rs Implements git CLI fetch/clone + checkout verification + prepare + packlist + CAS import.
crates/git-fetcher/src/error.rs Adds diagnostics/error surface for git fetcher, prepare, and packlist.
crates/git-fetcher/Cargo.toml New crate manifest.
crates/executor/src/lifecycle.rs Makes run_lifecycle_hook public for preparePackage port.
crates/executor/src/lib.rs Re-exports run_lifecycle_hook.
crates/config/src/workspace_yaml/tests.rs Adds test ensuring gitShallowHosts replaces defaults wholesale.
crates/config/src/workspace_yaml.rs Adds git_shallow_hosts to workspace YAML settings and applies it to config.
crates/config/src/lib.rs Exposes default_git_shallow_hosts; adds Config.git_shallow_hosts.
crates/config/src/defaults.rs Adds pnpm-aligned default list for git_shallow_hosts.
Cargo.toml Adds workspace dependency entry for pacquet-git-fetcher.
Cargo.lock Locks new crate dependency set.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/git-fetcher/src/fetcher.rs
Comment thread crates/git-fetcher/src/packlist.rs Outdated
@github-actions

github-actions Bot commented May 13, 2026

Copy link
Copy Markdown

Micro-Benchmark Results

Linux

group                          main                                   pr
-----                          ----                                   --
tarball/download_dependency    1.02     18.2±0.34ms   238.0 KB/sec    1.00     17.9±0.29ms   242.2 KB/sec

@codecov

codecov Bot commented May 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 70.24482% with 158 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.45%. Comparing base (71ad815) to head (d3aa967).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
crates/git-fetcher/src/prepare_package.rs 44.06% 66 Missing ⚠️
...package-manager/src/install_package_by_snapshot.rs 45.33% 41 Missing ⚠️
crates/git-fetcher/src/fetcher.rs 82.22% 24 Missing ⚠️
crates/git-fetcher/src/packlist.rs 88.81% 17 Missing ⚠️
crates/package-manager/src/create_virtual_store.rs 23.07% 10 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #446      +/-   ##
==========================================
+ Coverage   87.04%   87.45%   +0.40%     
==========================================
  Files          96      100       +4     
  Lines        7246     7865     +619     
==========================================
+ Hits         6307     6878     +571     
- Misses        939      987      +48     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (5)
crates/config/src/workspace_yaml/tests.rs (1)

452-452: ⚡ Quick win

Add failure context to the boolean assertion.

The assert! here will be harder to diagnose on failure without printing actual hosts. Add a message (or nearby eprintln!) with config.git_shallow_hosts.

Based on learnings: In Rust test code, assertions like assert! should log relevant actual/expected context so failures are diagnosable without reruns.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/config/src/workspace_yaml/tests.rs` at line 452, The boolean assertion
on config.git_shallow_hosts lacks failure context; update the test to include
the actual hosts when the assertion fails by replacing the bare assert! with one
that supplies a diagnostic message containing config.git_shallow_hosts (e.g.,
assert!(config.git_shallow_hosts.iter().any(|h| h == "github.com"),
"git_shallow_hosts = {:?}", config.git_shallow_hosts)) or alternatively print
config.git_shallow_hosts with eprintln! immediately before the assertion so
failures show the real value.
crates/git-fetcher/src/prepare_package/tests.rs (1)

37-43: ⚡ Quick win

Improve failure diagnostics for boolean assertions in tests.

Several assert!/matches! checks here have no extra context, which makes failures harder to triage. Add assertion messages (or eprintln! context) for the key actual values.

Based on learnings: In Rust test code, assertions like assert!/assert_ne! should include diagnostic logging/context unless assert_eq! already provides the necessary diff.

Also applies to: 45-52, 55-64, 67-75, 77-84, 87-95, 98-113, 137-143, 146-150

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/git-fetcher/src/prepare_package/tests.rs` around lines 37 - 43, The
boolean assertions in tests.rs (e.g., the test function
package_should_be_built_false_for_empty_scripts and other tests listed) lack
diagnostic context; update each assert!/matches! call to include a failure
message or emit contextual eprintln! output that shows the actual value(s) under
test (for example the returned bool from package_should_be_built, the manifest
JSON, or computed script lists). Locate usages of package_should_be_built and
any other boolean/matches assertions in this file (tests around lines
referenced: 45-52, 55-64, 67-75, 77-84, 87-95, 98-113, 137-143, 146-150) and
change asserts to include formatted messages like "expected X but got {:?}", or
print the relevant variables before asserting so test failures report the
concrete values.
crates/git-fetcher/src/packlist/tests.rs (2)

115-117: ⚡ Quick win

Add diagnostic logging before assertions.

These assert! calls should log the actual out value before the assertion so that test failures can be diagnosed without re-running.

Based on learnings, when using assert!, assert_ne!, etc. (not assert_eq!), ensure the test logs relevant actual/expected values before the assertion.

🔍 Proposed fix
+    eprintln!("packlist output: {out:?}");
     assert!(out.contains(&"lib/index.js".to_string()));
     assert!(out.contains(&"bin/cli".to_string()));
     assert!(out.contains(&"dist/index.js".to_string()));
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/git-fetcher/src/packlist/tests.rs` around lines 115 - 117, Before the
three assertions that check out.contains(...) in the test in
crates/git-fetcher/src/packlist/tests.rs, add diagnostic logging that prints the
value of the variable out (e.g., via eprintln! or debug!) so failing assertions
show the actual contents; update the test around the assertions that reference
"lib/index.js", "bin/cli", and "dist/index.js" to emit out before each assert
(or once before the group) so test failures can be diagnosed without re-running.

136-137: ⚡ Quick win

Add diagnostic logging before assertions.

These assert! calls should log the actual out value so test failures can be diagnosed without re-running.

Based on learnings, assertions that don't automatically show the differing values (like assert! and assert_ne!) should be preceded by logging the relevant context.

🔍 Proposed fix
+    eprintln!("packlist output: {out:?}");
     assert!(out.contains(&"lib/index.js".to_string()));
     assert!(!out.contains(&"lib/sub/inner.js".to_string()));
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/git-fetcher/src/packlist/tests.rs` around lines 136 - 137, Add
diagnostic logging that prints the `out` value immediately before the two
assertions so test failures show the actual content; locate the test in
crates/git-fetcher/src/packlist/tests.rs where `out` is asserted against
"lib/index.js" and "lib/sub/inner.js" and insert a diagnostic print (e.g.,
eprintln! or dbg!) of `out` right before those assert! calls to surface the
failing value during test runs.
crates/git-fetcher/src/fetcher/tests.rs (1)

102-106: ⚡ Quick win

Add diagnostic logging of the full result before assertions.

The test makes multiple assertions on received but doesn't log the full structure first. If one of these assertions fails, it would be helpful to see the entire returned value.

Based on learnings, tests should log diagnostic values around assertions (except assert_eq! which already shows diffs) so failures can be diagnosed without rerunning.

🔍 Proposed fix
+    eprintln!("GitFetcher returned: {received:#?}");
     assert!(!received.built, "package without scripts should not be 'built'");
     assert!(received.cas_paths.contains_key("package.json"));
     assert!(received.cas_paths.contains_key("index.js"));
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/git-fetcher/src/fetcher/tests.rs` around lines 102 - 106, Add
diagnostic logging of the full test result `received` before the assertions so
failures show the returned structure; locate the test in
crates/git-fetcher/src/fetcher/tests.rs where `received` is asserted and insert
a debug/log statement (e.g., using dbg! or formatted debug output) immediately
before the first assertion; ensure the log prints the entire `received`
(borrowed or cloned as needed) but keep the rest of the assertions
(`received.built`, `received.cas_paths`, `cas_path.exists()`) unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/config/src/workspace_yaml/tests.rs`:
- Line 452: The boolean assertion on config.git_shallow_hosts lacks failure
context; update the test to include the actual hosts when the assertion fails by
replacing the bare assert! with one that supplies a diagnostic message
containing config.git_shallow_hosts (e.g.,
assert!(config.git_shallow_hosts.iter().any(|h| h == "github.com"),
"git_shallow_hosts = {:?}", config.git_shallow_hosts)) or alternatively print
config.git_shallow_hosts with eprintln! immediately before the assertion so
failures show the real value.

In `@crates/git-fetcher/src/fetcher/tests.rs`:
- Around line 102-106: Add diagnostic logging of the full test result `received`
before the assertions so failures show the returned structure; locate the test
in crates/git-fetcher/src/fetcher/tests.rs where `received` is asserted and
insert a debug/log statement (e.g., using dbg! or formatted debug output)
immediately before the first assertion; ensure the log prints the entire
`received` (borrowed or cloned as needed) but keep the rest of the assertions
(`received.built`, `received.cas_paths`, `cas_path.exists()`) unchanged.

In `@crates/git-fetcher/src/packlist/tests.rs`:
- Around line 115-117: Before the three assertions that check out.contains(...)
in the test in crates/git-fetcher/src/packlist/tests.rs, add diagnostic logging
that prints the value of the variable out (e.g., via eprintln! or debug!) so
failing assertions show the actual contents; update the test around the
assertions that reference "lib/index.js", "bin/cli", and "dist/index.js" to emit
out before each assert (or once before the group) so test failures can be
diagnosed without re-running.
- Around line 136-137: Add diagnostic logging that prints the `out` value
immediately before the two assertions so test failures show the actual content;
locate the test in crates/git-fetcher/src/packlist/tests.rs where `out` is
asserted against "lib/index.js" and "lib/sub/inner.js" and insert a diagnostic
print (e.g., eprintln! or dbg!) of `out` right before those assert! calls to
surface the failing value during test runs.

In `@crates/git-fetcher/src/prepare_package/tests.rs`:
- Around line 37-43: The boolean assertions in tests.rs (e.g., the test function
package_should_be_built_false_for_empty_scripts and other tests listed) lack
diagnostic context; update each assert!/matches! call to include a failure
message or emit contextual eprintln! output that shows the actual value(s) under
test (for example the returned bool from package_should_be_built, the manifest
JSON, or computed script lists). Locate usages of package_should_be_built and
any other boolean/matches assertions in this file (tests around lines
referenced: 45-52, 55-64, 67-75, 77-84, 87-95, 98-113, 137-143, 146-150) and
change asserts to include formatted messages like "expected X but got {:?}", or
print the relevant variables before asserting so test failures report the
concrete values.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 33d7bf20-181c-4a71-8694-65d43171a0a0

📥 Commits

Reviewing files that changed from the base of the PR and between d6ff1d5 and d1be45a.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (25)
  • Cargo.toml
  • crates/config/src/defaults.rs
  • crates/config/src/lib.rs
  • crates/config/src/workspace_yaml.rs
  • crates/config/src/workspace_yaml/tests.rs
  • crates/executor/src/lib.rs
  • crates/executor/src/lifecycle.rs
  • crates/git-fetcher/Cargo.toml
  • crates/git-fetcher/src/error.rs
  • crates/git-fetcher/src/fetcher.rs
  • crates/git-fetcher/src/fetcher/tests.rs
  • crates/git-fetcher/src/lib.rs
  • crates/git-fetcher/src/packlist.rs
  • crates/git-fetcher/src/packlist/tests.rs
  • crates/git-fetcher/src/preferred_pm.rs
  • crates/git-fetcher/src/preferred_pm/tests.rs
  • crates/git-fetcher/src/prepare_package.rs
  • crates/git-fetcher/src/prepare_package/tests.rs
  • crates/lockfile/src/resolution.rs
  • crates/lockfile/src/resolution/tests.rs
  • crates/package-manager/Cargo.toml
  • crates/package-manager/src/create_virtual_store.rs
  • crates/package-manager/src/install_frozen_lockfile.rs
  • crates/package-manager/src/install_package_by_snapshot.rs
  • crates/package-manager/src/install_package_from_registry/tests.rs
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: Agent
  • GitHub Check: Run benchmark on ubuntu-latest
  • GitHub Check: Code Coverage
  • GitHub Check: Lint and Test (windows-latest)
  • GitHub Check: Run benchmark on ubuntu-latest
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Preserve existing method chains and pipe-trait chains; do not break them into intermediate let bindings unless there is a concrete justification such as a compilation failure, borrow checker rejection, meaningful performance improvement, or other technical necessity. Refactoring for style alone is not sufficient justification.
Choose owned vs. borrowed parameters to minimize copies; prefer borrowed types (&Path over &PathBuf, &str over &String) when it does not force extra copies.
Prefer Arc::clone(&x) and Rc::clone(&x) over x.clone() for reference-counted types to make the cost visible at the call site.
Do not use star imports inside module bodies. Write use super::{Foo, bar} instead of use super::*; for any glob whose target is a module you control. External-crate preludes (e.g., use rayon::prelude::*;) and root-of-module re-exports (e.g., pub use submodule::*; in lib.rs) are exceptions.
Follow Rust API Guidelines for naming, as documented in https://rust-lang.github.io/api-guidelines/naming.html.
Declare a newtype wrapper for any branded string type being ported from TypeScript pnpm. Do not collapse the brand into a plain String or &str; give the type its own struct so misuse is a type error.
When porting branded string types where upstream TypeScript always validates before construction, validate in the Rust port too. Construct the wrapper only via TryFrom<String> and/or FromStr; do not provide an infallible public constructor that takes an arbitrary string.
For branded string types where upstream TypeScript never validates (used purely for type-safety to prevent confusion between string slots), expose an infallible From<String> and From<&str> constructor in the Rust wrapper.
When upstream TypeScript occasionally constructs a branded type without validation (via bare as assertion), add a from_str_unchecked (or similarly named) constructor on the Rust side. Keep the validating constructor as well; `from_str_u...

Files:

  • crates/config/src/defaults.rs
  • crates/executor/src/lib.rs
  • crates/git-fetcher/src/preferred_pm.rs
  • crates/git-fetcher/src/lib.rs
  • crates/config/src/workspace_yaml.rs
  • crates/lockfile/src/resolution.rs
  • crates/config/src/workspace_yaml/tests.rs
  • crates/lockfile/src/resolution/tests.rs
  • crates/git-fetcher/src/prepare_package/tests.rs
  • crates/git-fetcher/src/preferred_pm/tests.rs
  • crates/package-manager/src/create_virtual_store.rs
  • crates/package-manager/src/install_package_from_registry/tests.rs
  • crates/package-manager/src/install_package_by_snapshot.rs
  • crates/git-fetcher/src/packlist/tests.rs
  • crates/executor/src/lifecycle.rs
  • crates/git-fetcher/src/fetcher/tests.rs
  • crates/git-fetcher/src/packlist.rs
  • crates/git-fetcher/src/prepare_package.rs
  • crates/config/src/lib.rs
  • crates/git-fetcher/src/fetcher.rs
  • crates/git-fetcher/src/error.rs
  • crates/package-manager/src/install_frozen_lockfile.rs
🧠 Learnings (2)
📚 Learning: 2026-05-07T23:19:08.272Z
Learnt from: KSXGitHub
Repo: pnpm/pacquet PR: 401
File: tasks/integrated-benchmark/src/work_env.rs:343-344
Timestamp: 2026-05-07T23:19:08.272Z
Learning: When reviewing Rust code in pnpm/pacquet for deprecated API usage, do not automatically treat `serde_saphyr::to_string` as deprecated. In `serde-saphyr` v0.0.25, `serde_saphyr::to_string` has no `#[deprecated]` attribute (the `#[deprecated]` later in `serde-saphyr-0.0.25/src/lib.rs` applies to a different function). Only flag `serde_saphyr::to_string` as deprecated if the resolved dependency version’s source shows `#[deprecated]` on that specific function.

Applied to files:

  • crates/config/src/defaults.rs
  • crates/executor/src/lib.rs
  • crates/git-fetcher/src/preferred_pm.rs
  • crates/git-fetcher/src/lib.rs
  • crates/config/src/workspace_yaml.rs
  • crates/lockfile/src/resolution.rs
  • crates/config/src/workspace_yaml/tests.rs
  • crates/lockfile/src/resolution/tests.rs
  • crates/git-fetcher/src/prepare_package/tests.rs
  • crates/git-fetcher/src/preferred_pm/tests.rs
  • crates/package-manager/src/create_virtual_store.rs
  • crates/package-manager/src/install_package_from_registry/tests.rs
  • crates/package-manager/src/install_package_by_snapshot.rs
  • crates/git-fetcher/src/packlist/tests.rs
  • crates/executor/src/lifecycle.rs
  • crates/git-fetcher/src/fetcher/tests.rs
  • crates/git-fetcher/src/packlist.rs
  • crates/git-fetcher/src/prepare_package.rs
  • crates/config/src/lib.rs
  • crates/git-fetcher/src/fetcher.rs
  • crates/git-fetcher/src/error.rs
  • crates/package-manager/src/install_frozen_lockfile.rs
📚 Learning: 2026-05-01T10:01:33.766Z
Learnt from: zkochan
Repo: pnpm/pacquet PR: 349
File: crates/reporter/src/tests.rs:121-121
Timestamp: 2026-05-01T10:01:33.766Z
Learning: In Rust test code, follow the repo’s CODE_STYLE_GUIDE test-logging rule: add logging (e.g., `eprintln!`/`eprintln!(...)`) so that useful diagnostic values are printed when a test fails, unless the assertion is `assert_eq!` (where the differing values are already included). Concretely, if you use assertions like `assert!`, `assert_ne!`, etc., ensure the test logs the relevant actual/expected values (or context) before/around the assertion so failures can be diagnosed without rerunning.

Applied to files:

  • crates/config/src/workspace_yaml/tests.rs
  • crates/lockfile/src/resolution/tests.rs
  • crates/git-fetcher/src/prepare_package/tests.rs
  • crates/git-fetcher/src/preferred_pm/tests.rs
  • crates/package-manager/src/install_package_from_registry/tests.rs
  • crates/git-fetcher/src/packlist/tests.rs
  • crates/git-fetcher/src/fetcher/tests.rs
🔇 Additional comments (30)
crates/executor/src/lifecycle.rs (1)

198-205: LGTM!

crates/executor/src/lib.rs (1)

7-9: LGTM!

crates/lockfile/src/resolution.rs (1)

48-54: LGTM!

crates/lockfile/src/resolution/tests.rs (1)

251-309: LGTM!

crates/package-manager/Cargo.toml (1)

17-17: LGTM!

Also applies to: 40-40

crates/package-manager/src/install_package_from_registry/tests.rs (1)

52-52: LGTM!

Cargo.toml (1)

29-29: LGTM!

crates/config/src/defaults.rs (1)

11-23: LGTM!

crates/config/src/workspace_yaml.rs (1)

126-130: LGTM!

Also applies to: 221-221

crates/git-fetcher/src/preferred_pm/tests.rs (1)

1-60: LGTM!

crates/package-manager/src/install_frozen_lockfile.rs (1)

100-106: LGTM!

Also applies to: 128-128

crates/git-fetcher/Cargo.toml (1)

1-34: LGTM!

crates/package-manager/src/create_virtual_store.rs (1)

82-87: LGTM!

Also applies to: 125-125, 445-445, 522-534

crates/config/src/lib.rs (1)

16-17: LGTM!

Also applies to: 387-397

crates/git-fetcher/src/preferred_pm.rs (1)

1-66: LGTM!

crates/git-fetcher/src/lib.rs (1)

1-24: LGTM!

crates/package-manager/src/install_package_by_snapshot.rs (2)

107-179: LGTM!


202-232: LGTM!

crates/git-fetcher/src/packlist.rs (4)

66-75: LGTM!


92-114: LGTM!


116-138: LGTM!


182-251: LGTM!

crates/git-fetcher/src/error.rs (1)

1-105: LGTM!

crates/git-fetcher/src/fetcher.rs (6)

78-80: LGTM!


86-102: LGTM!


104-151: LGTM!


174-201: LGTM!


207-262: LGTM!


266-310: LGTM!

crates/git-fetcher/src/prepare_package.rs (1)

201-217: 🏗️ Heavy lift

The semantic differences here are not as described.

The Rust implementation uses canonicalize() + starts_with() + is_dir() check, which is the recommended secure pattern for symlink-safe path validation. Using path.relative() alone (which the review suggests pnpm uses) is explicitly unsuitable for security purposes and would not properly handle symlink traversal attacks. If pnpm implements secure path validation, it uses fs.realpathSync() (equivalent to Rust's canonicalize), making the approaches equivalent for frozen-lockfile paths. Paths in lockfiles are normalized absolute paths, not symlink-dependent, so both validation methods produce identical results in practice.

			> Likely an incorrect or invalid review comment.

@github-actions

github-actions Bot commented May 13, 2026

Copy link
Copy Markdown

Integrated-Benchmark Report (Linux)

Scenario: Frozen Lockfile

Command Mean [s] Min [s] Max [s] Relative
pacquet@HEAD 2.000 ± 0.059 1.874 2.076 1.00
pacquet@main 2.036 ± 0.052 1.968 2.104 1.02 ± 0.04
pnpm 5.233 ± 0.035 5.189 5.306 2.62 ± 0.08
BENCHMARK_REPORT.json
{
  "results": [
    {
      "command": "pacquet@HEAD",
      "mean": 1.9998165459,
      "stddev": 0.05949518849178068,
      "median": 2.0079495865,
      "user": 2.6470526,
      "system": 1.97934992,
      "min": 1.873583213,
      "max": 2.076179697,
      "times": [
        2.074440664,
        2.01530629,
        2.030450853,
        1.9547701700000002,
        2.013725083,
        1.9680562410000002,
        1.989479158,
        2.00217409,
        1.873583213,
        2.076179697
      ]
    },
    {
      "command": "pacquet@main",
      "mean": 2.0364184907,
      "stddev": 0.05154882326796174,
      "median": 2.0371092255,
      "user": 2.6320842000000004,
      "system": 2.00229052,
      "min": 1.967672216,
      "max": 2.103556584,
      "times": [
        2.103556584,
        2.097393769,
        2.010021038,
        1.9792927260000002,
        1.967672216,
        1.987673762,
        2.006004958,
        2.067929592,
        2.080442849,
        2.064197413
      ]
    },
    {
      "command": "pnpm",
      "mean": 5.232911597399999,
      "stddev": 0.03486116673552246,
      "median": 5.233937748499999,
      "user": 8.371095299999999,
      "system": 2.58725322,
      "min": 5.188950392,
      "max": 5.306007235,
      "times": [
        5.306007235,
        5.20005739,
        5.196138402,
        5.218254046999999,
        5.256697475999999,
        5.231880147999999,
        5.188950392,
        5.245366632,
        5.249768903,
        5.2359953489999995
      ]
    }
  ]
}

Scenario: Frozen Lockfile (Hot Cache)

Command Mean [ms] Min [ms] Max [ms] Relative
pacquet@HEAD 482.6 ± 21.3 464.3 537.7 1.00
pacquet@main 556.3 ± 33.6 511.6 637.0 1.15 ± 0.09
pnpm 2151.3 ± 210.7 2006.5 2700.7 4.46 ± 0.48
BENCHMARK_REPORT.json
{
  "results": [
    {
      "command": "pacquet@HEAD",
      "mean": 0.48262932585999996,
      "stddev": 0.021345327124236434,
      "median": 0.47901604336000003,
      "user": 0.32932024,
      "system": 0.8636183599999999,
      "min": 0.46432251486000004,
      "max": 0.53773570386,
      "times": [
        0.53773570386,
        0.47215418786,
        0.49397690386000004,
        0.46803284886,
        0.46714235086000006,
        0.47841777186000006,
        0.47961431486000006,
        0.48473548086000007,
        0.46432251486000004,
        0.48016118086000004
      ]
    },
    {
      "command": "pacquet@main",
      "mean": 0.55632960626,
      "stddev": 0.033636682739527274,
      "median": 0.5481065763599999,
      "user": 0.33953493999999995,
      "system": 0.8649762599999999,
      "min": 0.51162007586,
      "max": 0.63701402386,
      "times": [
        0.63701402386,
        0.54458194086,
        0.53734497786,
        0.51162007586,
        0.53899851986,
        0.55438209886,
        0.54055670486,
        0.57198617786,
        0.57518033086,
        0.55163121186
      ]
    },
    {
      "command": "pnpm",
      "mean": 2.1512694427599994,
      "stddev": 0.21074501281194796,
      "median": 2.0567855378599997,
      "user": 2.7714596400000002,
      "system": 1.26036306,
      "min": 2.0064708688599997,
      "max": 2.70072290286,
      "times": [
        2.11970585586,
        2.05480314686,
        2.29592917486,
        2.15550724486,
        2.05876792886,
        2.04777119786,
        2.70072290286,
        2.03616242286,
        2.03685368386,
        2.0064708688599997
      ]
    }
  ]
}

zkochan added a commit that referenced this pull request May 13, 2026
… `/`, disambiguate doc links (#436 §B+D review)

Addresses the review comments and CI doc failure on #446:

- `fetcher.rs`: `extra_env: &HashMap::new()` relied on Rust's temporary-
  lifetime extension to keep the borrow alive across `prepare_package`.
  Bind the empty map to a local instead so the borrow has the same
  scope as `prepare_opts` and isn't sensitive to future expression
  reshape in the surrounding block.
- `packlist.rs`: the `?` glob path matched any byte without rejecting
  `/`, so `a?b` would incorrectly match `a/b`. Tighten the condition
  to `(pc == '?' && candidate[c] != '/') || pc == candidate[c]` and
  add a regression test that pins `a?b/index.js` doesn't match `a/b/
  index.js`.
- Doc links: `crate::prepare_package` and `crate::packlist` are
  ambiguous (each is both a function and the file-as-module that
  defines it), so `cargo doc -D warnings` failed CI. Suffix with `()`
  on every link site so rustdoc resolves them to the function.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/git-fetcher/src/error.rs (1)

7-7: ⚡ Quick win

Use pnpm/blob/main links instead of pinned SHA links in porting docs.

Please update these upstream references to https://github.com/pnpm/pnpm/blob/main/... so they always track pnpm’s current source of truth.

Suggested doc-link update
-/// [`exec/prepare-package`](https://github.com/pnpm/pnpm/blob/94240bc046/exec/prepare-package/src/index.ts).
+/// [`exec/prepare-package`](https://github.com/pnpm/pnpm/blob/main/exec/prepare-package/src/index.ts).

-    /// Mirrors [`exec/prepare-package/src/index.ts:37-46`](https://github.com/pnpm/pnpm/blob/94240bc046/exec/prepare-package/src/index.ts#L37-L46).
+    /// Mirrors [`exec/prepare-package/src/index.ts:37-46`](https://github.com/pnpm/pnpm/blob/main/exec/prepare-package/src/index.ts#L37-L46).

-    /// [`exec/prepare-package/src/index.ts:71-77`](https://github.com/pnpm/pnpm/blob/94240bc046/exec/prepare-package/src/index.ts#L71-L77).
+    /// [`exec/prepare-package/src/index.ts:71-77`](https://github.com/pnpm/pnpm/blob/main/exec/prepare-package/src/index.ts#L71-L77).

-    /// at [`exec/prepare-package/src/index.ts:92-103`](https://github.com/pnpm/pnpm/blob/94240bc046/exec/prepare-package/src/index.ts#L92-L103).
+    /// at [`exec/prepare-package/src/index.ts:92-103`](https://github.com/pnpm/pnpm/blob/main/exec/prepare-package/src/index.ts#L92-L103).

-    /// [`fetching/git-fetcher/src/index.ts:39-41`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/git-fetcher/src/index.ts#L39-L41).
+    /// [`fetching/git-fetcher/src/index.ts:39-41`](https://github.com/pnpm/pnpm/blob/main/fetching/git-fetcher/src/index.ts#L39-L41).

As per coding guidelines: “When reading on GitHub, open files from https://github.com/pnpm/pnpm/blob/main/... rather than from a permalinked SHA.”

Also applies to: 15-15, 29-29, 39-39, 87-87

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/git-fetcher/src/error.rs` at line 7, Replace the pinned SHA GitHub
links in the crate's doc comments with canonical `blob/main` references so they
track pnpm’s current source; specifically update the doc-comment that references
`exec/prepare-package` (and the other occurrences of pinned pnpm links in this
file) to use `https://github.com/pnpm/pnpm/blob/main/...` instead of the
permalinked SHA. Locate the doc comments in crates/git-fetcher/src/error.rs that
contain the pnpm links (e.g., the comment mentioning `exec/prepare-package`) and
swap the `/blob/<SHA>/` portion for `/blob/main/` for all occurrences in this
file.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/git-fetcher/src/error.rs`:
- Line 7: Replace the pinned SHA GitHub links in the crate's doc comments with
canonical `blob/main` references so they track pnpm’s current source;
specifically update the doc-comment that references `exec/prepare-package` (and
the other occurrences of pinned pnpm links in this file) to use
`https://github.com/pnpm/pnpm/blob/main/...` instead of the permalinked SHA.
Locate the doc comments in crates/git-fetcher/src/error.rs that contain the pnpm
links (e.g., the comment mentioning `exec/prepare-package`) and swap the
`/blob/<SHA>/` portion for `/blob/main/` for all occurrences in this file.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 424e9e3d-82da-4cbb-aeaa-b8b576791414

📥 Commits

Reviewing files that changed from the base of the PR and between d1be45a and 2823b69.

📒 Files selected for processing (5)
  • crates/git-fetcher/src/error.rs
  • crates/git-fetcher/src/fetcher.rs
  • crates/git-fetcher/src/packlist.rs
  • crates/git-fetcher/src/packlist/tests.rs
  • crates/git-fetcher/src/preferred_pm.rs
🚧 Files skipped from review as they are similar to previous changes (4)
  • crates/git-fetcher/src/preferred_pm.rs
  • crates/git-fetcher/src/packlist/tests.rs
  • crates/git-fetcher/src/fetcher.rs
  • crates/git-fetcher/src/packlist.rs
📜 Review details
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Preserve existing method chains and pipe-trait chains; do not break them into intermediate let bindings unless there is a concrete justification such as a compilation failure, borrow checker rejection, meaningful performance improvement, or other technical necessity. Refactoring for style alone is not sufficient justification.
Choose owned vs. borrowed parameters to minimize copies; prefer borrowed types (&Path over &PathBuf, &str over &String) when it does not force extra copies.
Prefer Arc::clone(&x) and Rc::clone(&x) over x.clone() for reference-counted types to make the cost visible at the call site.
Do not use star imports inside module bodies. Write use super::{Foo, bar} instead of use super::*; for any glob whose target is a module you control. External-crate preludes (e.g., use rayon::prelude::*;) and root-of-module re-exports (e.g., pub use submodule::*; in lib.rs) are exceptions.
Follow Rust API Guidelines for naming, as documented in https://rust-lang.github.io/api-guidelines/naming.html.
Declare a newtype wrapper for any branded string type being ported from TypeScript pnpm. Do not collapse the brand into a plain String or &str; give the type its own struct so misuse is a type error.
When porting branded string types where upstream TypeScript always validates before construction, validate in the Rust port too. Construct the wrapper only via TryFrom<String> and/or FromStr; do not provide an infallible public constructor that takes an arbitrary string.
For branded string types where upstream TypeScript never validates (used purely for type-safety to prevent confusion between string slots), expose an infallible From<String> and From<&str> constructor in the Rust wrapper.
When upstream TypeScript occasionally constructs a branded type without validation (via bare as assertion), add a from_str_unchecked (or similarly named) constructor on the Rust side. Keep the validating constructor as well; `from_str_u...

Files:

  • crates/git-fetcher/src/error.rs
🧠 Learnings (1)
📚 Learning: 2026-05-07T23:19:08.272Z
Learnt from: KSXGitHub
Repo: pnpm/pacquet PR: 401
File: tasks/integrated-benchmark/src/work_env.rs:343-344
Timestamp: 2026-05-07T23:19:08.272Z
Learning: When reviewing Rust code in pnpm/pacquet for deprecated API usage, do not automatically treat `serde_saphyr::to_string` as deprecated. In `serde-saphyr` v0.0.25, `serde_saphyr::to_string` has no `#[deprecated]` attribute (the `#[deprecated]` later in `serde-saphyr-0.0.25/src/lib.rs` applies to a different function). Only flag `serde_saphyr::to_string` as deprecated if the resolved dependency version’s source shows `#[deprecated]` on that specific function.

Applied to files:

  • crates/git-fetcher/src/error.rs

zkochan added 3 commits May 13, 2026 13:47
…ePackage (#436 §B+D)

Builds on §A (#440). Adds a new `pacquet-git-fetcher` crate that
handles `LockfileResolution::Git` snapshots during frozen-lockfile
installs, plus the supporting `preparePackage` port. The install
dispatcher now routes git resolutions through it instead of returning
`UnsupportedResolution { resolution_kind: "git" }`.

New crate `pacquet-git-fetcher`:

- `GitFetcher` shells out to the system `git` (clone, or `init` +
  `fetch --depth 1` on hosts in `git_shallow_hosts`), checks out the
  pinned commit, verifies it via `rev-parse HEAD`, runs
  `preparePackage`, removes `.git`, computes a packlist, and imports
  the resulting file set into the CAS. Surfaces a friendly
  `GitNotFound` when `git` isn't on `PATH` — pacquet does not bundle
  it. Mirrors `fetching/git-fetcher/src/index.ts`.

- `prepare_package` ports `exec/prepare-package/src/index.ts`. Reads
  the manifest, decides via `package_should_be_built`, honors an
  `AllowBuildFn` (the dispatcher adapts pacquet's `AllowBuildPolicy`
  into this shape), runs `<pm>-install` plus any `prepublish` /
  `prepack` / `publish` scripts via the existing
  `pacquet_executor::run_lifecycle_hook`, then deletes `node_modules`.
  Emits the upstream error codes `GIT_DEP_PREPARE_NOT_ALLOWED`,
  `ERR_PNPM_PREPARE_PACKAGE`, and `INVALID_PATH`.

- `detect_preferred_pm` sniffs the cloned tree for a lockfile to pick
  the install pm (`pnpm-lock.yaml` → pnpm, `yarn.lock` → yarn, etc.),
  defaulting to npm. Workspace-root walking is deferred — git-hosted
  snapshots almost always ship a lockfile at the repo root.

- A minimal `packlist` honors the manifest's `files` field with a
  tiny `*` / `**` / `?` glob matcher, always-includes the README /
  LICENSE / package.json set, and always-excludes `.git`,
  `node_modules`, lockfiles for sibling pms, and common cruft.
  Full `.npmignore` / `.gitignore` semantics and
  `bundleDependencies` walking are deferred and tracked in module
  docs; `bundleDependencies` emits a `warn!` so the gap is visible.

Wiring in `pacquet-package-manager`:

- `InstallPackageBySnapshot` now matches `LockfileResolution::Git`
  and runs the fetcher in place of `DownloadTarballToStore`. The
  helper `tarball_url_and_integrity` factors out the per-resolution
  branch so each variant builds its own `cas_paths`.
- Threads `&AllowBuildPolicy` from `InstallFrozenLockfile` → through
  `CreateVirtualStore` → into `InstallPackageBySnapshot`, computing
  the policy once per install instead of per snapshot.
- `snapshot_cache_key` returns `Ok(None)` for `Git` resolutions so
  they go cold-batch in this PR. Warm prefetch for git-hosted slots
  is a follow-up alongside §C.

Supporting changes:

- `GitResolution.path: Option<String>` (§A back-fill) so a lockfile
  pinning a sub-directory of a git-hosted monorepo deserializes
  without tripping `deny_unknown_fields`.
- `Config::git_shallow_hosts` with pnpm's default list
  (`github.com` / `gist.github.com` / `gitlab.com` / `bitbucket.com` /
  `bitbucket.org`) and `pnpm-workspace.yaml` override support.
- `pacquet_executor::run_lifecycle_hook` is now `pub` so the
  preparePackage port can dispatch by arbitrary stage name without
  going through `run_postinstall_hooks`'s pre/install/post triple.

Out of scope (follow-ups for #436):

- §C — git-hosted *tarball* fetcher (the `gitHosted: true` shape).
- §E — full integration-test matrix (`plans/TEST_PORTING.md`
  563-572). This PR ships the seven crate-level tests against a
  local bare repo that prove the path works end-to-end.
- Warm prefetch for git resolutions. The fetcher writes nothing to
  `index.db` today, so a second install re-clones — slow but
  correct.
- Full `npm-packlist` semantics (`.npmignore`, gitignore layering,
  `bundleDependencies` walking).
… `/`, disambiguate doc links (#436 §B+D review)

Addresses the review comments and CI doc failure on #446:

- `fetcher.rs`: `extra_env: &HashMap::new()` relied on Rust's temporary-
  lifetime extension to keep the borrow alive across `prepare_package`.
  Bind the empty map to a local instead so the borrow has the same
  scope as `prepare_opts` and isn't sensitive to future expression
  reshape in the surrounding block.
- `packlist.rs`: the `?` glob path matched any byte without rejecting
  `/`, so `a?b` would incorrectly match `a/b`. Tighten the condition
  to `(pc == '?' && candidate[c] != '/') || pc == candidate[c]` and
  add a regression test that pins `a?b/index.js` doesn't match `a/b/
  index.js`.
- Doc links: `crate::prepare_package` and `crate::packlist` are
  ambiguous (each is both a function and the file-as-module that
  defines it), so `cargo doc -D warnings` failed CI. Suffix with `()`
  on every link site so rustdoc resolves them to the function.
 §B+D dylint)

The perfectionist lint forbids trailing commas on single-line macro
invocations; `just dylint` runs a different toolchain locally and
didn't catch it. CI failure on 2823b69.
Copilot AI review requested due to automatic review settings May 13, 2026 11:49
@zkochan zkochan force-pushed the feat/436-git-fetcher branch from 2823b69 to d38f52b Compare May 13, 2026 11:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated 4 comments.

Comment thread crates/package-manager/src/install_package_by_snapshot.rs Outdated
Comment thread crates/lockfile/src/resolution/tests.rs
Comment thread crates/git-fetcher/Cargo.toml
Comment thread crates/git-fetcher/src/prepare_package/tests.rs
…used deps, remove dbg! (#436 §B+D review round 2)

Addresses the second batch of CodeRabbit review comments on #446:

- `install_package_by_snapshot.rs`: stop borrowing a closure temporary
  across the `.await` on `GitFetcher::run`. Switching the
  `let x: &dyn Fn(...) = &|...|` form to a plain `let x = |...|`
  binding gives the closure an explicit named storage that doesn't
  rely on temporary-lifetime extension reaching across the await
  suspension. The struct field now takes `&allow_build_closure`.
- `git-fetcher/Cargo.toml`: drop `pacquet-config`, `pacquet-fs`,
  and `pacquet-lockfile` — the crate's source has no references
  to any of them. They were inherited from the initial scaffold
  and were already dead.
- `prepare_package/tests.rs`: replace `Box::leak(Box::new(HashMap::
  new()))` with a process-wide `OnceLock<HashMap<_, _>>` so every
  test call shares one allocation instead of leaking a fresh one.
- `lockfile/resolution/tests.rs`: drop the leftover `dbg!(&received)`
  in `deserialize_git_resolution_with_path` — the `assert_eq!`
  immediately below already pins the value, so the print was just
  noise during `cargo test` runs.
@zkochan zkochan merged commit 1fa70a4 into main May 13, 2026
15 of 16 checks passed
@zkochan zkochan deleted the feat/436-git-fetcher branch May 13, 2026 12:16
zkochan added a commit that referenced this pull request May 13, 2026
…acklist (#436 §C) (#451)

## Summary

Builds on §B+D (#446). Adds the second half of the git-hosted install path: `TarballResolution { gitHosted: true }` lockfile entries — the ones whose tarballs land at `codeload.github.com` / `gitlab.com` / `bitbucket.org` archive endpoints — now run through a new `GitHostedTarballFetcher` after the existing `DownloadTarballToStore` settles their bytes into CAS.

Mirrors pnpm's [`gitHostedTarballFetcher.ts`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts):

1. Materialize the CAS-resident files into a writable temp dir (`fs::copy` per file — hardlinking here would let prepare scripts mutate shared CAS entries).
2. Run the existing `prepare_package` port that §B+D shipped — same `<pm>-install` + `prepublish` / `prepack` / `publish` dispatch, same `GIT_DEP_PREPARE_NOT_ALLOWED` gate.
3. Run the existing minimal `packlist` over the prepared tree to drop source maps, test fixtures, etc. that ship in the raw archive but not in the published file set.
4. Re-import the filtered file set back into CAS and hand the `HashMap<String, PathBuf>` to `CreateVirtualDirBySnapshot`.

### Shared `cas_io` module

The CAS-import helpers (`import_into_cas`, `is_file_executable`, `map_write_cas`) plus a new `materialize_into` lift out of `fetcher.rs` into a shared `cas_io` module so both fetchers consume the same code path. No public-API change — helpers stay `pub(crate)`.

`cas_io` also gains a `join_checked(root, rel)` helper that rejects absolute paths, `..` components, root, and drive-prefix components before joining, so a malicious tarball entry can't escape `target_dir` / `pkg_dir`. Errors surface as `GitFetcherError::Io(InvalidInput)`. Both `materialize_into` and `import_into_cas` route through it, taking `rel` directly (forward slashes are valid on every platform — `Path::components()` recognises both `/` and `\` on Windows — so no per-file `String` allocation is needed).

### Supporting changes

- `TarballResolution.path: Option<String>` mirrors pnpm's `TarballResolution.path` so a lockfile pinning a git-hosted tarball from a monorepo (`path: packages/sub`) deserializes without tripping `deny_unknown_fields`. The fetcher threads this through to `prepare_package`'s `safe_join_path` so prepare and packlist run inside the right sub-directory.
- The install dispatcher hoists the `allow_build` closure and the `ScriptsPrependNodePath` conversion above the resolution match so both the `Git` arm and the new `gitHosted: true` post-pass borrow from the same named locals — no temp closures across `.await`.
- `InstallPackageBySnapshotError::GitFetch`'s doc now spells out that the variant covers failures from both fetchers (the git-CLI path and the git-hosted-tarball post-pass), since both share the same `GitFetcherError` taxonomy.

## Out of scope (follow-ups for #436)

- Warm prefetch for git-hosted slots — neither fetcher writes `gitHostedStoreIndexKey` rows to `index.db` yet, so a re-install re-runs clone + prepare + packlist. Correct, but slow on the second run.
- Two-slot CAS layout (`${filesIndexFile}\traw` + final). Upstream optimizes "raw == prepared" by promoting the raw entry in place. Pacquet always re-imports today; hash-dedup means duplicate work but not duplicate storage.
- §E — full integration-test matrix in `plans/TEST_PORTING.md` 563-572.
- Real `npm-packlist` semantics (`.npmignore`, `.gitignore` layering, `bundleDependencies` walking).
KSXGitHub pushed a commit that referenced this pull request May 13, 2026
Pull in 28 commits from upstream main, including the
`pacquet-npmrc` → `pacquet-config` rename (#420) plus features:
- supportedArchitectures + --cpu/--os/--libc (#456)
- frozen-lockfile (#442, #443, #447, #450)
- git-fetcher (#436 / #446, #451, #454)
- side-effects cache (#421 / #422, #423, #424)
- real-hoist + global-virtual-store (#432 / #438 / #444, #449, #452)
- patchedDependencies + allow-builds (#425, #427, #428)
- engine/platform installability (#434 / #439)

Conflicts resolved:
- `crates/npmrc/` files migrated under the renamed
  `crates/config/` directory; `Npmrc` → `Config` everywhere
  except `NpmrcAuth` (which keeps the `.npmrc`-domain name).
- `Config::current` reads the env-var DI generic `Api: EnvVar`
  for ${VAR}-substitution in `.npmrc`. Production turbofish in
  `cli_args.rs` is `Config::current::<RealApi, _, _, _, _>(...)`.
- Two-phase `NpmrcAuth::apply_*` retained so default-registry
  creds key at the yaml-resolved registry URL.
- New `Config::auth_headers` field plumbed through
  `install_package_by_snapshot`'s `DownloadTarballToStore`.
- Tests under `crates/config/src/workspace_yaml/tests.rs`
  pick up the new ParseYaml unit test added on this branch.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants