test(git-fetcher): port §E git-fetcher and tarball-fetcher tests from upstream (#436 §E)#462
Conversation
… upstream (#436 §E) Ports five new tests against the local-bare-repo and write-to-CAS fixtures already established by §B+D and §C, plus the TEST_PORTING.md checkbox updates for the test-matrix subset that ship today (15 items checked off; the rest are noted with concrete reasons — either Stage 2 resolver work, real-npm requirements, perf benchmarks, or follow-up tests that need a PATH-shim helper). New tests (`crates/git-fetcher/src/fetcher/tests.rs`): - `fetcher_packs_subfolder_when_path_set` — ports `fetching/git- fetcher/test/index.ts:69`. Builds a tiny monorepo bare repo, runs `GitFetcher` with `path: Some("packages/sub")`, asserts the returned `cas_paths` only contain files relative to the sub-dir (never sibling packages, never carrying the `packages/` prefix). - `fetcher_handles_repo_without_package_json` — ports `fetching/git-fetcher/test/index.ts:150`. A repo with no `package.json` at root still imports cleanly: `prepare_package` returns `should_be_built: false` (no manifest means nothing to build), and the fetcher hands the install dispatcher the files the packlist found. - `fetcher_skips_build_when_ignore_scripts` — ports `fetching/git-fetcher/test/index.ts:247`. A repo whose `scripts.prepare` would fail if it ran (`exit 1`) still produces a successful fetcher run when `ignore_scripts: true`; observing success proves the lifecycle runner never spawned. Pins the `should_be_built: true` short-circuit so callers know the package wanted a build. New tests (`crates/git-fetcher/src/tarball_fetcher/tests.rs`): - `tarball_path_traversal_attack_is_rejected` — ports `fetching/tarball-fetcher/test/fetch.ts:610`. `path: Some("../escape")` must surface as `INVALID_PATH` from `prepare_package`'s `safe_join_path` before any extraction. - `tarball_path_to_missing_subdir_is_rejected` — ports `fetching/tarball-fetcher/test/fetch.ts:637`. `path: Some("does/not/exist")` for a tarball that doesn't contain the sub-dir must surface as `INVALID_PATH` rather than silently packing the root. `plans/TEST_PORTING.md` updates: marked 15 §E items as `[x]` with links to the covering pacquet test name. Deferrals carry concrete reasons (Stage 2 resolver, real-npm dependency, perf benchmark, or PATH-shim helper). Existing test-suite total goes from 765 → 842 (local `just ready`).
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📜 Recent review details⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
📝 WalkthroughWalkthroughAdds Rust test fixtures and five integration tests covering GitFetcher subfolder scoping, missing-root-manifest handling, script-ignore behavior, and GitHostedTarballFetcher path validation; updates TEST_PORTING.md to mark related git-hosted cases as ported. ChangesGit Fetcher and Tarball Path Tests
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plans/TEST_PORTING.md`:
- Line 569: The user-facing note in the checklist line containing "`TypeScript
repo: installing/deps-installer/test/install/fromRepo.ts:206` `from a github
repo that has no package.json file`" uses a lowercase "github"; update that
phrase to "from a GitHub repo that has no package.json file" so the platform
name is capitalized consistently (edit the string in TEST_PORTING.md near the
quoted snippet).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 1a35c81c-33b5-4a3b-9a4f-6094e2f0f4b5
📒 Files selected for processing (3)
crates/git-fetcher/src/fetcher/tests.rscrates/git-fetcher/src/tarball_fetcher/tests.rsplans/TEST_PORTING.md
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Agent
- GitHub Check: Dylint
- GitHub Check: Lint and Test (macos-latest)
- GitHub Check: Lint and Test (windows-latest)
- GitHub Check: Lint and Test (ubuntu-latest)
- GitHub Check: Code Coverage
- GitHub Check: Run benchmark on ubuntu-latest
- GitHub Check: Run benchmark on ubuntu-latest
🧰 Additional context used
📓 Path-based instructions (2)
**/*.rs
📄 CodeRabbit inference engine (AGENTS.md)
**/*.rs: Preserve existing method chains andpipe-traitchains; do not break them into intermediateletbindings unless there is a concrete justification such as a compilation failure, borrow checker rejection, meaningful performance improvement, or other technical necessity. Refactoring for style alone is not sufficient justification.
Choose owned vs. borrowed parameters to minimize copies; prefer borrowed types (&Pathover&PathBuf,&strover&String) when it does not force extra copies.
PreferArc::clone(&x)andRc::clone(&x)overx.clone()for reference-counted types to make the cost visible at the call site.
Do not use star imports inside module bodies. Writeuse super::{Foo, bar}instead ofuse super::*;for any glob whose target is a module you control. External-crate preludes (e.g.,use rayon::prelude::*;) and root-of-module re-exports (e.g.,pub use submodule::*;inlib.rs) are exceptions.
Follow Rust API Guidelines for naming, as documented in https://rust-lang.github.io/api-guidelines/naming.html.
Declare a newtype wrapper for any branded string type being ported from TypeScript pnpm. Do not collapse the brand into a plainStringor&str; give the type its own struct so misuse is a type error.
When porting branded string types where upstream TypeScript always validates before construction, validate in the Rust port too. Construct the wrapper only viaTryFrom<String>and/orFromStr; do not provide an infallible public constructor that takes an arbitrary string.
For branded string types where upstream TypeScript never validates (used purely for type-safety to prevent confusion between string slots), expose an infallibleFrom<String>andFrom<&str>constructor in the Rust wrapper.
When upstream TypeScript occasionally constructs a branded type without validation (via bareasassertion), add afrom_str_unchecked(or similarly named) constructor on the Rust side. Keep the validating constructor as well; `from_str_u...
Files:
crates/git-fetcher/src/tarball_fetcher/tests.rscrates/git-fetcher/src/fetcher/tests.rs
**/*.md
📄 CodeRabbit inference engine (AGENTS.md)
When citing upstream pnpm code anywhere (code comments, doc comments, Markdown docs, PR descriptions, or commit messages), link to a specific commit SHA, not a branch name. Use the first 10 hex characters of the SHA. Branch links like
github.com/<owner>/<repo>/blob/main/...are impermanent; permanent links pin the commit so the reference stays meaningful long after upstream changes. This rule applies to references to any GitHub repository, not only pnpm/pnpm.
Files:
plans/TEST_PORTING.md
🧠 Learnings (2)
📚 Learning: 2026-05-01T10:01:33.766Z
Learnt from: zkochan
Repo: pnpm/pacquet PR: 349
File: crates/reporter/src/tests.rs:121-121
Timestamp: 2026-05-01T10:01:33.766Z
Learning: In Rust test code, follow the repo’s CODE_STYLE_GUIDE test-logging rule: add logging (e.g., `eprintln!`/`eprintln!(...)`) so that useful diagnostic values are printed when a test fails, unless the assertion is `assert_eq!` (where the differing values are already included). Concretely, if you use assertions like `assert!`, `assert_ne!`, etc., ensure the test logs the relevant actual/expected values (or context) before/around the assertion so failures can be diagnosed without rerunning.
Applied to files:
crates/git-fetcher/src/tarball_fetcher/tests.rscrates/git-fetcher/src/fetcher/tests.rs
📚 Learning: 2026-05-07T23:19:08.272Z
Learnt from: KSXGitHub
Repo: pnpm/pacquet PR: 401
File: tasks/integrated-benchmark/src/work_env.rs:343-344
Timestamp: 2026-05-07T23:19:08.272Z
Learning: When reviewing Rust code in pnpm/pacquet for deprecated API usage, do not automatically treat `serde_saphyr::to_string` as deprecated. In `serde-saphyr` v0.0.25, `serde_saphyr::to_string` has no `#[deprecated]` attribute (the `#[deprecated]` later in `serde-saphyr-0.0.25/src/lib.rs` applies to a different function). Only flag `serde_saphyr::to_string` as deprecated if the resolved dependency version’s source shows `#[deprecated]` on that specific function.
Applied to files:
crates/git-fetcher/src/tarball_fetcher/tests.rscrates/git-fetcher/src/fetcher/tests.rs
🪛 LanguageTool
plans/TEST_PORTING.md
[uncategorized] ~569-~569: The official name of this software platform is spelled with a capital “H”.
Context: ...installer/test/install/fromRepo.ts:206 from a github repo that has no package.json file` — c...
(GITHUB)
🔇 Additional comments (3)
crates/git-fetcher/src/tarball_fetcher/tests.rs (1)
353-436: LGTM!crates/git-fetcher/src/fetcher/tests.rs (1)
224-431: LGTM!plans/TEST_PORTING.md (1)
573-609: LGTM!
There was a problem hiding this comment.
Pull request overview
Ports upstream pnpm §E git-fetcher and tarball-fetcher tests into pacquet to validate git-hosted fetch behavior (subdirectory packing, missing manifests, ignore-scripts behavior, and tarball path validation) and updates the test-porting plan to reflect new coverage.
Changes:
- Added new
GitFetcherfixture helpers and three new async tests covering subdir packing, missingpackage.json, andignore_scriptsbehavior. - Added two new
GitHostedTarballFetcherasync tests covering path traversal rejection and missing-subdir rejection. - Updated
plans/TEST_PORTING.mdto mark §E items as covered and document deferrals with reasons.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| plans/TEST_PORTING.md | Marks §E upstream test items as covered by the new Rust tests and records explicit deferral reasons. |
| crates/git-fetcher/src/tarball_fetcher/tests.rs | Adds tarball path validation tests (traversal + missing subdir) ensuring INVALID_PATH is surfaced pre-extraction. |
| crates/git-fetcher/src/fetcher/tests.rs | Adds git fetcher tests and repo-fixture builders for monorepo subdir packing, missing manifest tolerance, and ignore-scripts behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Micro-Benchmark ResultsLinux |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #462 +/- ##
==========================================
- Coverage 87.35% 87.34% -0.02%
==========================================
Files 113 114 +1
Lines 9356 9435 +79
==========================================
+ Hits 8173 8241 +68
- Misses 1183 1194 +11 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Integrated-Benchmark Report (Linux)Scenario: Frozen Lockfile
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 2.51845203078,
"stddev": 0.07454537420560206,
"median": 2.50115255458,
"user": 2.6005063799999997,
"system": 3.46303268,
"min": 2.4319759735799997,
"max": 2.68031382358,
"times": [
2.68031382358,
2.47348025458,
2.49895138858,
2.50335372058,
2.4319759735799997,
2.54002113858,
2.47798146458,
2.57289245758,
2.43895230758,
2.56659777858
]
},
{
"command": "pacquet@main",
"mean": 2.47575367078,
"stddev": 0.03723698554837816,
"median": 2.4719696455799998,
"user": 2.60051828,
"system": 3.4427464800000003,
"min": 2.43618771158,
"max": 2.5364361825799997,
"times": [
2.5364361825799997,
2.43618771158,
2.48894995058,
2.50159809758,
2.44059292058,
2.44506315858,
2.43958027258,
2.48663639458,
2.45730289658,
2.52518912258
]
},
{
"command": "pnpm",
"mean": 5.898621100880001,
"stddev": 0.09475936499921245,
"median": 5.877097020080001,
"user": 8.56891578,
"system": 4.32472858,
"min": 5.789179215580001,
"max": 6.08916231558,
"times": [
5.939778155580001,
5.86887382058,
5.8853202195800005,
6.08916231558,
5.81908562958,
5.810634284580001,
5.960898734580001,
5.835844149580001,
5.789179215580001,
5.98743448358
]
}
]
}Scenario: Frozen Lockfile (Hot Cache)
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 0.7039608471000001,
"stddev": 0.03164975046142958,
"median": 0.6939384473000001,
"user": 0.36534173999999997,
"system": 1.4973955199999998,
"min": 0.6778638448000001,
"max": 0.7844695078,
"times": [
0.7844695078,
0.6809777238000001,
0.7080190168,
0.7078425158,
0.6827894548000001,
0.6870106328000001,
0.6778638448000001,
0.6951837718,
0.7227588798000001,
0.6926931228000001
]
},
{
"command": "pacquet@main",
"mean": 0.6951652138,
"stddev": 0.02867623454843833,
"median": 0.6834080083,
"user": 0.3663634399999999,
"system": 1.50051122,
"min": 0.6714075638000001,
"max": 0.7633075788000001,
"times": [
0.7279181838000001,
0.6815169548000001,
0.6750443298000001,
0.6822415148000001,
0.6714075638000001,
0.6883117958,
0.6809647198000001,
0.7633075788000001,
0.6845745018,
0.6963649948
]
},
{
"command": "pnpm",
"mean": 2.4718791346999995,
"stddev": 0.12043040896624935,
"median": 2.4211457803,
"user": 2.8639106399999994,
"system": 2.2188443200000005,
"min": 2.3762026767999997,
"max": 2.7644012277999996,
"times": [
2.4152474248,
2.4802264478,
2.4031720628,
2.7644012277999996,
2.4407807158,
2.4111915317999997,
2.3762026767999997,
2.4014213017999997,
2.5991038218,
2.4270441357999997
]
}
]
} |
…explain should_be_built vs key dim (#462 review) CodeRabbit pointed out the visual inconsistency in `fetcher_skips_build_when_ignore_scripts`: `files_index_file` carried `\tnot-built` while `received.built` asserted `true`. The two are different concepts but the test made them look conflated. - Switch the key to `\tbuilt` so it matches what pacquet's dispatcher currently passes (the dispatcher hardcodes `built=true`; upstream would flip this with `!ignore_scripts` once pacquet adds the ignore-scripts config knob — both sites move together at that point). - Comment at the call site spells out the relationship: the key's `built` dimension is what `pickStoreIndexKey` produces from the dispatcher's POV (currently always `built`), while `received.built` is `prepare_package`'s `should_be_built` flag (does the manifest declare a build?). These can disagree — exactly the state this test exercises. - `assert!` message restated to highlight that `should_be_built` reports the manifest's *intent*, not whether scripts ran. No production code touched; pure test clarity.
Pull in 4 commits from upstream main: - feat(lockfile): BinaryResolution + VariationsResolution (#457) - feat: hoisting support (hoistPattern + publicHoistPattern) (#445) - test(git-fetcher): port §E git-fetcher tests (#462) - feat(real-hoist): ancestor-path-aware peer-shadow refusal (#461) Conflict: `crates/config/src/lib.rs` — the hoisting PR (#445) added `pub mod matcher;` adjacent to my `mod env_replace;`. Keep both module declarations.
Summary
Ports five new tests against the local-bare-repo and write-to-CAS fixtures already established by §B+D and §C, plus the
plans/TEST_PORTING.mdcheckbox updates for the §E subset that ships today (15 items checked off; the rest carry concrete deferral reasons — Stage 2 resolver work, real-npm requirements, perf benchmarks, or follow-up tests that need a PATH-shim helper).New tests in
crates/git-fetcher/src/fetcher/tests.rsfetcher_packs_subfolder_when_path_set— portsfetching/git-fetcher/test/index.ts:69. Builds a tiny monorepo bare repo, runsGitFetcherwithpath: Some("packages/sub"), asserts the returnedcas_pathsonly contain files relative to the sub-dir (never sibling packages, never carrying thepackages/prefix).fetcher_handles_repo_without_package_json— portsfetching/git-fetcher/test/index.ts:150. A repo with nopackage.jsonat root still imports cleanly:prepare_packagereturnsshould_be_built: false, and the fetcher hands the install dispatcher the files the packlist found.fetcher_skips_build_when_ignore_scripts— portsfetching/git-fetcher/test/index.ts:247. A repo whosescripts.preparewould fail if it ran (exit 1) still produces a successful fetcher run whenignore_scripts: true; observing success proves the lifecycle runner never spawned. Pins theshould_be_built: trueshort-circuit so callers know the package wanted a build.New tests in
crates/git-fetcher/src/tarball_fetcher/tests.rstarball_path_traversal_attack_is_rejected— portsfetching/tarball-fetcher/test/fetch.ts:610.path: Some("../escape")must surface asINVALID_PATHfromprepare_package'ssafe_join_pathbefore any extraction.tarball_path_to_missing_subdir_is_rejected— portsfetching/tarball-fetcher/test/fetch.ts:637.path: Some("does/not/exist")for a tarball that doesn't contain the sub-dir must surface asINVALID_PATHrather than silently packing the root.plans/TEST_PORTING.mdupdates15 §E items moved from
[ ]to[x]with the covering pacquet test name. Deferrals carry concrete reasons:fetch a package from Git that has a prepare script,fail when preparing a git-hosted package,allow git package with prepare script. Defer until askip-if-no-npmtest helper exists.still able to shallow fetch for allowed hosts. The predicate is unit-tested; the end-to-endfetch --depth 1assertion needs agit-binary spy.fetch a big repository.Out of scope (still on #436)
${filesIndexFile}\traw+ final).npm-packlistsemantics (.npmignore,.gitignore,bundleDependencieswalking).Test plan
cargo nextest run -p pacquet-git-fetcher— 47 tests pass (42 from previous PRs + 5 new).just ready— 842 tests run, 842 pass.RUSTDOCFLAGS=-D warnings cargo doc --no-deps --workspace --document-private-items— clean.taplo format --checkjust dylintWritten by an agent (Claude Code, claude-opus-4-7).
Summary by CodeRabbit
Tests
Documentation