Skip to content

fix: pin integrity of git-hosted tarballs in lockfile#11481

Merged
zkochan merged 15 commits into
mainfrom
fix-vuln-git-tarballs
May 6, 2026
Merged

fix: pin integrity of git-hosted tarballs in lockfile#11481
zkochan merged 15 commits into
mainfrom
fix-vuln-git-tarballs

Conversation

@zkochan

@zkochan zkochan commented May 5, 2026

Copy link
Copy Markdown
Member

Summary

For git-hosted tarballs (codeload.github.com / gitlab.com / bitbucket.org) the fetcher dropped the integrity it computed while downloading, so the lockfile only ever stored the URL. A compromised git host or man-in-the-middle could serve a substituted tarball on subsequent installs and pnpm would install it — the lockfile had no hash to compare against.

This pins the SHA-512 SRI of the raw tarball in the lockfile, in the same sha512-<base64> form npm-registry tarballs use. The only difference is the source: for npm we pass through dist.integrity, for git we compute it locally from the downloaded buffer. Subsequent installs validate the download against that integrity in the worker (addTarballToStoreparseIntegrity → hash compare), so a tampered tarball fails with TarballIntegrityError.

Why git-hosted stays on gitHostedStoreIndexKey

The lockfile pins integrity for security, but the store key for git-hosted resolutions stays on gitHostedStoreIndexKey(pkgId, { built }) rather than collapsing under the integrity-based key. Reason: git-hosted tarballs are post-processed (preparePackage / packlist), so the cached file set depends on whether build scripts ran during fetch. The integrity-only key would fold the built and not-built variants into a single slot, letting one overwrite the other and serving the wrong content if ignoreScripts was toggled between runs. Keeping git-hosted on the existing key shape preserves that dimension; the integrity is still validated on every fresh download.

How the routing stays clean

The naive way to express "use gitHostedStoreIndexKey for git-hosted, integrity key for npm" is to call isGitHostedPkgUrl(resolution.tarball) everywhere a store key is computed — fragile, scattered, and easy to forget when adding new readers (Copilot caught two of those during review). Instead, a typed annotation: TarballResolution gets an optional gitHosted: boolean field. The git resolver sets it; the lockfile loader (convertToLockfileObject) backfills it for entries written by older pnpm versions; toLockfileResolution carries it through on serialize. Every consumer reads resolution.gitHosted directly. URL detection lives in exactly two places — the resolver and the loader — instead of seven.

Changes

Security fix

  • fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts — return the integrity that the inner remote-tarball fetch already computed (was being silently dropped by the destructure).

Lockfile schema (additive)

  • @pnpm/lockfile.types and @pnpm/resolving.resolver-baseTarballResolution gains optional gitHosted: boolean.
  • @pnpm/resolving.git-resolver — sets gitHosted: true on every git-hosted tarball it produces.
  • @pnpm/lockfile.fs (convertToLockfileObject) — backfills the field on load for older lockfiles via inlined URL detection.
  • @pnpm/lockfile.utils (toLockfileResolution, pkgSnapshotToResolution) — preserve / read the field.

Store-key consumers (now one-line typed reads, dropped the URL-sniffing dep)

  • installing/package-requester (getFilesIndexFilePath)
  • store/pkg-finder (readPackageFileMap)
  • modules-mounter/daemon (createFuseHandlers)
  • building/after-install (side-effects-cache lookup + write)
  • store/commands/storeStatus
  • installing/deps-installer (agent-mode store-controller wrapper)

Fetcher routing

  • fetching/pick-fetcherpickFetcher prefers resolution.gitHosted; URL fallback retained for ad-hoc resolutions.

Tests

  • New integrity-validation test in tarball-fetcher (mismatched integrity on the resolution must throw TarballIntegrityError).
  • New git-hosted lookup test in pkg-finder asserting routing through gitHostedStoreIndexKey even when integrity is present.
  • New toLockfileResolution test asserting gitHosted: true flows through serialization.
  • fromRepo.ts lockfile snapshot updated for the now-pinned integrity + gitHosted: true.
  • git-resolver tests updated to assert gitHosted: true in produced resolutions.

Test plan

  • fetching/tarball-fetcher — 22/22 (includes the new TarballIntegrityError test).
  • installing/package-requester — 20/20.
  • store/pkg-finder — 6/6 (includes the new git-hosted-with-integrity test).
  • resolving/git-resolver — 71/71.
  • lockfile/utils — 10/10 (includes the new gitHosted serialization tests).
  • lockfile/fs — 39/39.
  • store/commands — 34/34.
  • installing/deps-installer fromRepo — 12/12 (3 pre-existing skipped).
  • deps/compliance/commands licenses — 11/11 (catches reader/writer-mismatches because it walks the lockfile + store right after install).
  • deps/inspection/outdated — 15/15.

Written by an agent (Claude Code, claude-opus-4-7).

The git-hosted tarball fetcher dropped the integrity computed during
download, so the lockfile only stored the codeload.github.com / gitlab /
bitbucket URL. A compromised git host or MITM could then serve a
different tarball on subsequent installs without lockfile changes.

Propagate the SHA-512 SRI of the raw tarball back to the lockfile so the
worker validates it on every fresh fetch and rejects tampered tarballs
with TarballIntegrityError.

---
Written by an agent (Claude Code, claude-opus-4-7).
Copilot AI review requested due to automatic review settings May 5, 2026 23:34
@coderabbitai

coderabbitai Bot commented May 5, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Fetchers now surface computed tarball integrity for git-hosted packages; callers forward package ids so fetchers can register integrity-keyed store-index entries; lockfile resolutions include an integrity field; tests and a changeset were added/updated to reflect these metadata and lookup changes.

Changes

Git-hosted tarball integrity propagation

Layer / File(s) Summary
Changeset / Lockfile expectations
.changeset/git-tarball-integrity.md, installing/deps-installer/test/install/fromRepo.ts
Adds a changeset and updates tests to expect lockfile tarball resolution objects to include both tarball and integrity.
Fetch API shape
fetching/fetcher-base/src/index.ts
FetchOptions gets optional pkgId?: string so fetchers can register integrity-keyed store index entries.
Core fetcher: compute & return integrity
fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts, fetching/tarball-fetcher/test/fetch.ts
Destructures and returns integrity from fetchRemoteTarball; registers a store-index entry at storeIndexKey(integrity, pkgId) when present; tests added to assert integrity is surfaced and verified.
Caller wiring: forward pkgId & storeIndex
installing/package-requester/src/packageRequester.ts, store/controller/src/storeController/index.ts
createPackageRequester accepts/forwards storeIndex; fetch invocations include pkgId: opts.pkg.id and pass storeIndex through to fetch logic.
Migration / index compatibility
installing/package-requester/src/packageRequester.ts
When legacy index entries exist but the new files index is missing, migration copies legacy git-hosted tarball entries into the new files index to preserve integrity-based resolution.
Store lookup by integrity
store/pkg-finder/src/index.ts, store/pkg-finder/test/readPackageFileMap.test.ts
Integrity-based lookup uses storeIndexKey(integrity, pkgId) with raw pkgId (removes parse-based derivation); test added to verify resolution by integrity finds stored files.
Consumer alignment: pkgId derivation & fallback
building/after-install/src/index.ts, modules-mounter/daemon/src/createFuseHandlers.ts
pkgId derived as nonSemverVersion ?? name@version for cache/index keys; getPkgInfo uses integrity-keyed lookup when present, else falls back to git-hosted store key.
sequenceDiagram
  participant PR as PackageRequester
  participant TF as TarballFetcher
  participant RT as RemoteTarball
  participant SI as StoreIndex
  participant LF as Lockfile

  PR->>TF: fetch(tarballUrl, pkgId, filesIndexFile, ...)
  TF->>RT: download tarball
  RT-->>TF: filesMap, manifest, requiresBuild, integrity
  TF->>SI: write files index at filesIndexFile
  TF->>SI: if integrity && pkgId write index at storeIndexKey(integrity, pkgId)
  TF-->>PR: return filesMap, manifest, requiresBuild, integrity
  PR->>LF: record resolution { tarball, integrity }
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

Poem

🐰
I sniffed the tarball, found its trace,
Stashed sha512 in the lockfile's lace.
Little hops, a safe-fetch cheer,
No tampered bytes shall linger near.
Hooray — the burrow's tidy here.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 11.11% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'fix: pin integrity of git-hosted tarballs in lockfile' directly and clearly summarizes the main change: adding integrity pinning to git-hosted tarballs in lockfiles to prevent tampering.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-vuln-git-tarballs

Comment @coderabbitai help to get the list of available commands and usage tips.

The previous deadbeef… literal was 128 hex chars stuffed after `sha512-`
which doesn't match the SRI shape pnpm actually emits (88 base64 chars).
Replace it with a real (but mismatched) sha512 base64 string so the
fixture mirrors what a tampered lockfile entry would look like.

---
Written by an agent (Claude Code, claude-opus-4-7).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to harden pnpm's handling of git-hosted tarball dependencies by persisting a computed tarball integrity in the lockfile, so later installs can verify downloaded content. It touches the tarball fetcher, package requester, install tests, and release notes.

Changes:

  • Propagate computed integrity from git-hosted tarball downloads back into package resolution data.
  • Keep git-hosted tarballs on the existing store index key path to avoid unnecessary refetches with older stores.
  • Add tests and snapshot updates asserting the new resolution.integrity behavior for git-hosted dependencies.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
installing/package-requester/src/packageRequester.ts Special-cases git-hosted tarballs when choosing the store index key.
installing/deps-installer/test/install/fromRepo.ts Updates lockfile expectations for git-hosted installs to include integrity.
fetching/tarball-fetcher/test/fetch.ts Adds fetcher tests for integrity propagation and mismatch rejection.
fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts Returns the raw tarball integrity from the git-hosted fetch path.
.changeset/git-tarball-integrity.md Documents the security-focused patch release.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread installing/package-requester/src/packageRequester.ts Outdated
Comment thread installing/package-requester/src/packageRequester.ts Outdated
zkochan added 2 commits May 6, 2026 01:46
…tarballs

The carve-out that kept git-hosted tarballs on gitHostedStoreIndexKey
even when the lockfile carried integrity made the writer disagree with
every other reader: pkg-finder, store status, the FUSE mounter, and
rebuild --skip-if-has-side-effects-cache all derive the index path from
storeIndexKey(integrity, pkgId). Lockfile entries with integrity would
silently miss the cached package and side-effects builds would
re-run (potentially landing under a second key).

Drop the carve-out and use the integrity-based key consistently. First
install after upgrading still re-fetches once (the old store entry is
under the gitHosted key), but the cafs deduplicates the underlying file
content, and subsequent installs reuse the entry like every other
tarball.

---
Written by an agent (Claude Code, claude-opus-4-7).
Once the lockfile pins integrity for git-hosted tarballs, every
post-fetch reader (pkg-finder, store status, FUSE mounter,
skipIfHasSideEffectsCache in building/after-install) looks the package
up under storeIndexKey(integrity, pkgId). Three issues were exposed:

1. pkg-finder ran parse(packageId) and rebuilt name@version, but
   packageIdFromSnapshot returns the URL alone for git-hosted (the
   `:` in `https://` makes tryGetPackageId strip the `name@`
   prefix). parse(URL) returns {} and the lookup key became
   `…\tundefined@undefined`. Use packageId directly — that's the
   form the writer in package-requester uses.

2. The FUSE handler and the after-install side-effects-cache lookup
   used `${name}@${version}`, which doesn't match the resolver's
   pkg.id for git-hosted (URL). Use
   `nonSemverVersion ?? ${name}@${version}` so the readers compute
   the same key the writer uses.

3. On the first install, integrity isn't known when
   getFilesIndexFilePath runs, so the writer parks the index under
   gitHostedStoreIndexKey. Once the worker computes integrity, the
   git-hosted fetcher now also registers the entry under
   storeIndexKey(integrity, pkgId), so subsequent integrity-based
   lookups (the same install or any later one) resolve to the same
   data without a refetch.

The license test that walks the lockfile right after install now
passes locally; that's the case that was failing in CI.

---
Written by an agent (Claude Code, claude-opus-4-7).
Copilot AI review requested due to automatic review settings May 6, 2026 00:01

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@modules-mounter/daemon/src/createFuseHandlers.ts`:
- Around line 179-185: The code in createFuseHandlers.ts uses a non-null
assertion on (pkgSnapshot.resolution as TarballResolution).integrity when
building pkgIndexFilePath with storeIndexKey, which can throw if integrity is
missing for git-hosted packages; change the logic in the block that computes
pkgIndexFilePath (used by getPkgInfo) to check whether integrity exists and, if
not, call gitHostedStoreIndexKey with the appropriate nonSemver identifier
(nameVer.nonSemverVersion ?? `${nameVer.name}@${nameVer.version}`) instead of
storeIndexKey, and add an import for gitHostedStoreIndexKey from
`@pnpm/store.index`. Ensure both branches produce the same pkgIndexFilePath
variable for downstream use.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7253807f-bf11-40d9-90e0-878e68f6f8bc

📥 Commits

Reviewing files that changed from the base of the PR and between fb2a477 and ec6473e.

📒 Files selected for processing (7)
  • building/after-install/src/index.ts
  • fetching/fetcher-base/src/index.ts
  • fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts
  • installing/package-requester/src/packageRequester.ts
  • modules-mounter/daemon/src/createFuseHandlers.ts
  • store/pkg-finder/src/index.ts
  • store/pkg-finder/test/readPackageFileMap.test.ts

Comment thread modules-mounter/daemon/src/createFuseHandlers.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts Outdated
Two cases surfaced by the latest reviewer round:

1) FUSE handler crashed with a non-null assertion on integrity for
   pre-CVE-fix lockfiles that don't carry integrity for git-hosted
   tarballs. Fall back to gitHostedStoreIndexKey so the daemon can
   still serve those packages.

2) Once a lockfile committed by a newer pnpm carries integrity,
   getFilesIndexFilePath probes only storeIndexKey(integrity, pkgId).
   An older local store may still have the entry under
   gitHostedStoreIndexKey, so the install would refetch — and in
   offline mode hard-fail with NO_OFFLINE_TARBALL — even though the
   tarball is already on disk. Plumb the StoreIndex into the
   package-requester and migrate the entry to the integrity key on
   the fly when we detect this state.

Both fallbacks are tagged TODO(v12): once v12 forces lockfile
regeneration and any pnpm v11-written store will already have the
integrity-keyed entry, this bridging code can be removed.

---
Written by an agent (Claude Code, claude-opus-4-7).
@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

# Scenario main branch
1 Headless (warm store+cache) 1.844s ± 0.052s 1.819s ± 0.029s
2 Re-resolution (add dep, warm) 2.409s ± 0.046s 2.400s ± 0.037s
3 Full resolution (warm, no lockfile) 5.508s ± 0.030s 5.496s ± 0.041s
4 Headless (cold store+cache) 4.624s ± 0.069s 4.603s ± 0.059s
5 Cold install (nothing warm) 10.727s ± 0.172s 10.760s ± 0.119s
6 GVS warm reinstall (warm global store) 0.918s ± 0.036s 0.909s ± 0.017s

Run 25431128576 · 15 runs per scenario · triggered by @zkochan

…checks

The migration block that copies a git-hosted tarball entry from the
legacy gitHostedStoreIndexKey to storeIndexKey(integrity, pkgId) was
calling storeIndex.has() on every fetch, including all npm-registry
packages it has no business probing — a SELECT per package, multiplied
across thousands of dependencies.

Reorder so the URL prefix check (isGitHostedPkgUrl) and the integrity
presence check run first, and the SQLite lookup only runs for
git-hosted resolutions with integrity in the lockfile. Same migration
behaviour, ~zero overhead on the hot path.

---
Written by an agent (Claude Code, claude-opus-4-7).
Copilot AI review requested due to automatic review settings May 6, 2026 07:57

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@installing/package-requester/src/packageRequester.ts`:
- Around line 532-545: The migration guard is re-reading opts.pkg.resolution
instead of the resolution chosen earlier (the one used by
getFilesIndexFilePath), so when that selected resolution is a concrete
AtomicResolution (type 'variations') the guard misfires; change the code in the
guard to use the previously selected resolution variable (e.g. resolution or
selectedResolution) for tarballUrl and integrity checks and when building
legacyKey (keep the TarballResolution/AtomicResolution type assertions in line
with that variable), so the isGitHostedPkgUrl(tarballUrl) &
ctx.storeIndex.has(filesIndexFile) logic correctly detects and migrates legacy
git-hosted entries.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0f042243-82f7-415f-a041-cc7c7eb34c56

📥 Commits

Reviewing files that changed from the base of the PR and between 15f0857 and d65a383.

📒 Files selected for processing (1)
  • installing/package-requester/src/packageRequester.ts

Comment thread installing/package-requester/src/packageRequester.ts Outdated
A v10/v11 lockfile that doesn't yet record integrity will keep working
fine: the resolver returns no integrity, getFilesIndexFilePath falls
back to gitHostedStoreIndexKey, and readPkgFromCafs still finds the
cached entry. The migration only mattered for one specific case — a
v11-written lockfile (with integrity) landing on top of a stale v10
store — and the cost of dropping it is one extra HTTP fetch per
git-hosted package, once. The fetcher's dual-write then registers the
entry under both keys, so subsequent installs are cache hits.

Trade-off: `pnpm install --offline` on a stale v10 store immediately
after upgrading would fail with NO_OFFLINE_TARBALL for git deps until
an online run warms the cache under the integrity key. That's narrow
enough that keeping a permanent state-mutating shim in the hot fetch
path isn't worth it.

Removes the optional StoreIndex parameter on createPackageRequester
and the wiring through fetchToStore that existed solely for this.

---
Written by an agent (Claude Code, claude-opus-4-7).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Comment thread fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts Outdated
Comment thread installing/package-requester/src/packageRequester.ts Outdated
…alias

Reviewer flagged that storeIndexKey(integrity, pkgId) collapses the
built/not-built dimension that gitHostedStoreIndexKey carries. Toggling
ignoreScripts between installs would let one variant overwrite the
other and a later install with the opposite setting would silently
read the wrong cached content.

Switch the design so git-hosted resolutions are addressed by
gitHostedStoreIndexKey(pkgId, { built }) consistently:

- Writer (getFilesIndexFilePath): detect git-hosted via tarball URL
  prefix and skip the integrity-key fast path; fall through to
  gitHostedStoreIndexKey regardless of whether integrity is present.
- Fetcher: drop the dual-write (no more storeIndexKey alias) and the
  pkgId field on FetchOptions that only existed for it.
- Readers (pkg-finder, modules-mounter/daemon, building/after-install):
  detect git-hosted via tarball URL and route to gitHostedStoreIndexKey.

The lockfile still pins integrity for security and the worker still
validates it on download — that part of the security fix is unchanged.

The FUSE TODO(v12) fallback that compensated for missing-integrity
lockfiles is gone — git-hosted is now keyed the same way regardless
of integrity, so there's no upgrade-only branch to clean up later.

---
Written by an agent (Claude Code, claude-opus-4-7).
Copilot AI review requested due to automatic review settings May 6, 2026 08:20
---
Written by an agent (Claude Code, claude-opus-4-7).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

Comment thread installing/package-requester/src/packageRequester.ts Outdated
Comment thread store/pkg-finder/src/index.ts Outdated
Comment thread .changeset/git-tarball-integrity.md
zkochan added 4 commits May 6, 2026 10:34
Two more readers Copilot flagged that needed the same git-hosted
detection used by pkg-finder / FUSE / after-install:

- store/commands/src/store/storeStatus/index.ts: when checking whether
  the store has the index entry for a lockfile-listed package, was using
  storeIndexKey(integrity, id) whenever integrity was present. With
  git-hosted now carrying integrity in the lockfile, those entries would
  never be found.
- installing/deps-installer/src/install/index.ts (agent-mode store
  controller wrapper): same shape — bypassed the regular store
  controller for any package with integrity, which would skip the
  fallback for git-hosted packages whose actual cached entry is keyed by
  gitHostedStoreIndexKey.

Both now detect git-hosted via the tarball URL prefix and either route
to gitHostedStoreIndexKey (storeStatus) or fall through to the regular
store controller (deps-installer agent path).

Also:
- Updated the readPackageFileMap docstring in pkg-finder to document
  the git-hosted exception explicitly (was implying every
  with-integrity package is keyed by integrity).
- Expanded the changeset to include @pnpm/building.after-install,
  @pnpm/installing.deps-installer, @pnpm/modules-mounter.daemon,
  @pnpm/store.commands, and @pnpm/store.pkg-finder so all touched
  published packages get a patch bump.

---
Written by an agent (Claude Code, claude-opus-4-7).
---
Written by an agent (Claude Code, claude-opus-4-7).
Six readers + a writer all repeated the same isGitHostedPkgUrl(...) check
against the resolution.tarball field — fragile, scattered, and the kind
of pattern that just lost a regression to Copilot's review loop. Lift it
to a typed annotation on the lockfile/resolution.

- TarballResolution gains an optional `gitHosted: boolean` field in
  both @pnpm/lockfile.types and @pnpm/resolving.resolver-base.
- The git resolver sets it whenever it produces a tarball-style
  resolution (codeload.github.com / gitlab.com / bitbucket.org).
- The lockfile loader (@pnpm/lockfile.fs convertToLockfileObject)
  enriches entries written by older pnpm versions: any tarball whose
  URL matches a known git host gets gitHosted=true on load. This
  centralizes URL detection to one place and lets every downstream
  consumer rely on the field.
- toLockfileResolution preserves the field when serializing, with a
  URL fallback for callers that hand it a resolution constructed
  ad-hoc (config-dep migrations, etc.).

Every reader simplifies from the multi-line URL check to a single
typed read of resolution.gitHosted:

  package-requester (getFilesIndexFilePath writer)
  store/pkg-finder (readPackageFileMap)
  modules-mounter/daemon (createFuseHandlers)
  building/after-install (skipIfHasSideEffectsCache + cache write)
  store/commands/storeStatus
  installing/deps-installer (agent-mode store-controller wrapper)

…and they drop their just-added @pnpm/fetching.pick-fetcher dep.

pickFetcher itself now prefers the field (URL fallback retained for
ad-hoc resolutions). pkgSnapshotToResolution does the same.

Lockfile schema: a new optional `gitHosted: true` appears on tarball
resolutions for git-hosted entries. Old lockfiles without it work
correctly via load-time enrichment.

---
Written by an agent (Claude Code, claude-opus-4-7).
---
Written by an agent (Claude Code, claude-opus-4-7).
Copilot AI review requested due to automatic review settings May 6, 2026 09:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 2 comments.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

Comment thread resolving/git-resolver/test/index.ts
Comment thread lockfile/utils/src/toLockfileResolution.ts Outdated
zkochan added 2 commits May 6, 2026 11:39
Six call sites repeated the same shape:

  resolution.gitHosted
    ? gitHostedStoreIndexKey(pkgId, { built })
    : storeIndexKey(resolution.integrity!, pkgId)

Hoist it to @pnpm/store.index next to the two key functions, behind a
single helper that takes the resolution shape and the desired `built`
flag. The helper also folds in the legacy fallback (no integrity, no
gitHosted → gitHostedStoreIndexKey).

Each consumer collapses from a 3-5 line conditional to a one-liner:

  pickStoreIndexKey(resolution, pkgId, { built })

Touched: package-requester (writer), pkg-finder, FUSE daemon,
after-install (both side-effects-cache lookup and write), and
storeStatus.

---
Written by an agent (Claude Code, claude-opus-4-7).
- toLockfileResolution: import TarballResolution from
  @pnpm/resolving.resolver-base instead of @pnpm/lockfile.types. The
  function takes a resolver-side Resolution, so casting to the
  resolver-side TarballResolution keeps type boundaries clear.
- git-resolver tests: split the bulk-appended `gitHosted: true` onto
  its own line for readability (linter handled the formatting once
  the inline pattern was no longer matched).

---
Written by an agent (Claude Code, claude-opus-4-7).
Copilot AI review requested due to automatic review settings May 6, 2026 09:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated no new comments.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

@zkochan zkochan merged commit 60fd205 into main May 6, 2026
17 checks passed
@zkochan zkochan deleted the fix-vuln-git-tarballs branch May 6, 2026 11:22
zkochan added a commit that referenced this pull request May 6, 2026
Cherry-pick of #11481 from main, adapted to the v10 layout.

For git-hosted tarballs (codeload.github.com / gitlab.com / bitbucket.org)
the fetcher dropped the integrity it computed while downloading, so the
lockfile only stored the URL. A compromised git host or man-in-the-middle
could serve a substituted tarball on subsequent installs and pnpm would
install it without lockfile changes.

This pins the SHA-512 SRI of the raw tarball in the lockfile in the same
sha512-<base64> form npm-registry tarballs use; subsequent installs verify
the download against that integrity in the worker.

A new optional gitHosted: boolean field is recorded on TarballResolution
so every store-key consumer can route by a single typed read instead of
re-deriving the routing from the URL. Lockfiles written by older pnpm
versions are enriched on load (URL fallback) so the field can be relied
on uniformly.

🤖 Cherry-picked by Claude (claude-opus-4-7) on behalf of @zkochan
zkochan added a commit that referenced this pull request May 6, 2026
For git-hosted tarballs (`codeload.github.com` / `gitlab.com` / `bitbucket.org`) the fetcher dropped the integrity it computed while downloading, so the lockfile only ever stored the URL. A compromised git host or man-in-the-middle could serve a substituted tarball on subsequent installs and pnpm would install it — the lockfile had no hash to compare against.

This pins the SHA-512 SRI of the raw tarball in the lockfile, in the same `sha512-<base64>` form npm-registry tarballs use. The only difference is the source: for npm we pass through `dist.integrity`, for git we compute it locally from the downloaded buffer. Subsequent installs validate the download against that integrity in the worker (`addTarballToStore` → `parseIntegrity` → hash compare), so a tampered tarball fails with `TarballIntegrityError`.

## Why git-hosted stays on `gitHostedStoreIndexKey`

The lockfile pins integrity for security, but the *store key* for git-hosted resolutions stays on `gitHostedStoreIndexKey(pkgId, { built })` rather than collapsing under the integrity-based key. Reason: git-hosted tarballs are post-processed (`preparePackage` / `packlist`), so the cached file set depends on whether build scripts ran during fetch. The integrity-only key would fold the built and not-built variants into a single slot, letting one overwrite the other and serving the wrong content if `ignoreScripts` was toggled between runs. Keeping git-hosted on the existing key shape preserves that dimension; the integrity is still validated on every fresh download.

## How the routing stays clean

The naive way to express "use gitHostedStoreIndexKey for git-hosted, integrity key for npm" is to call `isGitHostedPkgUrl(resolution.tarball)` everywhere a store key is computed — fragile, scattered, and easy to forget when adding new readers (Copilot caught two of those during review). Instead, a typed annotation: `TarballResolution` gets an optional `gitHosted: boolean` field. The git resolver sets it; the lockfile loader (`convertToLockfileObject`) backfills it for entries written by older pnpm versions; `toLockfileResolution` carries it through on serialize. Every consumer reads `resolution.gitHosted` directly. URL detection lives in exactly two places — the resolver and the loader — instead of seven.

## Changes

### Security fix
- `fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts` — return the `integrity` that the inner remote-tarball fetch already computed (was being silently dropped by the destructure).

### Lockfile schema (additive)
- `@pnpm/lockfile.types` and `@pnpm/resolving.resolver-base` — `TarballResolution` gains optional `gitHosted: boolean`.
- `@pnpm/resolving.git-resolver` — sets `gitHosted: true` on every git-hosted tarball it produces.
- `@pnpm/lockfile.fs` (`convertToLockfileObject`) — backfills the field on load for older lockfiles via inlined URL detection.
- `@pnpm/lockfile.utils` (`toLockfileResolution`, `pkgSnapshotToResolution`) — preserve / read the field.

### Store-key consumers (now one-line typed reads, dropped the URL-sniffing dep)
- `installing/package-requester` (`getFilesIndexFilePath`)
- `store/pkg-finder` (`readPackageFileMap`)
- `modules-mounter/daemon` (`createFuseHandlers`)
- `building/after-install` (side-effects-cache lookup + write)
- `store/commands/storeStatus`
- `installing/deps-installer` (agent-mode store-controller wrapper)

### Fetcher routing
- `fetching/pick-fetcher` — `pickFetcher` prefers `resolution.gitHosted`; URL fallback retained for ad-hoc resolutions.

### Tests
- New integrity-validation test in `tarball-fetcher` (mismatched `integrity` on the resolution must throw `TarballIntegrityError`).
- New git-hosted lookup test in `pkg-finder` asserting routing through `gitHostedStoreIndexKey` even when integrity is present.
- New `toLockfileResolution` test asserting `gitHosted: true` flows through serialization.
- `fromRepo.ts` lockfile snapshot updated for the now-pinned integrity + `gitHosted: true`.
- `git-resolver` tests updated to assert `gitHosted: true` in produced resolutions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants