Skip to content

fix(lockfiles): retain base packages entry for peer-variant injected workspace deps#12825

Closed
Eyalm321 wants to merge 3 commits into
vercel:mainfrom
Eyalm321:lockfiles/retain-peer-variant-base-entry
Closed

fix(lockfiles): retain base packages entry for peer-variant injected workspace deps#12825
Eyalm321 wants to merge 3 commits into
vercel:mainfrom
Eyalm321:lockfiles/retain-peer-variant-base-entry

Conversation

@Eyalm321

Copy link
Copy Markdown

Fixes #12824. Residual of #12073 / #12002 / #11059.

Problem

PnpmLockfile::subgraph retains the peer-variant snapshot key for an injected file: workspace dep (@repo/shared@file:packages/shared(react@18.2.0)) but get_packages(&key) is None for that suffixed key — the resolution lives on the base packages: entry @repo/shared@file:packages/shared. So the base entry is dropped from the pruned lockfile, and pnpm install --frozen-lockfile later crashes with Cannot use 'in' operator to search for 'directory' in undefined.

Fix

After retaining the variant, strip the peer suffix and also retain the base packages: entry — reusing the exact DepPath::parse(...).format_key(dp.name, dp.version) peer-suffix handling the transitive-deps path in this same function already uses a few lines below. ~13 lines, no new APIs.

Notes for review:

  • .or_insert_with so an already-retained base entry is never clobbered.
  • base_key != key makes it a strict no-op when there is no peer suffix, so the non-variant case fix: Retain injected workspace package entries during pnpm lockfile pruning #12073 added is unaffected.
  • turbo only ever retains an existing entry here (never synthesizes), so tarball file:*.tgz refs, non-injected peers, and multiple peer variants are all safe.

Test

test_subgraph_injected_peer_variant_retains_base_resolution, modeled on the existing test_subgraph_with_injected_workspace_packages_setting: an injected @repo/shared with a react peer → base packages: entry + peer-variant snapshots: entry. Asserts the pruned lockfile retains the base entry and its directory: resolution. Fails before this change, passes after.

cargo test -p turborepo-lockfiles: 181 passed, 0 failed.

…workspace deps

Residual of vercel#12073 / vercel#12002 / vercel#11059. `turbo prune` retains the
peer-variant snapshot key for an injected `file:` workspace dep but
not the base `packages:` entry that carries its `resolution`, so pnpm
later crashes on `pnpm install --frozen-lockfile`. Strip the peer
suffix and also retain the base entry, reusing the DepPath peer-suffix
handling the transitive-deps path in the same function already uses.
@Eyalm321 Eyalm321 requested a review from a team as a code owner May 18, 2026 02:26
@Eyalm321 Eyalm321 requested review from tknickman and removed request for a team May 18, 2026 02:26
@vercel

vercel Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

Someone is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

Eyalm321 pushed a commit to Eyalm321/pnpm that referenced this pull request May 18, 2026
vercel/turborepo#12819 was closed; the live root-cause fix (retain the
base packages entry for peer-variant injected deps) is tracked at
vercel/turborepo#12825. Keeps the source comment consistent with the
PR description.
github-actions Bot pushed a commit to Eyalm321/pnpm that referenced this pull request May 18, 2026
vercel/turborepo#12819 was closed; the live root-cause fix (retain the
base packages entry for peer-variant injected deps) is tracked at
vercel/turborepo#12825. Keeps the source comment consistent with the
PR description.
@anthonyshew

Copy link
Copy Markdown
Contributor

Closing for slop.

@Eyalm321

Eyalm321 commented May 18, 2026

Copy link
Copy Markdown
Author

Its a real bug and a real fix tho, it is technically coherent. You can try and reproduce it.

Edit: working on the complete repro for you

https://github.com/Eyalm321/turbo-12824-repro

zkochan added a commit to Eyalm321/pnpm that referenced this pull request May 19, 2026
Mirror the TS-side defense-in-depth fix on the pacquet stack so a
pruned `injectWorkspacePackages` lockfile (older `turbo prune --docker`
output, pre vercel/turborepo#12825) loads on Rust too. Two pieces:

1. Widen `PkgVerPeer` to accept `file:` versions. The version slot
   becomes a `VersionPart::{Semver(Version), File(String)}` sum type
   so a snapshot key like `pkg@file:packages/pkg(peer@1.0.0)` parses
   instead of failing at the semver `from_str`. Display writes the
   `file:` scheme back, so round-trips stay byte-stable. The single
   semver-required caller (`find_runtime_node_major` reading
   `.major`) is migrated to a new `version_semver()` accessor and
   stays gated by the existing `Prefix::Runtime` check; `runtime:`
   and `file:` are explicitly mutually exclusive at parse time.

2. Add `Lockfile::reconstruct_missing_directory_resolutions()` and
   run it at the end of `load_from_path`. This is the analog of the
   TS `convertToLockfileObject` synthesis at
   `lockfile/fs/src/lockfileFormatConverters.ts`: for any peer-variant
   snapshot whose `without_peer()` lookup misses, fabricate a
   `LockfileResolution::Directory` from the `file:` path so every
   downstream `packages.get(...).resolution` call keeps working. The
   tarball-vs-directory boundary mirrors the resolver's
   `/\\.(?:tgz|tar\\.gz|tar)$/i` regex via a case-insensitive suffix
   check on `.tgz` / `.tar.gz` / `.tar`.

Tests:

- `pkg_ver_peer/tests.rs`: round-trip + serde for `file:` (plain and
  with peer suffix); `parse_err` cases for empty path after `file:`
  and `runtime:file:` conflict.
- `load_lockfile/tests.rs`: pruned lockfile with one directory
  peer-variant + lowercase / uppercase / mixed-case tarball
  peer-variants asserts that only the directory case gets a
  synthesized resolution.

---
Written by an agent (Claude Code, claude-opus-4-7).
zkochan pushed a commit to Eyalm321/pnpm that referenced this pull request May 19, 2026
vercel/turborepo#12819 was closed; the live root-cause fix (retain the
base packages entry for peer-variant injected deps) is tracked at
vercel/turborepo#12825. Keeps the source comment consistent with the
PR description.
zkochan added a commit to Eyalm321/pnpm that referenced this pull request May 19, 2026
Mirror the TS-side defense-in-depth fix on the pacquet stack so a
pruned `injectWorkspacePackages` lockfile (older `turbo prune --docker`
output, pre vercel/turborepo#12825) loads on Rust too. Two pieces:

1. Widen `PkgVerPeer` to accept `file:` versions. The version slot
   becomes a `VersionPart::{Semver(Version), File(String)}` sum type
   so a snapshot key like `pkg@file:packages/pkg(peer@1.0.0)` parses
   instead of failing at the semver `from_str`. Display writes the
   `file:` scheme back, so round-trips stay byte-stable. The two
   semver-required callers (`find_runtime_node_major` and
   `find_own_runtime_node_major`) migrate to a new `version_semver()`
   accessor; `runtime:` and `file:` are explicitly mutually exclusive
   at parse time.

2. Add `Lockfile::reconstruct_missing_directory_resolutions()` and
   run it at the end of `load_from_path`. Analog of the TS
   `convertToLockfileObject` synthesis: for any peer-variant snapshot
   whose `without_peer()` lookup misses, fabricate a
   `LockfileResolution::Directory` from the `file:` path so every
   downstream `packages.get(...).resolution` call keeps working. The
   tarball-vs-directory boundary mirrors the resolver's
   `/\\.(?:tgz|tar\\.gz|tar)$/i` regex via a case-insensitive suffix
   check on `.tgz` / `.tar.gz` / `.tar`.

Tests:

- `pkg_ver_peer/tests.rs`: round-trip + serde for `file:` (plain and
  with peer suffix); `parse_err` cases for empty path after `file:`
  and `runtime:file:` conflict.
- `load_lockfile/tests.rs`: pruned lockfile with one directory
  peer-variant + lowercase / uppercase / mixed-case tarball
  peer-variants asserts that only the directory case gets a
  synthesized resolution.

---
Written by an agent (Claude Code, claude-opus-4-7).
zkochan added a commit to pnpm/pnpm that referenced this pull request May 19, 2026
…th + lifecycle re-import (#11662)

Fixes two **independent** crashes hitting `pnpm install --frozen-lockfile` on workspaces with `injectWorkspacePackages: true` (or `dependenciesMeta.*.injected`), surfaced via `turbo prune --docker` pipelines.

## Bug 1 — peer-variant snapshot missing `resolution` (lean, defense-in-depth)

A peer-variant injected workspace snapshot (`@scope/pkg@file:packages/pkg(peerA@1)(peerB@2)`) inherits its `resolution` from the base `packages:` entry (`@scope/pkg@file:packages/pkg`). When a tool prunes the lockfile and drops that base entry, readers that deref `pkgSnapshot.resolution` crash with the cryptic:

```
Cannot use 'in' operator to search for 'directory' in undefined
```

**The root cause is upstream of pnpm**: the pruner (e.g. `turbo prune`) emits an internally inconsistent lockfile. Fixed at the source in **vercel/turborepo#12825** (retain the base entry for peer-variant injected deps; minimal repro in **vercel/turborepo#12824**) — empirically verified to produce a correct pruned lockfile for a real multi-service workspace.

**pnpm side (this PR): one lean normalization at the read layer** — in `convertToLockfileObject`, where base→variant inheritance already happens via `Object.assign`. When the base entry is absent, reconstruct the directory resolution from the `file:` depPath. This is *reconstruction, not guessing*: for a workspace `file:` dep the directory **is** the depPath suffix — exactly what pnpm's own writer emits. It is **defense-in-depth, not load-bearing**: with a well-formed lockfile (turbo#12825 or any correct input) the branch never fires. Because the normalization sits at the single shared read layer, it also covers the sibling `Cannot use 'in' operator … 'integrity' in undefined` on the `pnpm deploy` path (same `resolution === undefined` root, different deref site).

Per review feedback: the earlier per-reader `inheritOrSynthesizeResolution` helper across 5 call sites is **removed**; normalization lives in exactly one place (`convertToLockfileObject`), and the readers are back to `main`.

## Bug 2 — lifecycle re-import wipes `.bin/<tool>` (pure pnpm; the substantive fix)

`runLifecycleHooksConcurrently` re-imports an injected workspace package into its targets after `prepare`/`postinstall`. The 2022 `scanDir`-into-`filesMap` workaround (#4299) fed target-internal paths to `importPackage`; once #11088 made `importIndexedDir`'s `makeEmptyDir` fast path the default, that path wipes the target's `node_modules` before copying, so the re-import dies with `ERR_PNPM_ENOENT` on `node_modules/.bin/<tool>`.

Fix: drop the `scanDir` workaround and pass `keepModulesDir: true` so `importIndexedDir` skips the destructive fast path and preserves the target's existing `node_modules` (bin symlinks + transitive deps) via its staging/move path. Stays on `storeController.importPackage`, so source files keep their **hardlinks** (no copy-loop regression). Net reduction vs `main`: the `scanDir` helper and the `node:fs` / `FilesMap` imports are removed.

## Tests

- The `deps-restorer` regression fixture `peer-variant-missing-resolution` **omits the base `packages:` entry**, so it encodes the actual pruned shape and reproduces the crash on `main`: reverting the `convertToLockfileObject` change yields `resolution: undefined` for the peer-variant (→ the `lockfileToDepGraph` crash); with this PR it is reconstructed as `{ type: 'directory', directory: … }`.
- A `lockfile.fs` unit test pins the heuristic boundary: a directory resolution is synthesized for a pruned `file:` peer-variant but **never** for a `file:` tarball.
- A `deps-installer` regression test covers the Bug 2 re-import (injected dep with a `prepare` script + a bin-having dependency).

## Validation

End-to-end on a real `injectWorkspacePackages` monorepo (`turbo prune --docker` → `pnpm install --frozen-lockfile`), on services that crash on **both** bugs with stock pnpm:

- pnpm with both fixes: the crashing services build.
- **vercel/turborepo#12825 + pnpm with only Bug 2** (Bug 1 fully reverted): the crashing services still **build** → confirms Bug 1 here is genuine defense-in-depth and turbo#12825 owns the root cause.
- Bug 2 reproduces on stock pnpm regardless of turbo (it is purely pnpm's importer fast-path).

Pairs with **vercel/turborepo#12825** (Bug 1 root cause; minimal repro **vercel/turborepo#12824**). Tracks #11663.

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Eyalm321 <eyal@sunsationsusa.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: UApply Developer <developer@uapply.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

turbo prune drops the base packages entry for peer-variant injected workspace deps, breaking pnpm install

2 participants