Conversation
Eyalm321
added a commit
to Eyalm321/pnpm
that referenced
this pull request
May 15, 2026
…t scanDir When `runLifecycleHooksConcurrently` re-imports a workspace package into its injected target dirs after running prepare/postinstall, it has to preserve the existing `node_modules/` inside each target — it was set up during the initial install (bin symlinks, transitive deps) and the post-script re-import only ships the workspace source. The previous approach was a manual `scanDir` over the existing `<targetDir>/node_modules`, adding each absolute path into the re-import's `filesMap`. That worked in 2022 (pnpm#4299) when the importer always used a staging-temp + rename, but pnpm#11088 made the fast path (write directly into the final dir) the default. The fast path's `makeEmptyDirSync(targetDir)` wipes the very files the scanDir entries point at, so `importIndexedDir → tryImportIndexedDir` crashes on the first copy with: ERR_PNPM_ENOENT on <targetDir>/node_modules/.bin/<tool> Reproduces on any workspace package that has a `prepare`/`postinstall` script and any dep with a bin entry, when injected via `injectWorkspacePackages: true` or `dependenciesMeta.<dep>.injected`. The fix passes `keepModulesDir: true` when an injected target already has a `node_modules/`, which routes the import through the staging path. `importIndexedDir`'s `moveOrMergeModulesDirs` then atomically moves the existing `node_modules` into the staging dir — the same mechanism the hoisted node-linker has been using since pnpm#4299. The manual scanDir is removed; `moveOrMergeModulesDirs` covers the same ground without the race against `makeEmptyDirSync`. Regression test added in `injectLocalPackages.ts`: an injected workspace package with a `prepublishOnly` script + `@pnpm.e2e/hello-world-js-bin` dep. Before the fix this throws `ERR_PNPM_ENOENT` on `.bin/hello-world-js-bin`; after the fix both the script output and the bin symlink survive into the injected copy.
Eyalm321
added a commit
to Eyalm321/pnpm
that referenced
this pull request
May 15, 2026
…ipts When `runLifecycleHooksConcurrently` re-imports a workspace package into its injected targets after running prepare/postinstall, it has to preserve the existing `node_modules/` inside each target — set up during the initial install (bin symlinks, transitive deps) — while overlaying the script's output (typically a freshly-built `dist/`). The previous approach was a manual `scanDir` over `<targetDir>/node_modules` that fed absolute paths into the re-import's `filesMap`. That worked in 2022 (pnpm#4299) when `importIndexedDir` always staged-and-renamed, but pnpm#11088 made the fast path (write directly into the final dir) the default. The fast path's `makeEmptyDirSync(targetDir)` wipes the very files the scanned entries point at, so `tryImportIndexedDir`'s first copy throws: ERR_PNPM_ENOENT on <targetDir>/node_modules/.bin/<tool> Reproduces on any workspace package that has a `prepare`/`postinstall` script + a dep with a bin entry, when injected via `injectWorkspacePackages: true` or `dependenciesMeta.<dep>.injected`. Fix: skip the `importPackage` round-trip entirely. After lifecycle scripts complete, iterate the source's `filesMap` (from `fetchFromDir`, which already excludes `node_modules/`) and copy each entry directly into each `targetDir`. The target's existing `node_modules/` is never touched — its bin links + transitive deps stay intact — and there's no tmp-dir staging swap to trip over `.bin/<tool>` symlinks. Regression test in `injectLocalPackages.ts`: an injected workspace package with a `prepublishOnly` script + `@pnpm.e2e/hello-world-js-bin` dep. Before the fix this throws `ERR_PNPM_ENOENT` on `.bin/hello-world-js-bin`; after the fix both the script's output and the bin symlink survive into the injected copy.
zkochan
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 17, 2026
…ipts When `runLifecycleHooksConcurrently` re-imports a workspace package into its injected targets after running prepare/postinstall, it has to preserve the existing `node_modules/` inside each target — set up during the initial install (bin symlinks, transitive deps) — while overlaying the script's output (typically a freshly-built `dist/`). The previous approach was a manual `scanDir` over `<targetDir>/node_modules` that fed absolute paths into the re-import's `filesMap`. That worked in 2022 (pnpm#4299) when `importIndexedDir` always staged-and-renamed, but pnpm#11088 made the fast path (write directly into the final dir) the default. The fast path's `makeEmptyDirSync(targetDir)` wipes the very files the scanned entries point at, so `tryImportIndexedDir`'s first copy throws: ERR_PNPM_ENOENT on <targetDir>/node_modules/.bin/<tool> Reproduces on any workspace package that has a `prepare`/`postinstall` script + a dep with a bin entry, when injected via `injectWorkspacePackages: true` or `dependenciesMeta.<dep>.injected`. Fix: skip the `importPackage` round-trip entirely. After lifecycle scripts complete, iterate the source's `filesMap` (from `fetchFromDir`, which already excludes `node_modules/`) and copy each entry directly into each `targetDir`. The target's existing `node_modules/` is never touched — its bin links + transitive deps stay intact — and there's no tmp-dir staging swap to trip over `.bin/<tool>` symlinks. Regression test in `injectLocalPackages.ts`: an injected workspace package with a `prepublishOnly` script + `@pnpm.e2e/hello-world-js-bin` dep. Before the fix this throws `ERR_PNPM_ENOENT` on `.bin/hello-world-js-bin`; after the fix both the script's output and the bin symlink survive into the injected copy.
Eyalm321
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 17, 2026
Per the PR review (zkochan); pairs with the turbo-side root-cause fix vercel/turborepo#12819. Bug 1 (peer-variant resolution): normalize once at the read layer in convertToLockfileObject — when a pruned lockfile dropped the base packages entry, synthesize the directory resolution from the file: depPath (reconstruction of exactly what pnpm's writer emits, not a guess). This makes inheritOrSynthesizeResolution + its 5 reader call sites inert idempotent guards (resolution is always populated before they run); the active normalization now lives solely in the converter. Mechanical removal of the now-dead helper is a trivial follow-up. Bug 2 (lifecycle re-import .bin ENOENT): drop the pnpm#4299 scanDir-into- filesMap workaround and pass keepModulesDir: true so importIndexedDir skips the pnpm#11088 makeEmptyDir fast path and preserves the target's existing node_modules via its staging/move path. Stays on storeController.importPackage so source files keep hardlinks — replaces the copy-loop mirror. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…ipts When `runLifecycleHooksConcurrently` re-imports a workspace package into its injected targets after running prepare/postinstall, it has to preserve the existing `node_modules/` inside each target — set up during the initial install (bin symlinks, transitive deps) — while overlaying the script's output (typically a freshly-built `dist/`). The previous approach was a manual `scanDir` over `<targetDir>/node_modules` that fed absolute paths into the re-import's `filesMap`. That worked in 2022 (pnpm#4299) when `importIndexedDir` always staged-and-renamed, but pnpm#11088 made the fast path (write directly into the final dir) the default. The fast path's `makeEmptyDirSync(targetDir)` wipes the very files the scanned entries point at, so `tryImportIndexedDir`'s first copy throws: ERR_PNPM_ENOENT on <targetDir>/node_modules/.bin/<tool> Reproduces on any workspace package that has a `prepare`/`postinstall` script + a dep with a bin entry, when injected via `injectWorkspacePackages: true` or `dependenciesMeta.<dep>.injected`. Fix: skip the `importPackage` round-trip entirely. After lifecycle scripts complete, iterate the source's `filesMap` (from `fetchFromDir`, which already excludes `node_modules/`) and copy each entry directly into each `targetDir`. The target's existing `node_modules/` is never touched — its bin links + transitive deps stay intact — and there's no tmp-dir staging swap to trip over `.bin/<tool>` symlinks. Regression test in `injectLocalPackages.ts`: an injected workspace package with a `prepublishOnly` script + `@pnpm.e2e/hello-world-js-bin` dep. Before the fix this throws `ERR_PNPM_ENOENT` on `.bin/hello-world-js-bin`; after the fix both the script's output and the bin symlink survive into the injected copy.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
Per the PR review (zkochan); pairs with the turbo-side root-cause fix vercel/turborepo#12819. Bug 1 (peer-variant resolution): normalize once at the read layer in convertToLockfileObject — when a pruned lockfile dropped the base packages entry, synthesize the directory resolution from the file: depPath (reconstruction of exactly what pnpm's writer emits, not a guess). This makes inheritOrSynthesizeResolution + its 5 reader call sites inert idempotent guards (resolution is always populated before they run); the active normalization now lives solely in the converter. Mechanical removal of the now-dead helper is a trivial follow-up. Bug 2 (lifecycle re-import .bin ENOENT): drop the pnpm#4299 scanDir-into- filesMap workaround and pass keepModulesDir: true so importIndexedDir skips the pnpm#11088 makeEmptyDir fast path and preserves the target's existing node_modules via its staging/move path. Stays on storeController.importPackage so source files keep hardlinks — replaces the copy-loop mirror. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zkochan
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 19, 2026
…ipts When `runLifecycleHooksConcurrently` re-imports a workspace package into its injected targets after running prepare/postinstall, it has to preserve the existing `node_modules/` inside each target — set up during the initial install (bin symlinks, transitive deps) — while overlaying the script's output (typically a freshly-built `dist/`). The previous approach was a manual `scanDir` over `<targetDir>/node_modules` that fed absolute paths into the re-import's `filesMap`. That worked in 2022 (pnpm#4299) when `importIndexedDir` always staged-and-renamed, but pnpm#11088 made the fast path (write directly into the final dir) the default. The fast path's `makeEmptyDirSync(targetDir)` wipes the very files the scanned entries point at, so `tryImportIndexedDir`'s first copy throws: ERR_PNPM_ENOENT on <targetDir>/node_modules/.bin/<tool> Reproduces on any workspace package that has a `prepare`/`postinstall` script + a dep with a bin entry, when injected via `injectWorkspacePackages: true` or `dependenciesMeta.<dep>.injected`. Fix: skip the `importPackage` round-trip entirely. After lifecycle scripts complete, iterate the source's `filesMap` (from `fetchFromDir`, which already excludes `node_modules/`) and copy each entry directly into each `targetDir`. The target's existing `node_modules/` is never touched — its bin links + transitive deps stay intact — and there's no tmp-dir staging swap to trip over `.bin/<tool>` symlinks. Regression test in `injectLocalPackages.ts`: an injected workspace package with a `prepublishOnly` script + `@pnpm.e2e/hello-world-js-bin` dep. Before the fix this throws `ERR_PNPM_ENOENT` on `.bin/hello-world-js-bin`; after the fix both the script's output and the bin symlink survive into the injected copy.
zkochan
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 19, 2026
Per the PR review (zkochan); pairs with the turbo-side root-cause fix vercel/turborepo#12819. Bug 1 (peer-variant resolution): normalize once at the read layer in convertToLockfileObject — when a pruned lockfile dropped the base packages entry, synthesize the directory resolution from the file: depPath (reconstruction of exactly what pnpm's writer emits, not a guess). This makes inheritOrSynthesizeResolution + its 5 reader call sites inert idempotent guards (resolution is always populated before they run); the active normalization now lives solely in the converter. Mechanical removal of the now-dead helper is a trivial follow-up. Bug 2 (lifecycle re-import .bin ENOENT): drop the pnpm#4299 scanDir-into- filesMap workaround and pass keepModulesDir: true so importIndexedDir skips the pnpm#11088 makeEmptyDir fast path and preserves the target's existing node_modules via its staging/move path. Stays on storeController.importPackage so source files keep hardlinks — replaces the copy-loop mirror. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zkochan
added a commit
that referenced
this pull request
May 19, 2026
…th + lifecycle re-import (#11662) Fixes two **independent** crashes hitting `pnpm install --frozen-lockfile` on workspaces with `injectWorkspacePackages: true` (or `dependenciesMeta.*.injected`), surfaced via `turbo prune --docker` pipelines. ## Bug 1 — peer-variant snapshot missing `resolution` (lean, defense-in-depth) A peer-variant injected workspace snapshot (`@scope/pkg@file:packages/pkg(peerA@1)(peerB@2)`) inherits its `resolution` from the base `packages:` entry (`@scope/pkg@file:packages/pkg`). When a tool prunes the lockfile and drops that base entry, readers that deref `pkgSnapshot.resolution` crash with the cryptic: ``` Cannot use 'in' operator to search for 'directory' in undefined ``` **The root cause is upstream of pnpm**: the pruner (e.g. `turbo prune`) emits an internally inconsistent lockfile. Fixed at the source in **vercel/turborepo#12825** (retain the base entry for peer-variant injected deps; minimal repro in **vercel/turborepo#12824**) — empirically verified to produce a correct pruned lockfile for a real multi-service workspace. **pnpm side (this PR): one lean normalization at the read layer** — in `convertToLockfileObject`, where base→variant inheritance already happens via `Object.assign`. When the base entry is absent, reconstruct the directory resolution from the `file:` depPath. This is *reconstruction, not guessing*: for a workspace `file:` dep the directory **is** the depPath suffix — exactly what pnpm's own writer emits. It is **defense-in-depth, not load-bearing**: with a well-formed lockfile (turbo#12825 or any correct input) the branch never fires. Because the normalization sits at the single shared read layer, it also covers the sibling `Cannot use 'in' operator … 'integrity' in undefined` on the `pnpm deploy` path (same `resolution === undefined` root, different deref site). Per review feedback: the earlier per-reader `inheritOrSynthesizeResolution` helper across 5 call sites is **removed**; normalization lives in exactly one place (`convertToLockfileObject`), and the readers are back to `main`. ## Bug 2 — lifecycle re-import wipes `.bin/<tool>` (pure pnpm; the substantive fix) `runLifecycleHooksConcurrently` re-imports an injected workspace package into its targets after `prepare`/`postinstall`. The 2022 `scanDir`-into-`filesMap` workaround (#4299) fed target-internal paths to `importPackage`; once #11088 made `importIndexedDir`'s `makeEmptyDir` fast path the default, that path wipes the target's `node_modules` before copying, so the re-import dies with `ERR_PNPM_ENOENT` on `node_modules/.bin/<tool>`. Fix: drop the `scanDir` workaround and pass `keepModulesDir: true` so `importIndexedDir` skips the destructive fast path and preserves the target's existing `node_modules` (bin symlinks + transitive deps) via its staging/move path. Stays on `storeController.importPackage`, so source files keep their **hardlinks** (no copy-loop regression). Net reduction vs `main`: the `scanDir` helper and the `node:fs` / `FilesMap` imports are removed. ## Tests - The `deps-restorer` regression fixture `peer-variant-missing-resolution` **omits the base `packages:` entry**, so it encodes the actual pruned shape and reproduces the crash on `main`: reverting the `convertToLockfileObject` change yields `resolution: undefined` for the peer-variant (→ the `lockfileToDepGraph` crash); with this PR it is reconstructed as `{ type: 'directory', directory: … }`. - A `lockfile.fs` unit test pins the heuristic boundary: a directory resolution is synthesized for a pruned `file:` peer-variant but **never** for a `file:` tarball. - A `deps-installer` regression test covers the Bug 2 re-import (injected dep with a `prepare` script + a bin-having dependency). ## Validation End-to-end on a real `injectWorkspacePackages` monorepo (`turbo prune --docker` → `pnpm install --frozen-lockfile`), on services that crash on **both** bugs with stock pnpm: - pnpm with both fixes: the crashing services build. - **vercel/turborepo#12825 + pnpm with only Bug 2** (Bug 1 fully reverted): the crashing services still **build** → confirms Bug 1 here is genuine defense-in-depth and turbo#12825 owns the root cause. - Bug 2 reproduces on stock pnpm regardless of turbo (it is purely pnpm's importer fast-path). Pairs with **vercel/turborepo#12825** (Bug 1 root cause; minimal repro **vercel/turborepo#12824**). Tracks #11663. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> Co-authored-by: Eyalm321 <eyal@sunsationsusa.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: UApply Developer <developer@uapply.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.