Skip to content

fix: stop install from recreating node_modules after pnpm fetch#11490

Merged
zkochan merged 4 commits into
mainfrom
fix/11488
May 6, 2026
Merged

fix: stop install from recreating node_modules after pnpm fetch#11490
zkochan merged 4 commits into
mainfrom
fix/11488

Conversation

@zkochan

@zkochan zkochan commented May 6, 2026

Copy link
Copy Markdown
Member

Summary

Closes #11488.

pnpm fetch writes forced-empty hoistPattern: [] and publicHoistPattern: [] into .modules.yaml (because its virtualStoreOnly install path skips hoisting). In v10 the follow-up pnpm install ignored these unless the user had explicitly set a hoist-pattern in their config. v11's #11199 removed that explicit-config gate, so validateModules now always sees the empty patterns as a hoist-pattern change and purges node_modules — slow on every CI run, and per the bug report sometimes leaves the modules dir in an ERR_MODULE_NOT_FOUND state on subsequent runs.

The fix marks .modules.yaml with a new virtualStoreOnly: true field after a fetch. validateModules recognizes this flag as "incomplete install state" and skips the PUBLIC_HOIST_PATTERN_DIFF / HOIST_PATTERN_DIFF comparisons. The next install then completes the missing post-import linking in place rather than purging. The flag is dropped from .modules.yaml once a normal install runs.

A genuine hoist-pattern change (without a fetch in between) still triggers the purge as before — verified manually with publicHoistPattern in pnpm-workspace.yaml.

Test plan

  • Added regression test install after fetch does not recreate node_modules in installing/commands/test/fetch.ts — asserts node_modules/.pnpm's inode is preserved across fetch→install and the package symlink exists afterwards.
  • All 7 fetch tests pass.
  • All 5 modules-yaml tests pass.
  • All 28 hoist tests + 2 modulesCache tests in deps-installer pass.
  • Manually reproduced the demo from the issue (fetch --prod + install --offline --prod --frozen-lockfile) against the rebuilt bundle: no more Recreating /private/tmp/.../node_modules, pnpm start works, modules dir is preserved.
  • Manually verified the negative case: changing publicHoistPattern in pnpm-workspace.yaml between two installs (no fetch involved) still triggers the purge.

Written by an agent (Claude Code, claude-opus-4-7).

Summary by CodeRabbit

  • Bug Fixes

    • Fetch no longer forces node_modules to be recreated: it records a virtual-store-only state so subsequent installs skip unnecessary hoist-pattern checks and complete linking.
  • New Features

    • Install command now accepts hoistPattern and publicHoistPattern options.
  • Tests

    • Added a regression test ensuring fetch followed by install preserves node_modules layout and linking.

`pnpm fetch` writes empty `hoistPattern`/`publicHoistPattern` into
.modules.yaml because the virtualStoreOnly install path skips hoisting.
v11 removed the explicit-config gate in `validateModules`, so the
follow-up `pnpm install` saw those forced-empty values as a hoist-pattern
change and purged node_modules.

Mark .modules.yaml with `virtualStoreOnly: true` after a fetch so the
next install can recognize the incomplete state, skip the hoist-pattern
comparison, and complete the missing post-import linking in place.
A real hoist-pattern change still triggers the purge as before.

Closes #11488
Copilot AI review requested due to automatic review settings May 6, 2026 11:35
@coderabbitai

coderabbitai Bot commented May 6, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e2a59337-e81d-4db0-904e-e8e05b7f6b2b

📥 Commits

Reviewing files that changed from the base of the PR and between 325fe7d and 54ee7bc.

📒 Files selected for processing (2)
  • installing/commands/test/fetch.ts
  • installing/deps-installer/src/install/index.ts

📝 Walkthrough

Walkthrough

Fetch now records a virtualStoreOnly flag in the modules manifest; manifest writing and validation logic use this flag so a subsequent install skips hoist-pattern comparisons and does not recreate node_modules. Adds a regression test and changeset referencing issue #11488.

Changes

Virtual Store-Only Flag & Fetch→Install Flow

Layer / File(s) Summary
Data Shape
installing/modules-yaml/src/index.ts
Added virtualStoreOnly?: boolean to ModulesRaw and only persist it when truthy.
Manifest Emission
installing/deps-restorer/src/index.ts, installing/deps-installer/src/install/index.ts
Pass virtualStoreOnly into writeModulesManifest from headless/restorer and installer paths.
Core Validation
installing/deps-installer/src/install/validateModules.ts
Skip PUBLIC_HOIST_PATTERN_DIFF and HOIST_PATTERN_DIFF checks when modules.virtualStoreOnly is true; rootProject hoist check guarded similarly.
CLI Surface
installing/commands/src/install.ts
Added hoistPattern and publicHoistPattern to InstallCommandOptions pick.
Regression Test
installing/commands/test/fetch.ts
New test "install after fetch completes linking without recreating node_modules" verifying inode stability, importer symlink, and hoisting after fetch + install.
Changelog
.changeset/fetch-install-no-recreate.md
New changeset documenting the fix and listing affected packages; references issue #11488.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Dev as Developer
  participant Fetch as pnpm fetch
  participant Manifest as modules.yaml
  participant Install as pnpm install
  participant Validator as validateModules

  Dev->>Fetch: run `pnpm fetch`
  Fetch->>Manifest: write modules manifest with `virtualStoreOnly: true` (rgba(30,144,255,0.5))
  Dev->>Install: run `pnpm install --frozen-lockfile`
  Install->>Manifest: read modules manifest
  Install->>Validator: pass modules.virtualStoreOnly flag
  Validator-->>Install: skip hoist-pattern diff checks when virtualStoreOnly is true (rgba(50,205,50,0.5))
  Install->>Dev: complete install without recreating node_modules
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • pnpm/pnpm#11452: Modifies InstallCommandOptions in the same install.ts file (adds different config fields), so related at the type/CLI surface level.

"I thumped my foot on the store,
wrote a flag to fix the chore,
fetch whispers 'virtual only' true,
install nods and skips the dew,
node_modules stays steady—hooray for more!" 🐇✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately summarizes the main fix: preventing node_modules recreation after pnpm fetch, which directly addresses the core issue and bug being resolved.
Linked Issues check ✅ Passed All code changes fully implement the fix for issue #11488: virtualStoreOnly flag prevents hoist-pattern diff checks during incomplete installs, regression test validates inode preservation and correct behavior.
Out of Scope Changes check ✅ Passed All changes are directly related to the virtualStoreOnly fix for issue #11488; no unrelated modifications or scope creep detected in the changeset.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/11488

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
installing/commands/test/fetch.ts (1)

233-277: ⚡ Quick win

Add an assertion that virtualStoreOnly is cleared after install.

This regression test is strong; asserting manifest cleanup would fully cover the intended state transition (fetch marks it, install removes it).

Proposed test extension
   expect(fs.statSync(virtualStoreDir).ino).toBe(virtualStoreInodeBefore)
   // The package symlink must be present after install completes the linking.
   expect(fs.existsSync(path.resolve(project.dir(), 'node_modules/is-positive'))).toBeTruthy()
+  const modulesManifest = JSON.parse(
+    fs.readFileSync(path.resolve(project.dir(), 'node_modules/.modules.yaml'), 'utf8')
+  )
+  expect(modulesManifest.virtualStoreOnly).toBeUndefined()
 })
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@installing/commands/test/fetch.ts` around lines 233 - 277, Extend the test
'install after fetch does not recreate node_modules' to assert that the
transient fetch marker is removed: after the second install.handler call, read
the modules metadata that fetch sets (the .modules.yaml / virtual store manifest
associated with virtualStoreDir) and assert that the virtualStoreOnly flag is no
longer present or is false; use the existing virtualStoreDir and project
variables to locate the manifest and add an expect(...) assertion verifying
virtualStoreOnly is cleared.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@installing/commands/test/fetch.ts`:
- Around line 233-277: Extend the test 'install after fetch does not recreate
node_modules' to assert that the transient fetch marker is removed: after the
second install.handler call, read the modules metadata that fetch sets (the
.modules.yaml / virtual store manifest associated with virtualStoreDir) and
assert that the virtualStoreOnly flag is no longer present or is false; use the
existing virtualStoreDir and project variables to locate the manifest and add an
expect(...) assertion verifying virtualStoreOnly is cleared.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c1856d89-9fe8-4069-8f1f-1af97edcc2ac

📥 Commits

Reviewing files that changed from the base of the PR and between 60fd205 and 597ddf7.

📒 Files selected for processing (5)
  • .changeset/fetch-install-no-recreate.md
  • installing/commands/test/fetch.ts
  • installing/deps-installer/src/install/validateModules.ts
  • installing/deps-restorer/src/index.ts
  • installing/modules-yaml/src/index.ts

The previous test only verified that node_modules wasn't purged and the
direct dep was symlinked. Strengthen it with @pnpm.e2e/pkg-with-1-dep so
we also assert the transitive dep ends up under .pnpm/node_modules — i.e.
the install actually completes the post-import linking that fetch skips,
not just leaves the modules dir in place.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a pnpm v11 regression where running pnpm install after pnpm fetch unnecessarily purges/recreates node_modules due to .modules.yaml containing forced-empty hoistPattern / publicHoistPattern values from the fetch’s virtualStoreOnly install path.

Changes:

  • Persist a new .modules.yaml flag (virtualStoreOnly: true) for fetch/virtual-store-only installs and avoid comparing hoist patterns during the next install’s validateModules step.
  • Propagate virtualStoreOnly into the modules manifest writer in the headless install path and ensure the field is omitted when not set.
  • Add a regression test to assert node_modules/.pnpm is not recreated across a fetch → install sequence.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
installing/modules-yaml/src/index.ts Adds the virtualStoreOnly field to the modules manifest type and strips it on write when not enabled.
installing/deps-restorer/src/index.ts Passes virtualStoreOnly through to writeModulesManifest() in the headless install writer.
installing/deps-installer/src/install/validateModules.ts Skips hoist/public-hoist pattern mismatch checks when .modules.yaml is marked virtualStoreOnly.
installing/commands/test/fetch.ts Adds regression test ensuring install after fetch preserves the virtual store directory and links dependencies.
.changeset/fetch-install-no-recreate.md Documents the fix and bumps affected packages.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 676 to 680
virtualStoreDir,
virtualStoreDirMaxLength: opts.virtualStoreDirMaxLength,
allowBuilds: opts.allowBuilds,
virtualStoreOnly: opts.virtualStoreOnly,
})

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — confirmed and fixed in 54ee7bc.

installing/deps-installer/src/install/index.ts:1593 (the non-headless write path) was indeed propagating virtualStoreOnly: true from ctx.modulesFile. Reproduced manually: after pnpm fetch followed by pnpm install --no-frozen-lockfile (which forces resolution and exits the headless path), .modules.yaml still carried the marker. Now overridden explicitly with opts.virtualStoreOnly, which is undefined for normal installs — so writeModulesManifest's falsy-strip drops it from the file.

building/after-install/src/index.ts doesn't need the same change: it re-reads .modules.yaml via getContext after deps-installer has already cleared the flag, so the spread sees the cleared value. Verified by running the headless and non-headless install paths after fetch — both end with no virtualStoreOnly in .modules.yaml. Test extended to assert the clear-on-install behavior.


Written by an agent (Claude Code, claude-opus-4-7).

zkochan added 2 commits May 6, 2026 13:45
Both keys are already declared in install's rcOptionsTypes but were
missing from the InstallCommandOptions Pick, so callers that build the
option object programmatically (rather than via the Config loader) had
no typed way to set them.
The non-headless install path in deps-installer writes .modules.yaml by
spreading ctx.modulesFile, which carried the virtualStoreOnly: true flag
forward when the previous step was a fetch. Future installs would then
keep skipping hoist-pattern checks even though the install was complete.

Override the field explicitly with opts.virtualStoreOnly so the marker
is cleared whenever a regular install runs after fetch. Also assert the
clear-on-install behavior in the regression test.
Copilot AI review requested due to automatic review settings May 6, 2026 11:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

@zkochan zkochan merged commit 832d898 into main May 6, 2026
17 checks passed
@zkochan zkochan deleted the fix/11488 branch May 6, 2026 12:39
zkochan added a commit that referenced this pull request May 6, 2026
Closes #11488.

`pnpm fetch` writes forced-empty `hoistPattern: []` and `publicHoistPattern: []` into `.modules.yaml` (because its `virtualStoreOnly` install path skips hoisting). In v10 the follow-up `pnpm install` ignored these unless the user had explicitly set a hoist-pattern in their config. v11's [#11199](#11199) removed that explicit-config gate, so `validateModules` now always sees the empty patterns as a hoist-pattern change and purges `node_modules` — slow on every CI run, and per the bug report sometimes leaves the modules dir in an `ERR_MODULE_NOT_FOUND` state on subsequent runs.

The fix marks `.modules.yaml` with a new `virtualStoreOnly: true` field after a fetch. `validateModules` recognizes this flag as "incomplete install state" and skips the `PUBLIC_HOIST_PATTERN_DIFF` / `HOIST_PATTERN_DIFF` comparisons. The next install then completes the missing post-import linking in place rather than purging. The flag is dropped from `.modules.yaml` once a normal install runs.

A genuine hoist-pattern change (without a fetch in between) still triggers the purge as before — verified manually with `publicHoistPattern` in `pnpm-workspace.yaml`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v11 fetch followed by install recreates node_modules

2 participants