Skip to content
This repository was archived by the owner on May 14, 2026. It is now read-only.
This repository was archived by the owner on May 14, 2026. It is now read-only.

Partial install with --frozen-lockfile: read+write node_modules/.pnpm/lock.yaml and skip unchanged snapshots #433

@zkochan

Description

@zkochan

Background

Pacquet's frozen-lockfile installer re-fetches and re-links every snapshot on every run, even when node_modules already has the right content. Upstream pnpm avoids this by writing a current lockfile at <project>/node_modules/.pnpm/lock.yaml after each install: it records what was actually materialized, and the next install diffs it against pnpm-lock.yaml (the wanted lockfile) to skip snapshots whose resolution + dependencies + optionalDependencies are unchanged and whose directory still exists on disk. This issue tracks both halves of that loop — they're two Roadmap (#299) checkboxes that only make sense together.

A correct partial-install path is the difference between a warm pacquet install --frozen-lockfile that finishes in well under a second and one that re-walks every snapshot in the graph. It is also load-bearing for the alot7 warm-install perf gap I've been investigating.

How upstream does it

References at pnpm v11 94240bc046:

  • Read. readCurrentLockfile(internalPnpmDir, ...) reads <internalPnpmDir>/lock.yaml. internalPnpmDir is <rootModulesDir>/.pnpm (installing/deps-restorer/src/index.ts:226), so the file lives at node_modules/.pnpm/lock.yaml regardless of where the wanted lockfile sits (workspace root vs. project root).

  • Diff at the snapshot level. lockfileToDepGraph decides per depPath whether to skip the fetch + import:

    const depIsPresent = !isDirectoryDep &&
      currentPackages[depPath] &&
      equals(currentPackages[depPath].dependencies, pkgSnapshot.dependencies)
    
    const depIntegrityIsUnchanged = isIntegrityEqual(pkgSnapshot.resolution, currentPackages[depPath]?.resolution)

    If both hold and the optional-deps map is empty on both sides, the existing directory at <virtualStoreDir>/<dir-in-virtual-store>/node_modules/<pkgName> is taken at face value: no fetch, no extract, no relink for that snapshot. A missing directory triggers a _broken_node_modules debug log and the snapshot is re-installed. Under GVS there's an additional .pnpm-needs-build marker check (lockfileToDepGraph.ts:246) — out of scope here unless GVS lands first (Add global virtual store support to pacquet install --frozen-lockfile #432).

  • Write. writeCurrentLockfile(virtualStoreDir, lockfile) is called from installing/deps-installer/src/install/index.ts:1597 at end-of-install. It rewrites <virtualStoreDir>/lock.yaml atomically (writeFileAtomic) with the filtered lockfile — i.e. the wanted lockfile narrowed to the importers and engine that were actually installed. Empty lockfiles are deleted rather than written.

  • Reporter signal. pnpm:context carries currentLockfileExists: boolean (installing/context/src/index.ts). The flag matters to the reporter; pacquet currently hard-codes it to false at crates/package-manager/src/install.rs:128-132, with a TODO that lines up with this work.

Today

Pacquet:

  • Never reads node_modules/.pnpm/lock.yaml. The current_lockfile_exists field is hard-coded false at crates/package-manager/src/install.rs:132, and the TODO at install.rs:126-128 already calls this work out.
  • Never writes that file. crates/lockfile/src/save_lockfile.rs:30 writes pnpm-lock.yaml only. There is no writeCurrentLockfile-equivalent.
  • Has a load-bearing TODO at crates/package-manager/src/install_package_by_snapshot.rs:118 (TODO: skip when already exists in store?) confirming the per-snapshot skip is not implemented. As a result, every snapshot goes through the full install path on a warm reinstall.
  • Has no concept of "current packages" to diff against — create_virtual_store.rs builds the virtual store unconditionally from the wanted lockfile.

Plan

A. Read the current lockfile

  • Add a read_current_lockfile(internal_pnpm_dir: &Path) -> io::Result<Option<Lockfile>> helper in crates/lockfile, mirroring readCurrentLockfile. internal_pnpm_dir is <root_modules_dir>/.pnpm. Returns Ok(None) on ENOENT. Use the same lockfile-version checks as the wanted-lockfile path.
  • Wire it into Install::run so current_lockfile_exists on pnpm:context reflects reality. Drop the TODO.

B. Per-snapshot skip

  • In crates/package-manager/src/install_package_by_snapshot.rs (today's "always install" path), accept a borrowed current_packages: Option<&HashMap<DepPath, PackageSnapshot>> and, for the current dep_path, evaluate three conditions before any fetch:

    • current.dependencies == wanted.dependencies (structural eq, both empty-equivalent).
    • current.optional_dependencies == wanted.optional_dependencies (or both empty).
    • isIntegrityEqual(current.resolution, wanted.resolution) — port the helper from lockfileToDepGraph.ts:366. It compares integrity field on tarball resolutions and handles directory-resolution edge cases.

    When all three hold and the destination dir under the virtual store exists, skip fetch + extract + symlink for that snapshot.

  • When the directory is missing despite the cache key matching, log the equivalent of pnpm's _broken_node_modules event (debug-level) and fall through to the full install path. Use a fresh log emit consistent with CODE_STYLE_GUIDE.md's reporter conventions.

  • Drop the TODO at install_package_by_snapshot.rs:118.

C. Write the current lockfile at end-of-install

  • Add a write_current_lockfile(virtual_store_dir: &Path, lockfile: &Lockfile) in crates/lockfile, mirroring writeCurrentLockfile. Writes to <virtual_store_dir>/lock.yaml atomically (write-temp + rename, matching write-file-atomic's behavior); deletes the file when the lockfile is empty.
  • Call it from Install::run after the install succeeds, before the pnpm:summary emit (matching upstream ordering at installing/deps-installer/src/install/index.ts:1593-1600). For now pass the wanted lockfile through unmodified — once workspace install (Add workspace support to pacquet install --frozen-lockfile #431) lands, narrow to the filtered lockfile (selected importers + engine filter). Capture the filtered-write follow-up here as a checkbox.
  • On a partial install where some snapshots were skipped, the current-lockfile write still records the full materialized graph (i.e. the wanted lockfile narrowed by importer/engine filters, not narrowed by what was newly fetched). This is the upstream behavior — result.currentLockfile at the write site is filteredLockfile, not "freshly installed only."

D. Reporter

  • Plumb the read result into ContextLog::current_lockfile_exists. Remove the hard-coded false and the TODO comment at crates/package-manager/src/install.rs:126-132.
  • Confirm pnpm:stats added count reflects only the newly-installed snapshots, not the skipped ones. Mirrors installing/deps-restorer/src/index.ts:397-400.

E. Tests

  • Cold install followed by an immediate warm reinstall: assert that the second install fetches zero tarballs (e.g. via the existing mock-registry hook-count or a counter on the registry client). Today this assertion would fail.
  • Warm reinstall after deleting one package directory under the virtual store: only the missing package gets fetched/extracted; _broken_node_modules-equivalent debug log is emitted.
  • Warm reinstall after editing pnpm-lock.yaml to bump one package's integrity: only that snapshot is re-fetched.
  • Warm reinstall after dropping a dependency from a project: the removed dep is not re-installed, surplus directory cleanup is not in scope (it's a separate prune story) — verify the rest of the install still skips cleanly.
  • node_modules/.pnpm/lock.yaml round-trips: read after first install equals what was written. Empty lockfile case: the file is deleted, not written.
  • pnpm:context.current_lockfile_exists is false on first install and true on the second.

Interactions / out of scope

  • GVS (Add global virtual store support to pacquet install --frozen-lockfile #432). When GVS is on, packages live at <storeDir>/links/..., the per-project hoist dir lives at <project>/node_modules/.pnpm/node_modules, and the current lockfile still lives at <project>/node_modules/.pnpm/lock.yaml. The skip logic gains a .pnpm-needs-build marker check (lockfileToDepGraph.ts:246-256). Track the GVS-specific bits there, not here — but design the per-snapshot skip so adding the marker check is a small extension, not a rewrite.
  • Workspace install (Add workspace support to pacquet install --frozen-lockfile #431). Once it lands, result.currentLockfile becomes the filtered lockfile (selected importers × engine filter). Until then, write the wanted lockfile unfiltered — there's only one importer to filter to.
  • Surplus / stale package cleanup. Removing packages that exist in node_modules/.pnpm but no longer appear in the wanted lockfile is upstream's pruneVirtualStore (installing/deps-installer/src/install/index.ts:362). Out of scope.
  • Lockfile compatibility versions. readCurrentLockfile uses ignoreIncompatible: false at the deps-restorer call site, so a version mismatch surfaces as an error. Match that behavior; pacquet already asserts v9 in install.rs:172.

Related


Written by an agent (Claude Code, claude-opus-4-7).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions