-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Description
Contribution
- I'd be willing to implement this feature (contributing guide)
Describe the user story
As a developer using the global virtual store (GVS), when I upgrade Node.js (e.g., 22 → 24) or switch architectures (e.g., x64 → arm64), every package in links/ is invalidated — even pure-JS packages that don't have native builds. This forces a full re-import of the entire store on every Node.js upgrade.
For tools that pre-build the store externally (Nix, Docker multistage builds, CI caching layers), this means the cached GVS is completely invalidated by a Node.js minor version bump, even though ~95% of packages are platform-independent.
Describe the solution you'd like
Re-add requiresBuild?: true to the lockfile's PackageSnapshot type, and use it in calcGraphNodeHash to conditionally omit ENGINE_NAME from the hash for packages where neither the package nor any transitive dependency requires a build.
Background
calcGraphNodeHash in packages/calc-dep-state/src/index.ts already documents this as a known limitation:
const state = {
// Unfortunately, we need to include the engine name in the hash,
// even though it's only required for packages that are built,
// or have dependencies that are built.
// We can't know for sure whether a package needs to be built
// before it's fetched from the registry.
// However, we fetch and write packages to node_modules in random order
// for performance, so we can't determine at this stage which
// dependencies will be built.
engine: ENGINE_NAME,
deps: calcDepGraphHash(graph, cache, new Set(), depPath),
}The comment is correct: at hash-computation time, requiresBuild is only available in the CAS index files (determined at fetch time), not in the lockfile. But this information is available at lockfile-generation time — pnpm install reads each package's package.json and knows whether it has lifecycle scripts (preinstall, install, postinstall, prepare) or native build indicators like binding.gyp.
What changed since Feb 2024
requiresBuild was intentionally removed from the lockfile in PR #7710 (Feb 2024, feat!: remove requiresBuild from the lockfile). The rationale was correct at the time: the information was derivable at runtime from package contents, so storing it in the lockfile was redundant.
However, GVS (PR #8190, Jul 2025) created a new requirement: the GVS hash must be computed before fetching, using only lockfile data. requiresBuild is now needed at a stage where it's no longer available. The "unfortunately" comment in calcGraphNodeHash was written after the field was removed, acknowledging this gap.
The existing calcDepState function already handles this distinction for the side-effects cache — it uses includeDepGraphHash which is only true when requiresBuild is true. Extending this pattern to calcGraphNodeHash would make GVS hashes engine-agnostic for pure-JS packages.
Proposed changes
- Re-add
requiresBuild?: truetoLockfilePackageInfoinlockfile/types/src/index.ts - Populate it during lockfile generation when the package has lifecycle scripts or native build indicators
- In
calcGraphNodeHash, walk the dependency graph to determine if any transitive dep requires a build. Only includeENGINE_NAMEwhen the package or a transitive dependency hasrequiresBuild. - When
requiresBuildis absent (old lockfiles), fall back to current behavior (always includeENGINE_NAME).
This would make ~95% of GVS paths platform-independent. A Node.js major version bump would only invalidate the ~5% of packages that actually have native builds.
Note: the sort key for requiresBuild still exists in lockfile/fs/src/sortLockfileKeys.ts (position 12) — it was left behind when the field was removed.
Describe the drawbacks of your solution
- Lockfile format addition: Adds an optional field to package snapshots. This is backward-compatible — old lockfiles without the field fall back to current behavior (always include
ENGINE_NAME). - Partially reverses PR feat!: remove requiresBuild from the lockfile #7710: Though the context has changed — GVS didn't exist when the field was removed, and the "unfortunately" comment proves the current state is recognized as suboptimal.
- Transitive walk complexity: Determining whether any transitive dep requires a build adds a graph traversal during hash computation. This can be cached per dep path, similar to how
calcDepGraphHashalready works.
Describe alternatives you've considered
-
Two-pass hash with warm CAS: Read CAS index files (which store
requiresBuildinPackageFilesResponse) before computing hashes. Works for warm-CAS scenarios (Nix, afterpnpm fetch) but breaks for fresh installs where CAS is empty. Also couples hash computation to the CAS format. -
Heuristic detection from lockfile data: Use
hasBin,engines, or package name patterns to guess whether a package needs native builds. Unreliable —hasBinindicates executables (not build scripts), and many packages withhasBinare pure JS. -
Accept the current behavior: Keep
ENGINE_NAMEin all GVS hashes. This works but wastes disk space (duplicatelinks/entries per Node version) and invalidates external caches unnecessarily on Node.js upgrades.