Update is-ci to version 1.0.10 🚀#438
Closed
greenkeeperio-bot wants to merge 1 commit into
Closed
Conversation
2bd0a23 to
057e97e
Compare
057e97e to
65d0500
Compare
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…pm#438 slice 1) (pnpm#441) Extends pacquet's package-import primitive to cover pnpm's `importIndexedDir(..., { force, keepModulesDir })` call shape so the hoisted node-linker (the umbrella of pnpm#438) has a primitive to build on. - Renames `create_cas_files` → `import_indexed_dir` to match pnpm's name (its docstring already said it mirrored `tryImportIndexedDir`), and adds an `ImportIndexedDirOpts` struct with `force` and `keep_modules_dir`. Both linkers now go through the same function — pnpm uses one function for both, so does pacquet. - **Default opts** (`force: false, keep_modules_dir: false`): the isolated linker's behavior. Existing target short-circuits; fresh target gets `mkdir` + parallel `link_file`. Unchanged hot path for the two existing call sites (`create_virtual_dir_by_snapshot`, `install_package_from_registry`). - **`force: true`**: stage the new contents in a sibling directory, remove the old tree, rename stage into place. A regular file or symlink occupying the target is unlinked first. - **`force: true` + `keep_modules_dir: true`**: before the swap, `dir_path/node_modules/` is moved into the staging directory so nested deps survive the rebuild. On any failure after the move, the staged copy is restored to `dir_path/node_modules/` before staging is cleaned up — staging never holds the user's only copy of nested deps. This is the call shape the hoisted linker will use. - No install-pipeline wiring for hoisted yet. Subsequent slices (`linkHoistedModules` analog, hoist algorithm, `Install::run` branch on `node_linker`) will pass `force + keep_modules_dir` from the new hoisted path. The default-opts behavior is exercised in production today via the existing isolated callers. ## Upstream reference Ported from `pnpm/pnpm@94240bc`: - [`fs/indexed-pkg-importer/src/importIndexedDir.ts`](https://github.com/pnpm/pnpm/blob/94240bc0464196bd52f7006b97f6d9a43df34633/fs/indexed-pkg-importer/src/importIndexedDir.ts) — the function this ports. - [`store/controller-types/src/index.ts`](https://github.com/pnpm/pnpm/blob/94240bc0464196bd52f7006b97f6d9a43df34633/store/controller-types/src/index.ts) — the `ImportOptions` shape `ImportIndexedDirOpts` mirrors. - [`installing/deps-restorer/src/linkHoistedModules.ts:134`](https://github.com/pnpm/pnpm/blob/94240bc0464196bd52f7006b97f6d9a43df34633/installing/deps-restorer/src/linkHoistedModules.ts#L134) — the call site that will drive the `force + keepModulesDir` combination once the later slices land. ## Scope omissions (intentional) - `moveOrMergeModulesDirs` semantics for the case where the indexed file map itself contains `node_modules/` entries. Upstream merges; pacquet errors with `NodeModulesCollision`. The hoisted-linker call site never produces that state in practice (npm and pnpm strip `node_modules/` at pack time), so erroring loudly until a real caller demands the merge is the right call. - No `safeToSkip` short-circuit. Upstream's pre-existence check beyond the default short-circuit lives in `index.ts` around the importer; that decision belongs at the Slice 5 call site, not in this primitive. ## Performance The isolated-linker hot path's syscall count is unchanged. The old `if dir_path.exists() { return Ok(()); }` (one `stat`) becomes `fs::symlink_metadata(dir_path)` (one `lstat`) plus a match dispatch; `populate_dir` is the old `create_cas_files` body verbatim and gets inlined under release LTO. The new `force` branches only run for hoisted, which has no callers yet. One behavior diff: a dangling symlink at the target under default opts used to fall through to `mkdir` and surface an IO error; under the new code it short-circuits as "already populated". The virtual store doesn't produce dangling symlinks under normal operation, so this is theoretical.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…npm#438 slice 3a) (pnpm#448) * feat(real-hoist): crate skeleton + lockfile-to-HoisterTree wrapper (pnpm#438 slice 3a) First sub-slice of the Slice 3 hoister port from the umbrella. Lands the IO surface and the pnpm-side wrapper; the inner `@yarnpkg/nm` algorithm is stubbed (returns the input tree unchanged) and replaced in 3b. New crate `pacquet-real-hoist`: - `HoisterTree` / `HoisterResult` / `HoisterDependencyKind` / `HoistingLimits` / `HoistOpts` / `HoistError`, mirroring upstream's `@yarnpkg/nm` types and the pnpm wrapper's option object. - `RcByPtr<T>` wrapper providing identity-hashed `Rc<T>` so children stored in `IndexSet` keep JS `Set<HoisterTree>` semantics (a node added via two parent paths stays shared) without paying the cost of structural hashing. - `hoist(&Lockfile, &HoistOpts)` — ports the wrapper at `installing/linking/real-hoist/src/index.ts`. Builds the HoisterTree from the lockfile (root importer's dependencies/devDependencies/optionalDependencies merged, then recursive descent through `snapshots`), adds workspace importer children, plus `link:` placeholders for `externalDependencies`, runs the inner stub, and post-filters `externalDependencies` out of the top-level result. - `LockfileMissingDependency` error surfaced as a `miette` diagnostic with code `ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY`, matching the upstream error code at https://pnpm.io/errors. Five unit tests pin the wrapper's observable behaviour: - The "broken lockfile" error case is a direct port of pnpm's `installing/linking/real-hoist/test/index.ts` test. - An empty lockfile yields an empty root. - A `root → a → b` lockfile, under the stub, gives `root → a → b` (nothing hoisted). Pinning this means 3b's algorithm replacement is observably correct — the tree shape will change to `root → {a, b}` once real hoisting lands. - A diamond dep (`root → {a, c} → b`) keeps `b` as one identity through both parents (proves the wrapper's dedup cache). - `external_dependencies` are stripped from the result. Upstream references pin `pnpm/pnpm@94240bc` and `yarnpkg/berry@4287909fa6`. * docs(real-hoist): drop slice/umbrella scaffolding from code comments Slice and sub-slice numbering is PR-organisation scaffolding — it's useful in commit messages and PR descriptions but it rots inside committed code the moment the referenced slices land, get renumbered, or are abandoned. Rewrite the relevant doc-comments to describe what the code *is* today rather than where it sits in a multi-PR sequence: - Top-level module doc drops the "This is Sub-slice 3a of pnpm#438" intro. - `hoist` doc drops the "Sub-slice 3a pins / later sub-slices replace" framing. - `nm_hoist` stub doc drops "Sub-slice 3b replaces this". - `external_dependencies` / non-root importer comments drop their "umbrella's Slice 10 / Slice 9" pointers. - One test doc rewords "Sub-slice 3b's algorithm replacement is observably correct" into a description of what the test pins without naming the future slice. * fix(real-hoist): aliased snapshot lookup, cycle-safe rc identity, non_exhaustive HoistError (pnpm#448 review) Address four Copilot review findings on the new crate: 1. `collect_snapshot_deps` looked up `SnapshotDepRef::Alias` deps under the alias name instead of the resolved target name. The snapshot key for an npm-alias is `<target>@<ver>`, not `<alias>@<ver>`, so any real aliased transitive would have surfaced as `LockfileMissingDependency`. `build_dep_node` now takes the resolved `&PkgNameVerPeer` directly and the caller builds it via `dep_ref.resolve(alias)` for snapshot deps or `PkgNameVerPeer::new(alias, spec.version)` for importer deps. 2. `build_dep_node`'s cycle handling re-allocated the node on the way out (placeholder Rc, recurse, *new* finished Rc) which left any cycle-visiting Rc pointing at the empty placeholder. `HoisterTree::dependencies` is now a `RefCell<IndexSet<...>>` and the construction populates the cell in place — the placeholder and the populated node are the same allocation, so a back-edge reads the eventually-populated set. Same JS `Set<HoisterTree>` mutation semantics the real hoist algorithm needs. 3. `convert` had the same bug and gets the same fix on `HoisterResult::dependencies` and `HoisterResult::references`. The stub's traversal now collects the input children into a Vec before recursing so the borrow on the input cell drops, and populates the output cell in place. 4. `HoistError` is now `#[non_exhaustive]`, matching the rest of pacquet's public error enums. A new regression test (`transitive_npm_alias_resolves_target_snapshot`) pins the fix for (1): the snapshot key the wrapper looks up matches the target package, the node's exposed `name` stays the alias, and `ident_name` / `reference` carry the resolved target's identity. Existing tests updated to take `Ref<'_, …>` via `.borrow()`. All six pass; `just ready`, `cargo doc --document-private-items`, `taplo`, and `just dylint` clean.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
## Summary Replaces the stub `nm_hoist` (landed in pnpm#448) with a working hoister and the surrounding guardrails: BFS over the result graph pulling every eligible descendant up to the root, plus upfront refusal of inputs the algorithm doesn't yet model. ### What the algorithm models - **Free hoist.** A transitive dep with no name collision at the root surfaces at the root. - **Identity dedup.** A dep reachable through multiple parents shares one `Rc` thanks to the wrapper's cache; the hoist preserves that identity and strips the duplicate reference at the other parent path. - **Parent-wins on version conflict.** When two distinct deps share an alias but resolve to different snapshot keys, the first BFS visitor takes the root slot and the other stays under its declaring parent. Visit order matches the wrapper's alias-sorted insertion order, so the outcome is deterministic. - **Deep chains flatten in one pass.** `root → a → b → c → d` becomes `root → {a, b, c, d}` — each node, once hoisted, is queued for further descent; its own children evaluate against root's slots rather than against the (now-empty) intermediate parent. - **O(1) root-slot lookup.** A side `HashMap<String, RcByPtr<HoisterResult>>` mirrors root's direct deps, so the per-edge "is this name taken?" check doesn't degrade to O(N²) `IndexSet` scans on flat graphs. ### Fail-fast guards The algorithm doesn't yet model peers, hoisting limits, or multi-importer (workspace) hoist trees. Rather than emit a layout pnpm would reject, the wrapper refuses these inputs upfront with three new `HoistError` variants: - `UnsupportedPeerDependency { ident, peers }` — fires when scanning the constructed `HoisterTree` finds any node with non-empty `peer_names` (either `peerDependencies` from the `packages:` map or `transitive_peer_dependencies` on a `snapshots:` entry). - `UnsupportedHoistingLimits { len }` — non-empty `opts.hoisting_limits`. - `UnsupportedWorkspace { extra_importers }` — any importer beyond the root `.`. Each carries enough context for an operator to identify what triggered it. The `UnsupportedWorkspace` help points at `SymlinkDirectDependencies` — workspaces *do* work in pacquet's wider install path (workspace support landed in pnpm#443 for the isolated linker); only the hoister is restricted. ### Rebase pickup `collect_importer_deps` carries the `Link`-variant skip introduced by pnpm#443 (workspace support) — `spec.version.as_regular()` extracts the snapshot key for `Regular` deps and `continue`s past `Link` deps, since workspace siblings are direct symlinks materialised by `SymlinkDirectDependencies` and have no snapshot to hoist. ### Performance Nothing in this PR is reachable from `pacquet install` today (`crates/package-manager/src/install*.rs` doesn't import `pacquet-real-hoist` — the crate is dead code from the install pipeline's perspective until the hoisted-linker wiring lands in later slices). The benchmark results bear that out: cold Frozen Lockfile is within CI variance (+3.2% mean, +3.5% median, driven by one outlier), Hot Cache is 6% *faster* (same `stat → lstat` improvement that shipped in slice 1's `import_indexed_dir`), micro is identical. No code path explains a real regression. ## What's deferred - Peer-dependency-aware hoisting (`peer_names` constraints, peer-promise satisfaction). - Multi-round convergence — the BFS handles deep chains in one pass, so the cases requiring true multi-round are limited to peer interactions. - `hoistingLimits` runtime enforcement. - `dependencyKind` distinctions for workspaces and external soft links (today only `ExternalSoftLink` placeholders are added by the wrapper and stripped post-hoist; `Workspace`-kind nodes are blocked by the `UnsupportedWorkspace` guard upfront). ## Upstream reference - [`hoist.ts` algorithm overview](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L1-L51) — the recipe pacquet's single-pass BFS approximates. - [`hoistTo` main loop](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L329) — the structural intent the port mirrors for the subset above.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…npm#461) ## Summary Lifts the previous `UnsupportedPeerDependency` upfront guard and replaces it with the real peer check from upstream's `getNodeHoistInfo`. The hoister now accepts lockfiles whose packages declare peer dependencies and produces the layout pnpm would: a peer-constrained dep only floats up to the root when doing so doesn't change which package its peers resolve to. ## How it works - `HoisterResult` carries the input `peer_names` set forward. Upstream's `HoisterResult` doesn't (peer info lives on its intermediate `HoisterWorkTree`); pacquet runs the algorithm on `HoisterResult` directly, so the peer set rides along. - The hoist driver is a recursive `hoist_subtree` that walks the result graph depth-first. Each recursion receives the candidate parent's *current* ancestor path (`Vec<Rc<HoisterResult>>` exclusive of the parent). After a child's hoist decision is applied, the path passed into the child's recursion reflects its *new* position — a freshly-hoisted child recurses with `[root]`, a child that stayed nested recurses with `parent_path + [parent]`. (An earlier version of this PR used BFS with paths captured at queue time; that path went stale whenever a node was hoisted between being queued and dequeued, leading to over-refused hoists. The recursive form was Copilot's recommendation in code review; the bug regression is pinned by `peer_check_uses_post_hoist_ancestor_path_not_queue_time_path`.) - `would_shadow_peer` walks that path to decide whether a candidate hoist would change its own peer resolution. - New `AbsorbDecision::PeerShadow` variant alongside `Free` / `SameNode` / `Conflict`. Fires when: 1. Root declares the candidate's own name as a peer (root-shadow guard — vacuous today since the wrapper's `.` root has empty `peer_names`, kept for parity). 2. For each peer name `P` the candidate declares, the closest ancestor providing `P` does so with a different ident than the root's `P`. Promoting the candidate would silently re-resolve the peer. - The previous `UnsupportedPeerDependency` error variant and `find_first_peer_constrained` upfront scan are gone. The `#[non_exhaustive]` tag stays so future variants can be added without breaking callers. ## DAG-vs-tree caveat Upstream's `cloneTree` duplicates the work tree into a strict tree per parent path, so each candidate visit has a unique ancestor chain. Pacquet preserves the DAG (its identity-dedup is a feature) and records only the path DFS actually used to reach the candidate. In the rare case where the same `Rc` is reachable through both a peer-compatible and a peer-shadowing path, pacquet refuses to hoist — the layout ends up at most over-nested, never under-nested. The trade-off is documented on `would_shadow_peer`. ## Upstream reference - [`hoist.ts:414`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L414) — root-shadow guard. - [`hoist.ts:454-479`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L454-L479) — ancestor-path peer check. - [`hoist.ts:670`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L670) — `cloneTree`, the per-path-tree shape pacquet skips. ## Test plan - [x] `peer_constrained_node_stays_under_parent_when_root_provides_different_ident` — `app → widget (peer: react) + react@17`, `root → react@18`. widget cannot hoist because hoisting would change its peer to react@18. - [x] `peer_constrained_node_hoists_when_ancestor_and_root_agree` — same shape but `app → react@18` matches root's `react@18` (shared `Rc` via wrapper dedup), so widget hoists freely. - [x] `peer_check_uses_post_hoist_ancestor_path_not_queue_time_path` — regression for the BFS-stale-path bug Copilot caught. `root → app → mid → terminal` (peer: `react`) where `mid` hoists past a conflicting `app.react@17`. Under the previous BFS the stale path `[root, app, mid]` made `terminal`'s peer check trip on `app.react@17 ≠ root.react@18` and refuse the hoist; under DFS the actual post-hoist path `[root, mid]` finds no provider mismatch and `terminal` correctly flattens to root. - [x] All 10 pre-existing tests still pass (`one_transitive_dep_hoists_to_root`, `diamond_dep_hoists_once_to_root`, `version_conflict_keeps_loser_at_parent`, `deep_chain_flattens_in_one_pass`, `transitive_npm_alias_resolves_target_snapshot`, `non_empty_hoisting_limits_surfaces_unsupported`, `multi_importer_lockfile_surfaces_unsupported_workspace`, `external_dependencies_are_stripped_from_the_result`, `empty_lockfile_yields_empty_root`, `hoist_throws_on_broken_lockfile`). - [x] The previously-failing `peer_dependency_in_lockfile_surfaces_unsupported` test from pnpm#452 is replaced (peers are no longer an unsupported input). - [x] All Copilot review threads addressed and resolved (stale ancestor path, misleading comment). - [x] `just ready` (836 tests pass), `cargo doc --document-private-items`, `taplo format --check`, `just dylint` all clean.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…slice 3 close-out) (pnpm#465) Closes out the Slice 3 hoist-algorithm port. Two additions on top of slice 3c's peer-aware hoist: 1. **Multi-round convergence.** The DFS runs in a fixed-point loop — a peer-shadow refusal in one round can be reconsidered in a later round once the blocking ident has moved out of the ancestor chain (or a previously-empty root slot has been filled with a compatible version). Bounded by O(N) rounds since every move is one-way. 2. **`hoistingLimits` enforcement.** The upfront `UnsupportedHoistingLimits` guard is gone; the algorithm now reads `opts.hoisting_limits[root_locator]` and refuses to hoist names listed there. New `AbsorbDecision::Border` variant alongside `Free` / `SameNode` / `Conflict` / `PeerShadow`. Plus a backfill of behavior-pinning tests ported from `@yarnpkg/nm/tests/hoist.test.ts`. ## How multi-round works - `hoist_subtree` returns `bool` for whether it moved any node in the current round. - `hoist_into_root` wraps it in a `loop` that resets `visited` each iteration and exits when a round makes no moves. - Mirrors upstream `hoistTo`'s `do { hoistGraph(); } while (anotherRoundNeeded)` without the per-candidate `dependsOn` bookkeeping; pacquet's coarse re-walk catches the same unlock cases at the cost of revisiting unchanged nodes. Canonical case (`multi_round_unlocks_peer_friendly_hoist_after_blocker_moves`): `root → app → {widget(peer: x), x@1}` with root carrying no `x`. - **Round 1.** Alphabetical iteration visits `widget` first. App supplies `x@1`, root has no `x` → mismatch → `PeerShadow`, widget stays at app. Then `x` is evaluated: `Free`, hoist to root. - **Round 2.** Reconsider widget. App no longer has `x`; root has `x@1`. No ancestor disagreement → `Free`. Widget hoists. - **Round 3.** No moves; loop exits. ## How hoistingLimits works - Computed locator: `"{ident_name}@{reference}"` — for the wrapper's root that's `".@"`, matching pnpm's keying convention. - Border-name set: `opts.hoisting_limits.get(root_locator)`, defaulting to empty when absent. - New `AbsorbDecision::Border` is layered between the basic decision and the peer check. Names in the set never hoist; they stay where the lockfile placed them. - Mirrors upstream's `isHoistBorder` flag set during `cloneTree`. ## Upstream test ports From `yarnpkg/berry@4287909fa6` `packages/yarnpkg-nm/tests/hoist.test.ts`, expressed via lockfile fixtures so they exercise the wrapper too: - **`hoisting_limits_keeps_blocked_name_at_parent`** — `should not hoist packages past hoist boundary`. - **`hoisting_limits_blocks_multiple_names`** — `should not hoist multiple package past nohoist root`. - **`hoisting_limits_keyed_on_unrelated_importer_is_inert`** — non-interference sanity check. - **`self_dependency_does_not_loop`** — `should tolerate self-dependencies`. - **`basic_cyclic_dependency_terminates`** — `should support basic cyclic dependencies`. ## Documented gaps The `nm_hoist` doc-comment now lists what's deferred *intentionally* rather than "not done yet": - **Popularity-based ident preference.** Upstream's `buildPreferenceMap` picks the ident with more incoming references when two distinct deps share an alias; pacquet picks the first-visited. Affects the handful of upstream test cases that exercise the tie-break (`should honor package popularity when hoisting`, etc.). - **Multi-importer workspace hoist trees.** Still guarded upfront by `HoistError::UnsupportedWorkspace`. Workspace-aware hoisting needs per-importer roots and a different output shape. - **`ExternalSoftLink` descendants.** The wrapper only creates soft-links as zero-children placeholders, so upstream's "only-hoist-when-all-descendants-hoist" rule has nothing to delay today. Tests for `should not hoist portal with unhoistable dependencies` and similar were skipped. ## Upstream reference Aligned with `yarnpkg/berry@4287909fa6`: - [`hoist.ts:109-119`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L109-L119) — outer round loop in `hoist`. - [`hoist.ts:352-367`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L352-L367) — inner round loop in `hoistTo`. - [`hoist.ts:707`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L707) — `isHoistBorder` flag during `cloneTree`.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
pnpm#473) Slice 2 of pnpm#438 (the `nodeLinker: 'hoisted'` umbrella). Smaller than expected — when I looked, the schema field was already in place from #332's original `.modules.yaml` port, so the work reduced to pinning behavior rather than building plumbing. Two round-trip tests + a doc-comment on the field. ## What was actually missing The umbrella's §"Pacquet's current state" table claimed `hoistedLocations` was "Missing entirely" — that was wrong. The field is at `crates/modules-yaml/src/lib.rs:212` and serde-handles read/write correctly. What was missing: - **A test pinning the wire shape.** Nothing was stopping a future edit from breaking the camelCase mapping, the optional-with-skip-on-None behavior, or the `Record<string, string[]>` shape upstream relies on. - **A doc-comment** explaining what the field is for. Other Slice-related fields (`hoisted_dependencies`, `hoisted_aliases`) have full docstrings; this one was silent. ## Tests added Both live in `crates/modules-yaml/tests/real_fs.rs` (pacquet-side, no upstream counterpart — upstream's `installing/modules-yaml/test/index.ts` doesn't exercise `hoistedLocations` directly): - `hoisted_locations_round_trips` — a manifest with `hoistedLocations` populated survives write+read; the raw on-disk JSON keeps the per-depPath array shape (multi-entry arrays included, to pin the nested-conflict layout). - `absent_hoisted_locations_is_omitted_on_write` — `None` (the state every pacquet install writes today) serializes as the field *absent* from the file, matching upstream's `JSON.stringify(undefined)` behavior. Guards against accidental `Option::is_some` regressions on the `skip_serializing_if`. ## Regression catch verified Per `plans/TEST_PORTING.md`'s "break the subject to verify the test catches it" convention, I temporarily added `#[serde(rename = "wrongName")]` to the field. `hoisted_locations_round_trips` failed at the `expect("present")` on the deserialized value (the camelCase key no longer mapped). Reverted before commit. ## Type-shape decision Upstream's actual `ModulesRaw.hoistedLocations` is `Record<string, string[]> | undefined` — *not* `Record<DepPath, string[]>` despite the values being populated from depPaths internally. The umbrella's scope item ("`Option<BTreeMap<DepPath, Vec<String>>>`") was inconsistent with the upstream schema; I kept the existing `BTreeMap<String, Vec<String>>` because: 1. It already matches the upstream wire shape exactly. 2. Same pattern as the `hoisted_dependencies` field below, which deliberately keeps `String` keys (per its doc-comment at lines 148-153) because pnpm's `DepPath | ProjectId` union can't be statically disambiguated.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…e 4a) (pnpm#478) * feat(package-manager): hoisted dep-graph type skeleton (pnpm#438 slice 4a) Defines the directory-keyed dependency graph types used by the hoisted-linker install path. Types only — the walker that fills them from a lockfile lands in a follow-up. Ported from upstream: - `DependenciesGraphNode` mirrors deps/graph-builder `DependenciesGraphNode` minus the `fetching` / `files_index_file` fields that only the store-controller-bound walker can populate. - `DependenciesGraph` = `BTreeMap<PathBuf, DependenciesGraphNode>`, keyed by absolute directory path (unlike pacquet's existing depPath-keyed `deps_graph` module — hoisted nodes can occupy several directories when a name conflict forces nesting). - `DepHierarchy` is the recursive directory tree the linker walks to decide population order and which `<dir>/node_modules/.bin` to wire up. Newtype-wrapped because Rust doesn't allow recursive type aliases. - `DirectDependenciesByImporterId` is the per-importer alias-to-directory table the linker hands to the bin pass. - `LockfileToDepGraphResult` bundles everything the walker returns to the install pipeline, including `prevGraph` for the orphan-diff pass and `injectionTargetsByDepPath` for the injected-workspace re-mirror step. - `LockfileToHoistedDepGraphOptions` carries the subset of upstream's options that the to-be-ported walker actually reads today; fields tied to the store controller, fetch concurrency, and workspace project list will be added when their consumers land. Smoke tests cover empty-result default construction, node insertion by `dir`, recursive hierarchy nesting, and default-options shape. Upstream: - <https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/lockfileToHoistedDepGraph.ts> - <https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts> * docs(package-manager): v9 depPath in 4a tests + map-key typing notes (pnpm#478 review) - Replace v5-era `/accepts/1.3.7` depPath strings in the smoke test with v9-shape `accepts@1.3.7` (the format pacquet's `PkgNameVerPeer` produces and the lockfile snapshots use). The legacy `/name/version` shape is only kept for read-side `hoistedAliases` compatibility. Caught by Copilot. - Tighten doc-comments on `hoisted_locations`, `injection_targets_by_dep_path`, and `skipped` to spell out *why* the keys are plain `String` (not `DepPath`): upstream types those fields with raw `string` in the hoisted-specific interface, even though the values are populated from depPaths. Mirrors upstream's literal typing.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…npm#486) Sub-slice 4b of pnpm#438. Implements `lockfile_to_hoisted_dep_graph(lockfile, opts) -> LockfileToDepGraphResult` — the walker that consumes the Slice 3 hoister output and produces the directory-keyed graph + hierarchy + hoisted_locations + direct-deps-by-importer-id + injection-targets-by-dep-path that 4a (pnpm#478) pinned the type shape of. Single-importer only (multi-importer lockfiles surface as `HoistError::UnsupportedWorkspace` via the underlying hoister). ## What's deferred Three things are intentionally left to follow-ups so 4b stays focused: - **Store fetch wiring** — `fetching` / `files_index_file` on the graph node are still defaulted. 4c will wire `StoreController::fetchPackage` (and the skip-fetch optimization that consults `currentHoistedLocations`). - **Installability check** — the walker honors `opts.skipped` as input but doesn't run `packageIsInstallable` to add new entries. 4c will plumb `package_is_installable` from `pacquet-package-is-installable` (already used by the isolated linker path). - **`prev_graph` diff** — `prev_graph` is `None`. 4d will walk the *current* lockfile alongside the wanted one (using `force: true` + empty `skipped` per upstream) and produce the orphan-removal input Slice 5's linker consumes. ## Two-pass design Upstream's `fetchDeps` walks asynchronously via `await Promise.all(deps.map(async (dep) => { ... }))`. The key invariant: each sibling's async function runs its sync prologue (insert + push to `pkgLocationsByDepPath`) before any continuation fires, because `Promise.all` sync-invokes each function and the inner `await fetchDeps(...)` only yields after the prologue completes. So by the time any node's post-recursion `graph[dir].children = getChildren(...)` line runs, every node in the entire tree is already in `pkgLocationsByDepPath`. Pacquet runs synchronously. To preserve that invariant cleanly: 1. **Pass 1 (recursive walk).** Insert each node into the graph with `children: BTreeMap::new()` and push its dir to `pkg_locations_by_dep_path`. Recursion order matches upstream's outer body (insert → push → recurse → push to `hoistedLocations`). 2. **Pass 2 (`fill_children`).** Iterate every graph node and resolve `children: alias → dir` from the snapshot's `dependencies` + `optionalDependencies` via `SnapshotDepRef::resolve`, consulting the now-complete `pkg_locations_by_dep_path`. A single-pass walker that computed children inline produced incorrect `children` maps for hoisted transitive deps — the canonical case `root → a → b` with `b` flattened to root left `a.children["b"]` empty because `b`'s pkg_locations entry hadn't been recorded yet. Caught by `walker_transitive_dep_flattens_under_root` before the refactor. ## Tests - `walker_empty_lockfile_produces_empty_result` — empty importer → empty graph, with the `"."` root importer key still present. - `walker_single_root_dep_emits_one_node` — `root → a`, node at `<lockfile_dir>/node_modules/a`. - `walker_transitive_dep_flattens_under_root` — `root → a → b`: `b` hoists to root, `a.children["b"]` points at the root-level dir. - `walker_version_conflict_keeps_loser_nested` — `root → {a@1, c}` with `c → a@2`: `a@1` at root, `a@2` nested under `c`; `hoisted_locations` records both directories; `c.children["a"]` points at the nested copy. - `walker_honors_pre_skipped_dep_path` — depPaths in `opts.skipped` are dropped from the walk entirely (and not recorded in `hoisted_locations`). - `walker_records_directory_resolution_as_injection_target` — `directory:` resolutions populate `injection_targets_by_dep_path` for the post-install re-mirror pass.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
… slice 4c) (pnpm#491) Sub-slice 4c of pnpm#438. Wires `package_is_installable` into the hoisted-graph walker shipped in 4b (pnpm#486), so optional packages on unsupported platforms get filtered into `result.skipped` and engine-strict mismatches surface as typed errors. Single-importer only; store I/O and `prev_graph` diff still to come (4d, then 5). ## Behavior Mirrors upstream's `if (!opts.force && packageIsInstallable(...) === false)` gate at [lockfileToHoistedDepGraph.ts:200-211](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/lockfileToHoistedDepGraph.ts#L200-L211): | Upstream return | `package_is_installable` (Rust) | Walker action | |---|---|---| | `null` (no constraint) | `Ok(Installable)` | emit node | | `true` (warn but proceed) | `Ok(ProceedWithWarning)` | emit node (warning emit deferred) | | `false` (optional + incompatible) | `Ok(SkipOptional)` | add to `result.skipped`, skip node | | throws `UnsupportedEngineError` (strict) | `Err(InstallabilityError::Engine)` | `HoistedDepGraphError::Installability` | | throws `InvalidNodeVersionError` | `Err(InstallabilityError::InvalidNodeVersion)` | `HoistedDepGraphError::Installability` | ## New options on `LockfileToHoistedDepGraphOptions` - `force: bool` — bypass the check entirely. Used by Slice 4d's `prev_graph` walk, where the previous lockfile is replayed wholesale so the orphan diff catches packages that would now filter out. Mirrors upstream's `force` at [lockfileToHoistedDepGraph.ts:73](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/lockfileToHoistedDepGraph.ts#L73-L76). - `engine_strict`, `current_node_version`, `current_os`, `current_cpu`, `current_libc`, `supported_architectures` — host-derived axes the check consumes. ## New output on `LockfileToDepGraphResult` - `skipped: BTreeSet<String>` — the input `opts.skipped` cloned and extended with depPaths added during the walk. Upstream mutates the input set in place; pacquet returns the augmented set so the caller can persist it into `.modules.yaml.skipped` without sharing mutable state. ## Tests 15 walker tests total — the 10 from 4b survive (one extended to assert the input depPath survives into output `skipped`), plus five new ones: - `walker_skips_optional_dep_on_unsupported_platform` — Linux host, package targets darwin only, `optional: true` → added to `result.skipped`, no graph entry, no `hoisted_locations`. - `walker_emits_required_dep_with_unsupported_platform_as_warning` — same shape but `optional: false` → walker proceeds (matches upstream `packageIsInstallable === true`); warning log emit is out of scope for 4c. - `walker_errors_on_engine_strict_mismatch` — `engine_strict: true` + `engines.node = ">=99.0.0"` → `HoistedDepGraphError::Installability`. - `walker_force_bypasses_installability_check` — `force: true` emits an incompatible required dep without erroring. - `walker_emits_compatible_dep` — sanity: compatible host + no constraint mismatch → graph entry, no skip.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…k.yaml (pnpm#434 slice 6) (pnpm#495) Pacquet wrote the raw wanted lockfile as the current lockfile. Upstream pnpm writes a filtered version with optional + skipped subtrees pruned and `include` flags applied, so the next install diffs against what was actually materialized rather than the resolver's full ambition. Without the prune, dropped snapshots (slice 1 installability, slice 4 fetch failures, slice 5 `--no-optional`) were claimed present in the current lockfile and the follow-up install would skip work that should have run. Ports <https://github.com/pnpm/pnpm/blob/94240bc046/lockfile/filtering/src/filterLockfileByImportersAndEngine.ts#L46-L94> → <https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L687-L695>. Pacquet-specific simplification: instead of re-running the engine + supportedArchitectures + skipped checks at filter time, `filter_lockfile_for_current` reuses the `SkippedSnapshots` set already produced by the install pipeline. Its three subsets (`installability`, `fetch_failed`, `optional_excluded`) are the exact set upstream's filter would drop — same observable result, no duplicated walk. ## Changes - **`pacquet-lockfile`**: `Clone` derives on `Lockfile`, `LockfileSettings`, `ProjectSnapshot`, `SnapshotEntry`, `PackageMetadata`, `PeerDependencyMeta`, `ResolvedDependencySpec`. Needed to build a filtered lockfile by clone-and-mutate. - **`pacquet-package-manager/src/current_lockfile.rs`** (new): `filter_lockfile_for_current(lockfile, included, skipped) -> Lockfile`. Three-phase filter: 1. BFS the snapshot graph from filtered importer roots, skipping keys in `skipped`. Produces the reachable set. 2. Per-importer: clear dep maps whose `include` flag is false; trim `optional_dependencies` to entries whose target survived. 3. `snapshots:` / `packages:` filtered to the reachable set (packages key off `without_peer()` so peer-variant survivors keep their shared metadata row). - **`install.rs`**: replace the raw `save_current_to_virtual_store_dir` call with `filter_lockfile_for_current(...).save_current_to_virtual_store_dir(...)`. ## Tests 8 unit tests covering each filter behavior: - `skipped_snapshot_pruned_from_snapshots_and_importer_optional` - `include_optional_false_clears_importer_section` - `transitive_under_skipped_snapshot_is_pruned` - `snapshot_reachable_via_kept_path_survives` (mirrors upstream's `:712` shared-dep case at the filter level) - `packages_filtered_to_surviving_metadata_keys` (peer-variant metadata sharing) - `link_optional_entries_survive_post_filter` (workspace link: deps don't get post-filtered) - `empty_skipped_and_full_include_is_identity_for_reachables` (baseline) - `orphan_snapshots_are_pruned` (snapshots unreachable from any importer get dropped) Test-the-test verified by breaking the BFS walker — two tests fail. ## Out of scope - Hoisted-linker current-lockfile variant (`:633`) — pacquet's hoisted node-linker isn't fully wired through the lockfile-write path yet; tracked separately under pnpm#438. - `pnpm install --filter` slicing — pacquet has no `--filter` yet. Closes pnpm#493.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…slice 4d) (pnpm#494) * feat(package-manager): prev_graph diff from current lockfile (pnpm#438 slice 4d) `lockfile_to_hoisted_dep_graph` now takes an optional `current_lockfile: Option<&Lockfile>` and populates `result.prev_graph` from a second walk over that lockfile. Ports upstream's wrapper at installing/deps-restorer/src/lockfileToHoistedDepGraph.ts. When the current lockfile is `Some(lf)` with a non-empty `packages:` map, the second walk runs with `force: true` and `skipped: BTreeSet::new()`. Force matters: an orphan that landed under the previous install but would now fail installability (e.g., the host changed platforms) still surfaces in `prev_graph` so Slice 5's linker can find and rimraf the stale directory. Empty skipped matters: the previous install's own filter set is unrelated to "which directories still exist on disk." API change - The public `lockfile_to_hoisted_dep_graph(lockfile, opts)` signature gains a middle `current_lockfile: Option<&Lockfile>` argument. Single existing caller (the tests) updated to pass `None`. Splits the previous body into a private `build_dep_graph` helper that the wrapper calls once or twice depending on whether a current lockfile is present. prev_graph shape - `None` when no current lockfile is supplied, or when the supplied lockfile has no `packages:` map (a brand-new install in progress). Mirrors upstream's `prevGraph = {}` fallback — pacquet uses `None` rather than an empty map, but the linker treats both the same. - `Some(graph)` otherwise, keyed by directory just like the wanted-lockfile graph. Tests - `prev_graph_none_when_current_lockfile_absent` — no current lockfile → `prev_graph` is `None`, wanted graph still produced. - `prev_graph_none_when_current_lockfile_has_no_packages` — current lockfile with `packages: None` → `prev_graph` is `None`. - `prev_graph_contains_orphan_from_current_only_lockfile` — package in current but not wanted appears in `prev_graph`, not in `graph`. - `prev_graph_includes_orphan_even_when_now_incompatible` — darwin-only orphan in the current lockfile, host is linux, wanted lockfile is empty: `prev_graph` still contains it (proves `force: true` overrides the installability check), and the wanted-walk's `skipped` stays empty (proves the prev-walk's skipped set is independent). All 4 new tests pass alongside the 15 walker tests from 4a-4c. * fix(package-manager): collapse empty current packages to None for prev_graph (pnpm#494 review) `current.packages.is_some()` matched `Some(empty_map)` too, causing 4d to do an unnecessary second walk and return `Some(empty_graph)` for a case the doc-comment described as `None`. Tighten the guard to require a non-empty `packages` map. Pacquet uses `Option<DependenciesGraph>` for `prev_graph` (upstream uses an always-present `DependenciesGraph` with `{}` for the no-current case). The no-packages and empty-packages cases both produce the same observable behavior — "no orphans to consider" — so they should share one representation rather than be inconsistent. The doc-comment's stated contract was `None`; the code now matches. Added `prev_graph_none_when_current_lockfile_has_empty_packages` to pin the empty-map case. Caught by Coderabbit on pnpm#494. * style(package-manager): rustfmt the 4d follow-up
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…npm#505) * feat(package-manager): linkHoistedModules linker (pnpm#438 slice 5) New module `crates/package-manager/src/link_hoisted_modules.rs` produces the on-disk `node_modules/` tree from Slice 4's `LockfileToDepGraphResult`. Ports installing/deps-restorer/src/linkHoistedModules.ts. ## Three phases 1. **Orphan removal.** Every directory in `prev_graph` but not in `graph` is silently `rimraf`'d before any insert. `try_remove_dir` swallows all errors (NotFound + PermissionDenied + EBUSY), matching upstream's `tryRemoveDir` tolerance. 2. **Per-node import.** Hierarchy walked top-down, rayon-parallel at each level. Every node goes through `import_indexed_dir` with `force: true, keep_modules_dir: true` — the hoisted-linker call shape Slice 1 was designed for. 3. **Per-`node_modules` bin link.** After a level's children are all imported, `link_direct_dep_bins` populates `<parent>/node_modules/.bin` from the just-materialized direct children. Bin link runs only after every child's subtree is done so package.json reads always see the fully-populated package. ## API shape ```rust pub fn link_hoisted_modules<R: Reporter>( opts: &LinkHoistedModulesOpts<'_>, ) -> Result<(), LinkHoistedModulesError>; ``` `LinkHoistedModulesOpts` borrows `graph`, `prev_graph`, `hierarchy`, and a `cas_paths_by_pkg_id: HashMap<String, HashMap<String, PathBuf>>` keyed by `pkg_id_with_patch_hash`. ## Decoupling from the store Upstream's linker is async and calls `storeController.fetchPackage()` inline during the walk — pacquet's is sync and takes pre-fetched CAS paths. Slice 6 (the install pipeline) is responsible for fetching every package through pacquet's existing tarball / store-dir / git-fetcher machinery before invoking the linker. This keeps the linker focused on file-system layout and reuses the proven fetch chain that the isolated path already exercises. ## Optional-dep tolerance A graph node whose `pkg_id_with_patch_hash` is missing from `cas_paths_by_pkg_id`: - If `node.optional`: silently skipped, no directory created. Mirrors upstream's `if (depNode.optional) return` on fetch failure. - Otherwise: surfaces as `LinkHoistedModulesError::MissingCasPaths { pkg_id, dir }`. ## Tests 7 real-tempdir tests in `link_hoisted_modules/tests.rs`: - `import_pass_creates_package_directory` — single-package smoke. - `orphan_directory_is_removed` — `prev_graph` diff produces rimraf of stale directory; planted contents are gone after. - `nested_hierarchy_materializes_inner_node_modules` — version-conflict layout; loser ends up at `<outer>/node_modules/<inner>`. - `missing_cas_for_required_dep_errors` — required + missing CAS → typed `MissingCasPaths` error. - `missing_cas_for_optional_dep_skips_silently` — optional + missing CAS → no error, no directory. - `no_prev_graph_skips_orphan_pass` — fresh install (no prior lockfile) path. - `orphan_already_removed_is_tolerated` — phantom orphan in `prev_graph` not present on disk doesn't error. Each test plants synthetic CAS files in a tempdir and asserts the on-disk tree after the linker runs. * fix(package-manager): fail-fast on hierarchy/graph inconsistency (pnpm#505 review) The hierarchy walk silently skipped entries missing from `graph`, which would produce a partial install layout instead of surfacing the bug. Slice 4's walker keeps the two in sync today, but a future bug there shouldn't yield a partial tree. Add `LinkHoistedModulesError::MissingGraphNode { dir }` and return it when a hierarchy directory has no graph entry. Upstream effectively errors here too — its `graph[dir].fetching` read would throw `Cannot read properties of undefined` — pacquet just spells the failure out. Regression test `hierarchy_entry_missing_from_graph_errors` exercises the path with an empty graph and a hierarchy referencing a phantom dir. Caught by Coderabbit.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
Add the `--node-linker [isolated|hoisted|pnp]` CLI flag to `pacquet install`, mirroring pnpm's flag. Overrides `Config.node_linker` for the invocation; absent flag → config's yaml/npmrc value wins. The CLI parses into a `NodeLinkerArg` mirror enum (kept in the CLI crate so `pacquet-config` stays free of `clap`), then `into_config()` converts to the canonical `pacquet_config::NodeLinker`. Threaded into the `Install` runner as an explicit `node_linker: NodeLinker` field rather than reading `config.node_linker` directly. Matches the existing pattern `supported_architectures` uses (`state.config` is `&'static`, so the override-merge happens at the CLI layer and lands here as a fully-resolved value). `build_modules_manifest` consumes it on the `.modules.yaml.nodeLinker` write so the persisted setting reflects the invocation, not just the config. `pacquet add` uses Config's value by default (`add` doesn't expose `--node-linker` per the umbrella scope; Slice 6's pipeline integration will revisit if needed). 5 new CLI tests: - `node_linker_default_is_none` — flag absent → field is None. - `node_linker_hoisted` / `node_linker_isolated` / `node_linker_pnp` — each value parses and round-trips through `into_config()`. - `node_linker_invalid_value_rejected` — clap rejects unknown values with the bad value in the error message. - `node_linker_arg_into_config_matches_every_variant` — every ValueEnum variant has a canonical mapping (compile-fails on future variants that forget the mapping). `NodeLinker` gains `Clone + Copy + Eq` so it can be threaded by value into `Install` and matched in tests.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…npm#438 slice 6) (pnpm#518) * feat(package-manager): wire node_linker hoisted into install pipeline (pnpm#438 slice 6) Branches `Install::run` / `InstallFrozenLockfile::run` on `config.node_linker == Hoisted`. Threads the slice 4 `lockfile_to_hoisted_dep_graph` walker output and the per-package CAS index produced by `CreateVirtualStore` (slot writes skipped under hoisted) into the slice 5 `link_hoisted_modules` linker. Persists the walker's `hoisted_locations` into `.modules.yaml` for rebuild and follow-up installs to consume. Pipeline changes under hoisted: - `CreateVirtualStore` skips `CreateVirtualDirBySnapshot` for both warm and cold batches, collects each snapshot's CAS file index keyed by `PkgIdWithPatchHash` into a new `cas_paths_by_pkg_id` output field. - `InstallPackageBySnapshot::run` returns the per-package CAS map unconditionally and skips the virtual-store-slot write when its new `node_linker` field is `Hoisted`. - `InstallFrozenLockfile::run` skips `SymlinkDirectDependencies`, `LinkVirtualStoreBins`, the isolated hoist pass, and `BuildModules` under hoisted; runs `lockfile_to_hoisted_dep_graph` + `link_hoisted_modules` in their place. Folds the walker's augmented skip set back into the install-time `SkippedSnapshots` so `.modules.yaml.skipped` reflects the union. - `Install::run`'s `build_modules_manifest` now takes the walker's `hoisted_locations` and writes it through `Modules.hoisted_locations` (only when non-empty so the isolated path doesn't produce a hoisted-only key). The build phase under hoisted (rebuild over `hoistedLocations` with ancestor-`.bin` lookup, `MISSING_HOISTED_LOCATIONS`) is slice 7's scope and is intentionally left as a no-op here. Workspace and `hoistingLimits` / `externalDependencies` knobs are slices 9-10. Mirrors upstream's hoisted branch in `headlessInstall` at https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L369-L425. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(package-manager): docs intra-link disambiguation + skip-set merge fix (pnpm#438 slice 6) Docs CI was failing because `[`crate::link_hoisted_modules`]` is ambiguous between the function and the module of the same name. Disambiguate by switching to the function form `[`crate::link_hoisted_modules()`]` everywhere. Slice 6's hoisted-walker skip-set merge previously folded every entry in `walker_result.skipped` into `SkippedSnapshots::insert_installability`, which would promote pre-existing transient skips (`fetch_failed` / `optional_excluded`) into the persisted-on-disk `.modules.yaml.skipped` set. Diff against the input `walker_skipped` so only walker-discovered (genuinely-new) installability skips flow into the persisted subset.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
… slice 7) (pnpm#520) Re-enables `BuildModules` for `nodeLinker: hoisted` installs. Slice 6 landed the hoisted install pipeline but skipped the build phase entirely because `BuildModules` walked virtual-store slot directories that don't exist under hoisted. Slice 7 routes the build phase through the slice 4 walker's per-node `dir` so postinstall scripts can run against the on-disk hoisted tree. Changes: - `BuildModules` gains two fields: `pkg_root_by_key: Option<&HashMap<PackageKey, PathBuf>>` overrides the per-snapshot pkg_root lookup with the slice 4 walker's `DependenciesGraphNode::dir` values; `gather_ancestor_bin_paths: bool` switches `extra_bin_paths` to the new `bin_dirs_in_all_parent_dirs` helper, a port of upstream's `binDirsInAllParentDirs` at https://github.com/pnpm/pnpm/blob/94240bc046/building/after-install/src/index.ts#L476-L487. - `pkg_root_for_key` helper: routes between the layout-based slot computation (isolated) and the override map (hoisted). Hoisted snapshots absent from the map (walker-dropped) take the same exit as the isolated `!pkg_dir.exists()` skip. - `InstallFrozenLockfile::run` now builds a snapshot-key → first-recorded-dir map from `walker_result.graph.values()` and threads it (plus `gather_ancestor_bin_paths: true`) into `BuildModules` instead of skipping the phase. Multiple graph nodes with the same dep_path collapse to the first entry, matching upstream's `pkgRoots[0]` pick at https://github.com/pnpm/pnpm/blob/94240bc046/building/after-install/src/index.ts#L348. - New private `HoistedLinkerOutput` struct bundles `hoisted_locations` and `pkg_root_by_key` so the hoisted-branch return doesn't trip `clippy::type_complexity`. Side-effects-cache key shape is unchanged — it's keyed by `pkg_id_with_patch_hash` + dep-graph hash, both layout-independent (`crates/graph-hasher/src/dep_state.rs`). `MISSING_HOISTED_LOCATIONS` is intentionally deferred — pacquet has no `rebuild` command, so the install path always re-runs the walker and never reads `.modules.yaml.hoisted_locations`. Tracked as a follow-up for when `pacquet rebuild` lands. Workspace-aware hoisting (slice 9) and `hoistingLimits` / `externalDependencies` (slice 10) remain. Tests: - Three new helper tests pin `bin_dirs_in_all_parent_dirs` against top-level, conflict-nested, and scoped-package shapes. - Three new helper tests pin `pkg_root_for_key` for the isolated pass-through, the hoisted override hit, and the hoisted-missing short-circuit.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…slice 9) (pnpm#521) Enables `nodeLinker: hoisted` for multi-importer (workspace) lockfiles. Previously the hoister rejected any lockfile with importers beyond `.` with `UnsupportedWorkspace`; now the whole workspace shares one hoist plan so conflicting versions across projects dedupe, and each importer's project-tree node_modules is materialized per-project. real-hoist: - Drop the upfront UnsupportedWorkspace guard in `hoist()`. The wrapper already constructed non-root importers as Workspace-kind children of the virtual `.` root; only the guard needed to go. - Add `HoistOpts::hoist_workspace_packages: bool` (default true). When false, non-root importers stay out of the shared tree (matches upstream's `hoistWorkspacePackages: false` mode for the Bit CLI). Removed the corresponding error variant. hoisted_dep_graph walker: - Replace the `workspace:`-prefix skip with a recurse-into branch: for each Workspace-kind hoister child, walk its post-hoist children under `<lockfile_dir>/<importer_id>/node_modules`. The workspace node itself is NOT added to the graph or to the parent's hierarchy — it has no contents to import. - Add per-importer `hierarchy` and `direct_dependencies_by_importer_id` entries. Per-importer direct deps are computed from the lockfile (not from the workspace node's post-hoist children) because hoisted-up siblings don't show up in the workspace node's tree. First-recorded location wins, matching upstream's `pkgLocationsByDepPath[depPath][0]` pick. - Add `LockfileToHoistedDepGraphOptions::hoist_workspace_packages` (default true) and plumb to HoistOpts. Config: - Add `Config::hoist_workspace_packages: bool` (#[default = true]). Read from `pnpm-workspace.yaml` via `WorkspaceSettings`. SymlinkDirectDependencies: - Add `link_only: bool` flag. When true, skip every Regular dep and only materialize Link entries (workspace siblings). Used by the hoisted branch in InstallFrozenLockfile::run so workspace-sibling symlinks land under each importer's `node_modules/<alias>` even though the regular deps now live as real directories from the hoisted linker. InstallFrozenLockfile::run: - Plumb `config.hoist_workspace_packages` into the walker. - After link_hoisted_modules, run SymlinkDirectDependencies with `link_only: true` so workspace siblings get their per-project symlinks. Mirrors upstream's hoisted branch at https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L411-L440. Tests: - real-hoist: replace the now-removed `UnsupportedWorkspace` test with one that pins multi-importer hoister output (Workspace children with percent-encoded names + `workspace:<id>` references) and one that pins the `hoist_workspace_packages: false` opt-out. - walker: three new tests cover per-importer direct_deps emission, per-importer hierarchy entries, the `hoist_workspace_packages: false` root-only mode, and version-conflict-across-importers nesting.
pull Bot
pushed a commit
to dwongdev/pnpm
that referenced
this pull request
May 14, 2026
…nobs (pnpm#438 slice 10) (pnpm#522) Plumbs the two programmatic-only hoister knobs from `pnpm-workspace.yaml` through to the slice 4 walker and the slice 3 hoister. Both fields already existed on `HoistOpts`; this slice wires them end-to-end. - `Config::hoisting_limits: BTreeMap<String, BTreeSet<String>>` — per-importer block-list, locator-keyed (`'.@'` for the root). Reads `hoistingLimits: { ".@": [foo, bar] }` from yaml. Mirrors upstream's https://github.com/pnpm/pnpm/blob/94240bc046/installing/linking/real-hoist/src/index.ts#L10 programmatic-only knob, exposed as yaml for parity since the ergonomics of the locator-keyed map don't translate to a CLI flag. - `Config::external_dependencies: BTreeSet<String>` — name slots reserved at the root for an external linker (the Bit CLI is the only known consumer upstream). Reads `externalDependencies: [...]` from yaml. - `LockfileToHoistedDepGraphOptions` gains both fields and forwards them to `HoistOpts` in `build_dep_graph`. - `InstallFrozenLockfile::run` clones the two `Config` fields into the walker opts. Both knobs default to empty (no limits, no externals), matching upstream's default. Neither has any effect under `nodeLinker: isolated` — the isolated linker keeps per-importer subtrees by construction and doesn't consult the hoister. Tests: - `parses_hoisting_limits_from_yaml_and_applies` — yaml round-trip + apply_to. - `parses_external_dependencies_from_yaml_and_applies` — same. - `omitting_hoisting_limits_and_external_dependencies_keeps_defaults` — pins the apply_to skip-on-None branch so a yaml without these keys doesn't accidentally overwrite Config defaults. - `walker_forwards_external_dependencies_to_hoister` — end-to-end: the walker observes an empty graph for an externalised alias because the hoister stripped it. Pins the slice 10 plumbing.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…pm#438 slice 1) (pnpm#441) Extends pacquet's package-import primitive to cover pnpm's `importIndexedDir(..., { force, keepModulesDir })` call shape so the hoisted node-linker (the umbrella of pnpm#438) has a primitive to build on. - Renames `create_cas_files` → `import_indexed_dir` to match pnpm's name (its docstring already said it mirrored `tryImportIndexedDir`), and adds an `ImportIndexedDirOpts` struct with `force` and `keep_modules_dir`. Both linkers now go through the same function — pnpm uses one function for both, so does pacquet. - **Default opts** (`force: false, keep_modules_dir: false`): the isolated linker's behavior. Existing target short-circuits; fresh target gets `mkdir` + parallel `link_file`. Unchanged hot path for the two existing call sites (`create_virtual_dir_by_snapshot`, `install_package_from_registry`). - **`force: true`**: stage the new contents in a sibling directory, remove the old tree, rename stage into place. A regular file or symlink occupying the target is unlinked first. - **`force: true` + `keep_modules_dir: true`**: before the swap, `dir_path/node_modules/` is moved into the staging directory so nested deps survive the rebuild. On any failure after the move, the staged copy is restored to `dir_path/node_modules/` before staging is cleaned up — staging never holds the user's only copy of nested deps. This is the call shape the hoisted linker will use. - No install-pipeline wiring for hoisted yet. Subsequent slices (`linkHoistedModules` analog, hoist algorithm, `Install::run` branch on `node_linker`) will pass `force + keep_modules_dir` from the new hoisted path. The default-opts behavior is exercised in production today via the existing isolated callers. ## Upstream reference Ported from `pnpm/pnpm@94240bc`: - [`fs/indexed-pkg-importer/src/importIndexedDir.ts`](https://github.com/pnpm/pnpm/blob/94240bc0464196bd52f7006b97f6d9a43df34633/fs/indexed-pkg-importer/src/importIndexedDir.ts) — the function this ports. - [`store/controller-types/src/index.ts`](https://github.com/pnpm/pnpm/blob/94240bc0464196bd52f7006b97f6d9a43df34633/store/controller-types/src/index.ts) — the `ImportOptions` shape `ImportIndexedDirOpts` mirrors. - [`installing/deps-restorer/src/linkHoistedModules.ts:134`](https://github.com/pnpm/pnpm/blob/94240bc0464196bd52f7006b97f6d9a43df34633/installing/deps-restorer/src/linkHoistedModules.ts#L134) — the call site that will drive the `force + keepModulesDir` combination once the later slices land. ## Scope omissions (intentional) - `moveOrMergeModulesDirs` semantics for the case where the indexed file map itself contains `node_modules/` entries. Upstream merges; pacquet errors with `NodeModulesCollision`. The hoisted-linker call site never produces that state in practice (npm and pnpm strip `node_modules/` at pack time), so erroring loudly until a real caller demands the merge is the right call. - No `safeToSkip` short-circuit. Upstream's pre-existence check beyond the default short-circuit lives in `index.ts` around the importer; that decision belongs at the Slice 5 call site, not in this primitive. ## Performance The isolated-linker hot path's syscall count is unchanged. The old `if dir_path.exists() { return Ok(()); }` (one `stat`) becomes `fs::symlink_metadata(dir_path)` (one `lstat`) plus a match dispatch; `populate_dir` is the old `create_cas_files` body verbatim and gets inlined under release LTO. The new `force` branches only run for hoisted, which has no callers yet. One behavior diff: a dangling symlink at the target under default opts used to fall through to `mkdir` and surface an IO error; under the new code it short-circuits as "already populated". The virtual store doesn't produce dangling symlinks under normal operation, so this is theoretical.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…npm#438 slice 3a) (pnpm#448) * feat(real-hoist): crate skeleton + lockfile-to-HoisterTree wrapper (pnpm#438 slice 3a) First sub-slice of the Slice 3 hoister port from the umbrella. Lands the IO surface and the pnpm-side wrapper; the inner `@yarnpkg/nm` algorithm is stubbed (returns the input tree unchanged) and replaced in 3b. New crate `pacquet-real-hoist`: - `HoisterTree` / `HoisterResult` / `HoisterDependencyKind` / `HoistingLimits` / `HoistOpts` / `HoistError`, mirroring upstream's `@yarnpkg/nm` types and the pnpm wrapper's option object. - `RcByPtr<T>` wrapper providing identity-hashed `Rc<T>` so children stored in `IndexSet` keep JS `Set<HoisterTree>` semantics (a node added via two parent paths stays shared) without paying the cost of structural hashing. - `hoist(&Lockfile, &HoistOpts)` — ports the wrapper at `installing/linking/real-hoist/src/index.ts`. Builds the HoisterTree from the lockfile (root importer's dependencies/devDependencies/optionalDependencies merged, then recursive descent through `snapshots`), adds workspace importer children, plus `link:` placeholders for `externalDependencies`, runs the inner stub, and post-filters `externalDependencies` out of the top-level result. - `LockfileMissingDependency` error surfaced as a `miette` diagnostic with code `ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY`, matching the upstream error code at https://pnpm.io/errors. Five unit tests pin the wrapper's observable behaviour: - The "broken lockfile" error case is a direct port of pnpm's `installing/linking/real-hoist/test/index.ts` test. - An empty lockfile yields an empty root. - A `root → a → b` lockfile, under the stub, gives `root → a → b` (nothing hoisted). Pinning this means 3b's algorithm replacement is observably correct — the tree shape will change to `root → {a, b}` once real hoisting lands. - A diamond dep (`root → {a, c} → b`) keeps `b` as one identity through both parents (proves the wrapper's dedup cache). - `external_dependencies` are stripped from the result. Upstream references pin `pnpm/pnpm@94240bc` and `yarnpkg/berry@4287909fa6`. * docs(real-hoist): drop slice/umbrella scaffolding from code comments Slice and sub-slice numbering is PR-organisation scaffolding — it's useful in commit messages and PR descriptions but it rots inside committed code the moment the referenced slices land, get renumbered, or are abandoned. Rewrite the relevant doc-comments to describe what the code *is* today rather than where it sits in a multi-PR sequence: - Top-level module doc drops the "This is Sub-slice 3a of pnpm#438" intro. - `hoist` doc drops the "Sub-slice 3a pins / later sub-slices replace" framing. - `nm_hoist` stub doc drops "Sub-slice 3b replaces this". - `external_dependencies` / non-root importer comments drop their "umbrella's Slice 10 / Slice 9" pointers. - One test doc rewords "Sub-slice 3b's algorithm replacement is observably correct" into a description of what the test pins without naming the future slice. * fix(real-hoist): aliased snapshot lookup, cycle-safe rc identity, non_exhaustive HoistError (pnpm#448 review) Address four Copilot review findings on the new crate: 1. `collect_snapshot_deps` looked up `SnapshotDepRef::Alias` deps under the alias name instead of the resolved target name. The snapshot key for an npm-alias is `<target>@<ver>`, not `<alias>@<ver>`, so any real aliased transitive would have surfaced as `LockfileMissingDependency`. `build_dep_node` now takes the resolved `&PkgNameVerPeer` directly and the caller builds it via `dep_ref.resolve(alias)` for snapshot deps or `PkgNameVerPeer::new(alias, spec.version)` for importer deps. 2. `build_dep_node`'s cycle handling re-allocated the node on the way out (placeholder Rc, recurse, *new* finished Rc) which left any cycle-visiting Rc pointing at the empty placeholder. `HoisterTree::dependencies` is now a `RefCell<IndexSet<...>>` and the construction populates the cell in place — the placeholder and the populated node are the same allocation, so a back-edge reads the eventually-populated set. Same JS `Set<HoisterTree>` mutation semantics the real hoist algorithm needs. 3. `convert` had the same bug and gets the same fix on `HoisterResult::dependencies` and `HoisterResult::references`. The stub's traversal now collects the input children into a Vec before recursing so the borrow on the input cell drops, and populates the output cell in place. 4. `HoistError` is now `#[non_exhaustive]`, matching the rest of pacquet's public error enums. A new regression test (`transitive_npm_alias_resolves_target_snapshot`) pins the fix for (1): the snapshot key the wrapper looks up matches the target package, the node's exposed `name` stays the alias, and `ident_name` / `reference` carry the resolved target's identity. Existing tests updated to take `Ref<'_, …>` via `.borrow()`. All six pass; `just ready`, `cargo doc --document-private-items`, `taplo`, and `just dylint` clean.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
## Summary Replaces the stub `nm_hoist` (landed in pnpm#448) with a working hoister and the surrounding guardrails: BFS over the result graph pulling every eligible descendant up to the root, plus upfront refusal of inputs the algorithm doesn't yet model. ### What the algorithm models - **Free hoist.** A transitive dep with no name collision at the root surfaces at the root. - **Identity dedup.** A dep reachable through multiple parents shares one `Rc` thanks to the wrapper's cache; the hoist preserves that identity and strips the duplicate reference at the other parent path. - **Parent-wins on version conflict.** When two distinct deps share an alias but resolve to different snapshot keys, the first BFS visitor takes the root slot and the other stays under its declaring parent. Visit order matches the wrapper's alias-sorted insertion order, so the outcome is deterministic. - **Deep chains flatten in one pass.** `root → a → b → c → d` becomes `root → {a, b, c, d}` — each node, once hoisted, is queued for further descent; its own children evaluate against root's slots rather than against the (now-empty) intermediate parent. - **O(1) root-slot lookup.** A side `HashMap<String, RcByPtr<HoisterResult>>` mirrors root's direct deps, so the per-edge "is this name taken?" check doesn't degrade to O(N²) `IndexSet` scans on flat graphs. ### Fail-fast guards The algorithm doesn't yet model peers, hoisting limits, or multi-importer (workspace) hoist trees. Rather than emit a layout pnpm would reject, the wrapper refuses these inputs upfront with three new `HoistError` variants: - `UnsupportedPeerDependency { ident, peers }` — fires when scanning the constructed `HoisterTree` finds any node with non-empty `peer_names` (either `peerDependencies` from the `packages:` map or `transitive_peer_dependencies` on a `snapshots:` entry). - `UnsupportedHoistingLimits { len }` — non-empty `opts.hoisting_limits`. - `UnsupportedWorkspace { extra_importers }` — any importer beyond the root `.`. Each carries enough context for an operator to identify what triggered it. The `UnsupportedWorkspace` help points at `SymlinkDirectDependencies` — workspaces *do* work in pacquet's wider install path (workspace support landed in pnpm#443 for the isolated linker); only the hoister is restricted. ### Rebase pickup `collect_importer_deps` carries the `Link`-variant skip introduced by pnpm#443 (workspace support) — `spec.version.as_regular()` extracts the snapshot key for `Regular` deps and `continue`s past `Link` deps, since workspace siblings are direct symlinks materialised by `SymlinkDirectDependencies` and have no snapshot to hoist. ### Performance Nothing in this PR is reachable from `pacquet install` today (`crates/package-manager/src/install*.rs` doesn't import `pacquet-real-hoist` — the crate is dead code from the install pipeline's perspective until the hoisted-linker wiring lands in later slices). The benchmark results bear that out: cold Frozen Lockfile is within CI variance (+3.2% mean, +3.5% median, driven by one outlier), Hot Cache is 6% *faster* (same `stat → lstat` improvement that shipped in slice 1's `import_indexed_dir`), micro is identical. No code path explains a real regression. ## What's deferred - Peer-dependency-aware hoisting (`peer_names` constraints, peer-promise satisfaction). - Multi-round convergence — the BFS handles deep chains in one pass, so the cases requiring true multi-round are limited to peer interactions. - `hoistingLimits` runtime enforcement. - `dependencyKind` distinctions for workspaces and external soft links (today only `ExternalSoftLink` placeholders are added by the wrapper and stripped post-hoist; `Workspace`-kind nodes are blocked by the `UnsupportedWorkspace` guard upfront). ## Upstream reference - [`hoist.ts` algorithm overview](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L1-L51) — the recipe pacquet's single-pass BFS approximates. - [`hoistTo` main loop](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L329) — the structural intent the port mirrors for the subset above.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…npm#461) ## Summary Lifts the previous `UnsupportedPeerDependency` upfront guard and replaces it with the real peer check from upstream's `getNodeHoistInfo`. The hoister now accepts lockfiles whose packages declare peer dependencies and produces the layout pnpm would: a peer-constrained dep only floats up to the root when doing so doesn't change which package its peers resolve to. ## How it works - `HoisterResult` carries the input `peer_names` set forward. Upstream's `HoisterResult` doesn't (peer info lives on its intermediate `HoisterWorkTree`); pacquet runs the algorithm on `HoisterResult` directly, so the peer set rides along. - The hoist driver is a recursive `hoist_subtree` that walks the result graph depth-first. Each recursion receives the candidate parent's *current* ancestor path (`Vec<Rc<HoisterResult>>` exclusive of the parent). After a child's hoist decision is applied, the path passed into the child's recursion reflects its *new* position — a freshly-hoisted child recurses with `[root]`, a child that stayed nested recurses with `parent_path + [parent]`. (An earlier version of this PR used BFS with paths captured at queue time; that path went stale whenever a node was hoisted between being queued and dequeued, leading to over-refused hoists. The recursive form was Copilot's recommendation in code review; the bug regression is pinned by `peer_check_uses_post_hoist_ancestor_path_not_queue_time_path`.) - `would_shadow_peer` walks that path to decide whether a candidate hoist would change its own peer resolution. - New `AbsorbDecision::PeerShadow` variant alongside `Free` / `SameNode` / `Conflict`. Fires when: 1. Root declares the candidate's own name as a peer (root-shadow guard — vacuous today since the wrapper's `.` root has empty `peer_names`, kept for parity). 2. For each peer name `P` the candidate declares, the closest ancestor providing `P` does so with a different ident than the root's `P`. Promoting the candidate would silently re-resolve the peer. - The previous `UnsupportedPeerDependency` error variant and `find_first_peer_constrained` upfront scan are gone. The `#[non_exhaustive]` tag stays so future variants can be added without breaking callers. ## DAG-vs-tree caveat Upstream's `cloneTree` duplicates the work tree into a strict tree per parent path, so each candidate visit has a unique ancestor chain. Pacquet preserves the DAG (its identity-dedup is a feature) and records only the path DFS actually used to reach the candidate. In the rare case where the same `Rc` is reachable through both a peer-compatible and a peer-shadowing path, pacquet refuses to hoist — the layout ends up at most over-nested, never under-nested. The trade-off is documented on `would_shadow_peer`. ## Upstream reference - [`hoist.ts:414`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L414) — root-shadow guard. - [`hoist.ts:454-479`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L454-L479) — ancestor-path peer check. - [`hoist.ts:670`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L670) — `cloneTree`, the per-path-tree shape pacquet skips. ## Test plan - [x] `peer_constrained_node_stays_under_parent_when_root_provides_different_ident` — `app → widget (peer: react) + react@17`, `root → react@18`. widget cannot hoist because hoisting would change its peer to react@18. - [x] `peer_constrained_node_hoists_when_ancestor_and_root_agree` — same shape but `app → react@18` matches root's `react@18` (shared `Rc` via wrapper dedup), so widget hoists freely. - [x] `peer_check_uses_post_hoist_ancestor_path_not_queue_time_path` — regression for the BFS-stale-path bug Copilot caught. `root → app → mid → terminal` (peer: `react`) where `mid` hoists past a conflicting `app.react@17`. Under the previous BFS the stale path `[root, app, mid]` made `terminal`'s peer check trip on `app.react@17 ≠ root.react@18` and refuse the hoist; under DFS the actual post-hoist path `[root, mid]` finds no provider mismatch and `terminal` correctly flattens to root. - [x] All 10 pre-existing tests still pass (`one_transitive_dep_hoists_to_root`, `diamond_dep_hoists_once_to_root`, `version_conflict_keeps_loser_at_parent`, `deep_chain_flattens_in_one_pass`, `transitive_npm_alias_resolves_target_snapshot`, `non_empty_hoisting_limits_surfaces_unsupported`, `multi_importer_lockfile_surfaces_unsupported_workspace`, `external_dependencies_are_stripped_from_the_result`, `empty_lockfile_yields_empty_root`, `hoist_throws_on_broken_lockfile`). - [x] The previously-failing `peer_dependency_in_lockfile_surfaces_unsupported` test from pnpm#452 is replaced (peers are no longer an unsupported input). - [x] All Copilot review threads addressed and resolved (stale ancestor path, misleading comment). - [x] `just ready` (836 tests pass), `cargo doc --document-private-items`, `taplo format --check`, `just dylint` all clean.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…slice 3 close-out) (pnpm#465) Closes out the Slice 3 hoist-algorithm port. Two additions on top of slice 3c's peer-aware hoist: 1. **Multi-round convergence.** The DFS runs in a fixed-point loop — a peer-shadow refusal in one round can be reconsidered in a later round once the blocking ident has moved out of the ancestor chain (or a previously-empty root slot has been filled with a compatible version). Bounded by O(N) rounds since every move is one-way. 2. **`hoistingLimits` enforcement.** The upfront `UnsupportedHoistingLimits` guard is gone; the algorithm now reads `opts.hoisting_limits[root_locator]` and refuses to hoist names listed there. New `AbsorbDecision::Border` variant alongside `Free` / `SameNode` / `Conflict` / `PeerShadow`. Plus a backfill of behavior-pinning tests ported from `@yarnpkg/nm/tests/hoist.test.ts`. ## How multi-round works - `hoist_subtree` returns `bool` for whether it moved any node in the current round. - `hoist_into_root` wraps it in a `loop` that resets `visited` each iteration and exits when a round makes no moves. - Mirrors upstream `hoistTo`'s `do { hoistGraph(); } while (anotherRoundNeeded)` without the per-candidate `dependsOn` bookkeeping; pacquet's coarse re-walk catches the same unlock cases at the cost of revisiting unchanged nodes. Canonical case (`multi_round_unlocks_peer_friendly_hoist_after_blocker_moves`): `root → app → {widget(peer: x), x@1}` with root carrying no `x`. - **Round 1.** Alphabetical iteration visits `widget` first. App supplies `x@1`, root has no `x` → mismatch → `PeerShadow`, widget stays at app. Then `x` is evaluated: `Free`, hoist to root. - **Round 2.** Reconsider widget. App no longer has `x`; root has `x@1`. No ancestor disagreement → `Free`. Widget hoists. - **Round 3.** No moves; loop exits. ## How hoistingLimits works - Computed locator: `"{ident_name}@{reference}"` — for the wrapper's root that's `".@"`, matching pnpm's keying convention. - Border-name set: `opts.hoisting_limits.get(root_locator)`, defaulting to empty when absent. - New `AbsorbDecision::Border` is layered between the basic decision and the peer check. Names in the set never hoist; they stay where the lockfile placed them. - Mirrors upstream's `isHoistBorder` flag set during `cloneTree`. ## Upstream test ports From `yarnpkg/berry@4287909fa6` `packages/yarnpkg-nm/tests/hoist.test.ts`, expressed via lockfile fixtures so they exercise the wrapper too: - **`hoisting_limits_keeps_blocked_name_at_parent`** — `should not hoist packages past hoist boundary`. - **`hoisting_limits_blocks_multiple_names`** — `should not hoist multiple package past nohoist root`. - **`hoisting_limits_keyed_on_unrelated_importer_is_inert`** — non-interference sanity check. - **`self_dependency_does_not_loop`** — `should tolerate self-dependencies`. - **`basic_cyclic_dependency_terminates`** — `should support basic cyclic dependencies`. ## Documented gaps The `nm_hoist` doc-comment now lists what's deferred *intentionally* rather than "not done yet": - **Popularity-based ident preference.** Upstream's `buildPreferenceMap` picks the ident with more incoming references when two distinct deps share an alias; pacquet picks the first-visited. Affects the handful of upstream test cases that exercise the tie-break (`should honor package popularity when hoisting`, etc.). - **Multi-importer workspace hoist trees.** Still guarded upfront by `HoistError::UnsupportedWorkspace`. Workspace-aware hoisting needs per-importer roots and a different output shape. - **`ExternalSoftLink` descendants.** The wrapper only creates soft-links as zero-children placeholders, so upstream's "only-hoist-when-all-descendants-hoist" rule has nothing to delay today. Tests for `should not hoist portal with unhoistable dependencies` and similar were skipped. ## Upstream reference Aligned with `yarnpkg/berry@4287909fa6`: - [`hoist.ts:109-119`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L109-L119) — outer round loop in `hoist`. - [`hoist.ts:352-367`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L352-L367) — inner round loop in `hoistTo`. - [`hoist.ts:707`](https://github.com/yarnpkg/berry/blob/4287909fa6a0a1ec976a55776bff606864b31990/packages/yarnpkg-nm/sources/hoist.ts#L707) — `isHoistBorder` flag during `cloneTree`.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
pnpm#473) Slice 2 of pnpm#438 (the `nodeLinker: 'hoisted'` umbrella). Smaller than expected — when I looked, the schema field was already in place from pnpm#332's original `.modules.yaml` port, so the work reduced to pinning behavior rather than building plumbing. Two round-trip tests + a doc-comment on the field. ## What was actually missing The umbrella's §"Pacquet's current state" table claimed `hoistedLocations` was "Missing entirely" — that was wrong. The field is at `crates/modules-yaml/src/lib.rs:212` and serde-handles read/write correctly. What was missing: - **A test pinning the wire shape.** Nothing was stopping a future edit from breaking the camelCase mapping, the optional-with-skip-on-None behavior, or the `Record<string, string[]>` shape upstream relies on. - **A doc-comment** explaining what the field is for. Other Slice-related fields (`hoisted_dependencies`, `hoisted_aliases`) have full docstrings; this one was silent. ## Tests added Both live in `crates/modules-yaml/tests/real_fs.rs` (pacquet-side, no upstream counterpart — upstream's `installing/modules-yaml/test/index.ts` doesn't exercise `hoistedLocations` directly): - `hoisted_locations_round_trips` — a manifest with `hoistedLocations` populated survives write+read; the raw on-disk JSON keeps the per-depPath array shape (multi-entry arrays included, to pin the nested-conflict layout). - `absent_hoisted_locations_is_omitted_on_write` — `None` (the state every pacquet install writes today) serializes as the field *absent* from the file, matching upstream's `JSON.stringify(undefined)` behavior. Guards against accidental `Option::is_some` regressions on the `skip_serializing_if`. ## Regression catch verified Per `plans/TEST_PORTING.md`'s "break the subject to verify the test catches it" convention, I temporarily added `#[serde(rename = "wrongName")]` to the field. `hoisted_locations_round_trips` failed at the `expect("present")` on the deserialized value (the camelCase key no longer mapped). Reverted before commit. ## Type-shape decision Upstream's actual `ModulesRaw.hoistedLocations` is `Record<string, string[]> | undefined` — *not* `Record<DepPath, string[]>` despite the values being populated from depPaths internally. The umbrella's scope item ("`Option<BTreeMap<DepPath, Vec<String>>>`") was inconsistent with the upstream schema; I kept the existing `BTreeMap<String, Vec<String>>` because: 1. It already matches the upstream wire shape exactly. 2. Same pattern as the `hoisted_dependencies` field below, which deliberately keeps `String` keys (per its doc-comment at lines 148-153) because pnpm's `DepPath | ProjectId` union can't be statically disambiguated.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…e 4a) (pnpm#478) * feat(package-manager): hoisted dep-graph type skeleton (pnpm#438 slice 4a) Defines the directory-keyed dependency graph types used by the hoisted-linker install path. Types only — the walker that fills them from a lockfile lands in a follow-up. Ported from upstream: - `DependenciesGraphNode` mirrors deps/graph-builder `DependenciesGraphNode` minus the `fetching` / `files_index_file` fields that only the store-controller-bound walker can populate. - `DependenciesGraph` = `BTreeMap<PathBuf, DependenciesGraphNode>`, keyed by absolute directory path (unlike pacquet's existing depPath-keyed `deps_graph` module — hoisted nodes can occupy several directories when a name conflict forces nesting). - `DepHierarchy` is the recursive directory tree the linker walks to decide population order and which `<dir>/node_modules/.bin` to wire up. Newtype-wrapped because Rust doesn't allow recursive type aliases. - `DirectDependenciesByImporterId` is the per-importer alias-to-directory table the linker hands to the bin pass. - `LockfileToDepGraphResult` bundles everything the walker returns to the install pipeline, including `prevGraph` for the orphan-diff pass and `injectionTargetsByDepPath` for the injected-workspace re-mirror step. - `LockfileToHoistedDepGraphOptions` carries the subset of upstream's options that the to-be-ported walker actually reads today; fields tied to the store controller, fetch concurrency, and workspace project list will be added when their consumers land. Smoke tests cover empty-result default construction, node insertion by `dir`, recursive hierarchy nesting, and default-options shape. Upstream: - <https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/lockfileToHoistedDepGraph.ts> - <https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts> * docs(package-manager): v9 depPath in 4a tests + map-key typing notes (pnpm#478 review) - Replace v5-era `/accepts/1.3.7` depPath strings in the smoke test with v9-shape `accepts@1.3.7` (the format pacquet's `PkgNameVerPeer` produces and the lockfile snapshots use). The legacy `/name/version` shape is only kept for read-side `hoistedAliases` compatibility. Caught by Copilot. - Tighten doc-comments on `hoisted_locations`, `injection_targets_by_dep_path`, and `skipped` to spell out *why* the keys are plain `String` (not `DepPath`): upstream types those fields with raw `string` in the hoisted-specific interface, even though the values are populated from depPaths. Mirrors upstream's literal typing.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…npm#486) Sub-slice 4b of pnpm#438. Implements `lockfile_to_hoisted_dep_graph(lockfile, opts) -> LockfileToDepGraphResult` — the walker that consumes the Slice 3 hoister output and produces the directory-keyed graph + hierarchy + hoisted_locations + direct-deps-by-importer-id + injection-targets-by-dep-path that 4a (pnpm#478) pinned the type shape of. Single-importer only (multi-importer lockfiles surface as `HoistError::UnsupportedWorkspace` via the underlying hoister). ## What's deferred Three things are intentionally left to follow-ups so 4b stays focused: - **Store fetch wiring** — `fetching` / `files_index_file` on the graph node are still defaulted. 4c will wire `StoreController::fetchPackage` (and the skip-fetch optimization that consults `currentHoistedLocations`). - **Installability check** — the walker honors `opts.skipped` as input but doesn't run `packageIsInstallable` to add new entries. 4c will plumb `package_is_installable` from `pacquet-package-is-installable` (already used by the isolated linker path). - **`prev_graph` diff** — `prev_graph` is `None`. 4d will walk the *current* lockfile alongside the wanted one (using `force: true` + empty `skipped` per upstream) and produce the orphan-removal input Slice 5's linker consumes. ## Two-pass design Upstream's `fetchDeps` walks asynchronously via `await Promise.all(deps.map(async (dep) => { ... }))`. The key invariant: each sibling's async function runs its sync prologue (insert + push to `pkgLocationsByDepPath`) before any continuation fires, because `Promise.all` sync-invokes each function and the inner `await fetchDeps(...)` only yields after the prologue completes. So by the time any node's post-recursion `graph[dir].children = getChildren(...)` line runs, every node in the entire tree is already in `pkgLocationsByDepPath`. Pacquet runs synchronously. To preserve that invariant cleanly: 1. **Pass 1 (recursive walk).** Insert each node into the graph with `children: BTreeMap::new()` and push its dir to `pkg_locations_by_dep_path`. Recursion order matches upstream's outer body (insert → push → recurse → push to `hoistedLocations`). 2. **Pass 2 (`fill_children`).** Iterate every graph node and resolve `children: alias → dir` from the snapshot's `dependencies` + `optionalDependencies` via `SnapshotDepRef::resolve`, consulting the now-complete `pkg_locations_by_dep_path`. A single-pass walker that computed children inline produced incorrect `children` maps for hoisted transitive deps — the canonical case `root → a → b` with `b` flattened to root left `a.children["b"]` empty because `b`'s pkg_locations entry hadn't been recorded yet. Caught by `walker_transitive_dep_flattens_under_root` before the refactor. ## Tests - `walker_empty_lockfile_produces_empty_result` — empty importer → empty graph, with the `"."` root importer key still present. - `walker_single_root_dep_emits_one_node` — `root → a`, node at `<lockfile_dir>/node_modules/a`. - `walker_transitive_dep_flattens_under_root` — `root → a → b`: `b` hoists to root, `a.children["b"]` points at the root-level dir. - `walker_version_conflict_keeps_loser_nested` — `root → {a@1, c}` with `c → a@2`: `a@1` at root, `a@2` nested under `c`; `hoisted_locations` records both directories; `c.children["a"]` points at the nested copy. - `walker_honors_pre_skipped_dep_path` — depPaths in `opts.skipped` are dropped from the walk entirely (and not recorded in `hoisted_locations`). - `walker_records_directory_resolution_as_injection_target` — `directory:` resolutions populate `injection_targets_by_dep_path` for the post-install re-mirror pass.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
… slice 4c) (pnpm#491) Sub-slice 4c of pnpm#438. Wires `package_is_installable` into the hoisted-graph walker shipped in 4b (pnpm#486), so optional packages on unsupported platforms get filtered into `result.skipped` and engine-strict mismatches surface as typed errors. Single-importer only; store I/O and `prev_graph` diff still to come (4d, then 5). ## Behavior Mirrors upstream's `if (!opts.force && packageIsInstallable(...) === false)` gate at [lockfileToHoistedDepGraph.ts:200-211](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/lockfileToHoistedDepGraph.ts#L200-L211): | Upstream return | `package_is_installable` (Rust) | Walker action | |---|---|---| | `null` (no constraint) | `Ok(Installable)` | emit node | | `true` (warn but proceed) | `Ok(ProceedWithWarning)` | emit node (warning emit deferred) | | `false` (optional + incompatible) | `Ok(SkipOptional)` | add to `result.skipped`, skip node | | throws `UnsupportedEngineError` (strict) | `Err(InstallabilityError::Engine)` | `HoistedDepGraphError::Installability` | | throws `InvalidNodeVersionError` | `Err(InstallabilityError::InvalidNodeVersion)` | `HoistedDepGraphError::Installability` | ## New options on `LockfileToHoistedDepGraphOptions` - `force: bool` — bypass the check entirely. Used by Slice 4d's `prev_graph` walk, where the previous lockfile is replayed wholesale so the orphan diff catches packages that would now filter out. Mirrors upstream's `force` at [lockfileToHoistedDepGraph.ts:73](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/lockfileToHoistedDepGraph.ts#L73-L76). - `engine_strict`, `current_node_version`, `current_os`, `current_cpu`, `current_libc`, `supported_architectures` — host-derived axes the check consumes. ## New output on `LockfileToDepGraphResult` - `skipped: BTreeSet<String>` — the input `opts.skipped` cloned and extended with depPaths added during the walk. Upstream mutates the input set in place; pacquet returns the augmented set so the caller can persist it into `.modules.yaml.skipped` without sharing mutable state. ## Tests 15 walker tests total — the 10 from 4b survive (one extended to assert the input depPath survives into output `skipped`), plus five new ones: - `walker_skips_optional_dep_on_unsupported_platform` — Linux host, package targets darwin only, `optional: true` → added to `result.skipped`, no graph entry, no `hoisted_locations`. - `walker_emits_required_dep_with_unsupported_platform_as_warning` — same shape but `optional: false` → walker proceeds (matches upstream `packageIsInstallable === true`); warning log emit is out of scope for 4c. - `walker_errors_on_engine_strict_mismatch` — `engine_strict: true` + `engines.node = ">=99.0.0"` → `HoistedDepGraphError::Installability`. - `walker_force_bypasses_installability_check` — `force: true` emits an incompatible required dep without erroring. - `walker_emits_compatible_dep` — sanity: compatible host + no constraint mismatch → graph entry, no skip.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…k.yaml (pnpm#434 slice 6) (pnpm#495) Pacquet wrote the raw wanted lockfile as the current lockfile. Upstream pnpm writes a filtered version with optional + skipped subtrees pruned and `include` flags applied, so the next install diffs against what was actually materialized rather than the resolver's full ambition. Without the prune, dropped snapshots (slice 1 installability, slice 4 fetch failures, slice 5 `--no-optional`) were claimed present in the current lockfile and the follow-up install would skip work that should have run. Ports <https://github.com/pnpm/pnpm/blob/94240bc046/lockfile/filtering/src/filterLockfileByImportersAndEngine.ts#L46-L94> → <https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L687-L695>. Pacquet-specific simplification: instead of re-running the engine + supportedArchitectures + skipped checks at filter time, `filter_lockfile_for_current` reuses the `SkippedSnapshots` set already produced by the install pipeline. Its three subsets (`installability`, `fetch_failed`, `optional_excluded`) are the exact set upstream's filter would drop — same observable result, no duplicated walk. ## Changes - **`pacquet-lockfile`**: `Clone` derives on `Lockfile`, `LockfileSettings`, `ProjectSnapshot`, `SnapshotEntry`, `PackageMetadata`, `PeerDependencyMeta`, `ResolvedDependencySpec`. Needed to build a filtered lockfile by clone-and-mutate. - **`pacquet-package-manager/src/current_lockfile.rs`** (new): `filter_lockfile_for_current(lockfile, included, skipped) -> Lockfile`. Three-phase filter: 1. BFS the snapshot graph from filtered importer roots, skipping keys in `skipped`. Produces the reachable set. 2. Per-importer: clear dep maps whose `include` flag is false; trim `optional_dependencies` to entries whose target survived. 3. `snapshots:` / `packages:` filtered to the reachable set (packages key off `without_peer()` so peer-variant survivors keep their shared metadata row). - **`install.rs`**: replace the raw `save_current_to_virtual_store_dir` call with `filter_lockfile_for_current(...).save_current_to_virtual_store_dir(...)`. ## Tests 8 unit tests covering each filter behavior: - `skipped_snapshot_pruned_from_snapshots_and_importer_optional` - `include_optional_false_clears_importer_section` - `transitive_under_skipped_snapshot_is_pruned` - `snapshot_reachable_via_kept_path_survives` (mirrors upstream's `:712` shared-dep case at the filter level) - `packages_filtered_to_surviving_metadata_keys` (peer-variant metadata sharing) - `link_optional_entries_survive_post_filter` (workspace link: deps don't get post-filtered) - `empty_skipped_and_full_include_is_identity_for_reachables` (baseline) - `orphan_snapshots_are_pruned` (snapshots unreachable from any importer get dropped) Test-the-test verified by breaking the BFS walker — two tests fail. ## Out of scope - Hoisted-linker current-lockfile variant (`:633`) — pacquet's hoisted node-linker isn't fully wired through the lockfile-write path yet; tracked separately under pnpm#438. - `pnpm install --filter` slicing — pacquet has no `--filter` yet. Closes pnpm#493.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…slice 4d) (pnpm#494) * feat(package-manager): prev_graph diff from current lockfile (pnpm#438 slice 4d) `lockfile_to_hoisted_dep_graph` now takes an optional `current_lockfile: Option<&Lockfile>` and populates `result.prev_graph` from a second walk over that lockfile. Ports upstream's wrapper at installing/deps-restorer/src/lockfileToHoistedDepGraph.ts. When the current lockfile is `Some(lf)` with a non-empty `packages:` map, the second walk runs with `force: true` and `skipped: BTreeSet::new()`. Force matters: an orphan that landed under the previous install but would now fail installability (e.g., the host changed platforms) still surfaces in `prev_graph` so Slice 5's linker can find and rimraf the stale directory. Empty skipped matters: the previous install's own filter set is unrelated to "which directories still exist on disk." API change - The public `lockfile_to_hoisted_dep_graph(lockfile, opts)` signature gains a middle `current_lockfile: Option<&Lockfile>` argument. Single existing caller (the tests) updated to pass `None`. Splits the previous body into a private `build_dep_graph` helper that the wrapper calls once or twice depending on whether a current lockfile is present. prev_graph shape - `None` when no current lockfile is supplied, or when the supplied lockfile has no `packages:` map (a brand-new install in progress). Mirrors upstream's `prevGraph = {}` fallback — pacquet uses `None` rather than an empty map, but the linker treats both the same. - `Some(graph)` otherwise, keyed by directory just like the wanted-lockfile graph. Tests - `prev_graph_none_when_current_lockfile_absent` — no current lockfile → `prev_graph` is `None`, wanted graph still produced. - `prev_graph_none_when_current_lockfile_has_no_packages` — current lockfile with `packages: None` → `prev_graph` is `None`. - `prev_graph_contains_orphan_from_current_only_lockfile` — package in current but not wanted appears in `prev_graph`, not in `graph`. - `prev_graph_includes_orphan_even_when_now_incompatible` — darwin-only orphan in the current lockfile, host is linux, wanted lockfile is empty: `prev_graph` still contains it (proves `force: true` overrides the installability check), and the wanted-walk's `skipped` stays empty (proves the prev-walk's skipped set is independent). All 4 new tests pass alongside the 15 walker tests from 4a-4c. * fix(package-manager): collapse empty current packages to None for prev_graph (pnpm#494 review) `current.packages.is_some()` matched `Some(empty_map)` too, causing 4d to do an unnecessary second walk and return `Some(empty_graph)` for a case the doc-comment described as `None`. Tighten the guard to require a non-empty `packages` map. Pacquet uses `Option<DependenciesGraph>` for `prev_graph` (upstream uses an always-present `DependenciesGraph` with `{}` for the no-current case). The no-packages and empty-packages cases both produce the same observable behavior — "no orphans to consider" — so they should share one representation rather than be inconsistent. The doc-comment's stated contract was `None`; the code now matches. Added `prev_graph_none_when_current_lockfile_has_empty_packages` to pin the empty-map case. Caught by Coderabbit on pnpm#494. * style(package-manager): rustfmt the 4d follow-up
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…npm#505) * feat(package-manager): linkHoistedModules linker (pnpm#438 slice 5) New module `crates/package-manager/src/link_hoisted_modules.rs` produces the on-disk `node_modules/` tree from Slice 4's `LockfileToDepGraphResult`. Ports installing/deps-restorer/src/linkHoistedModules.ts. ## Three phases 1. **Orphan removal.** Every directory in `prev_graph` but not in `graph` is silently `rimraf`'d before any insert. `try_remove_dir` swallows all errors (NotFound + PermissionDenied + EBUSY), matching upstream's `tryRemoveDir` tolerance. 2. **Per-node import.** Hierarchy walked top-down, rayon-parallel at each level. Every node goes through `import_indexed_dir` with `force: true, keep_modules_dir: true` — the hoisted-linker call shape Slice 1 was designed for. 3. **Per-`node_modules` bin link.** After a level's children are all imported, `link_direct_dep_bins` populates `<parent>/node_modules/.bin` from the just-materialized direct children. Bin link runs only after every child's subtree is done so package.json reads always see the fully-populated package. ## API shape ```rust pub fn link_hoisted_modules<R: Reporter>( opts: &LinkHoistedModulesOpts<'_>, ) -> Result<(), LinkHoistedModulesError>; ``` `LinkHoistedModulesOpts` borrows `graph`, `prev_graph`, `hierarchy`, and a `cas_paths_by_pkg_id: HashMap<String, HashMap<String, PathBuf>>` keyed by `pkg_id_with_patch_hash`. ## Decoupling from the store Upstream's linker is async and calls `storeController.fetchPackage()` inline during the walk — pacquet's is sync and takes pre-fetched CAS paths. Slice 6 (the install pipeline) is responsible for fetching every package through pacquet's existing tarball / store-dir / git-fetcher machinery before invoking the linker. This keeps the linker focused on file-system layout and reuses the proven fetch chain that the isolated path already exercises. ## Optional-dep tolerance A graph node whose `pkg_id_with_patch_hash` is missing from `cas_paths_by_pkg_id`: - If `node.optional`: silently skipped, no directory created. Mirrors upstream's `if (depNode.optional) return` on fetch failure. - Otherwise: surfaces as `LinkHoistedModulesError::MissingCasPaths { pkg_id, dir }`. ## Tests 7 real-tempdir tests in `link_hoisted_modules/tests.rs`: - `import_pass_creates_package_directory` — single-package smoke. - `orphan_directory_is_removed` — `prev_graph` diff produces rimraf of stale directory; planted contents are gone after. - `nested_hierarchy_materializes_inner_node_modules` — version-conflict layout; loser ends up at `<outer>/node_modules/<inner>`. - `missing_cas_for_required_dep_errors` — required + missing CAS → typed `MissingCasPaths` error. - `missing_cas_for_optional_dep_skips_silently` — optional + missing CAS → no error, no directory. - `no_prev_graph_skips_orphan_pass` — fresh install (no prior lockfile) path. - `orphan_already_removed_is_tolerated` — phantom orphan in `prev_graph` not present on disk doesn't error. Each test plants synthetic CAS files in a tempdir and asserts the on-disk tree after the linker runs. * fix(package-manager): fail-fast on hierarchy/graph inconsistency (pnpm#505 review) The hierarchy walk silently skipped entries missing from `graph`, which would produce a partial install layout instead of surfacing the bug. Slice 4's walker keeps the two in sync today, but a future bug there shouldn't yield a partial tree. Add `LinkHoistedModulesError::MissingGraphNode { dir }` and return it when a hierarchy directory has no graph entry. Upstream effectively errors here too — its `graph[dir].fetching` read would throw `Cannot read properties of undefined` — pacquet just spells the failure out. Regression test `hierarchy_entry_missing_from_graph_errors` exercises the path with an empty graph and a hierarchy referencing a phantom dir. Caught by Coderabbit.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
Add the `--node-linker [isolated|hoisted|pnp]` CLI flag to `pacquet install`, mirroring pnpm's flag. Overrides `Config.node_linker` for the invocation; absent flag → config's yaml/npmrc value wins. The CLI parses into a `NodeLinkerArg` mirror enum (kept in the CLI crate so `pacquet-config` stays free of `clap`), then `into_config()` converts to the canonical `pacquet_config::NodeLinker`. Threaded into the `Install` runner as an explicit `node_linker: NodeLinker` field rather than reading `config.node_linker` directly. Matches the existing pattern `supported_architectures` uses (`state.config` is `&'static`, so the override-merge happens at the CLI layer and lands here as a fully-resolved value). `build_modules_manifest` consumes it on the `.modules.yaml.nodeLinker` write so the persisted setting reflects the invocation, not just the config. `pacquet add` uses Config's value by default (`add` doesn't expose `--node-linker` per the umbrella scope; Slice 6's pipeline integration will revisit if needed). 5 new CLI tests: - `node_linker_default_is_none` — flag absent → field is None. - `node_linker_hoisted` / `node_linker_isolated` / `node_linker_pnp` — each value parses and round-trips through `into_config()`. - `node_linker_invalid_value_rejected` — clap rejects unknown values with the bad value in the error message. - `node_linker_arg_into_config_matches_every_variant` — every ValueEnum variant has a canonical mapping (compile-fails on future variants that forget the mapping). `NodeLinker` gains `Clone + Copy + Eq` so it can be threaded by value into `Install` and matched in tests.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…npm#438 slice 6) (pnpm#518) * feat(package-manager): wire node_linker hoisted into install pipeline (pnpm#438 slice 6) Branches `Install::run` / `InstallFrozenLockfile::run` on `config.node_linker == Hoisted`. Threads the slice 4 `lockfile_to_hoisted_dep_graph` walker output and the per-package CAS index produced by `CreateVirtualStore` (slot writes skipped under hoisted) into the slice 5 `link_hoisted_modules` linker. Persists the walker's `hoisted_locations` into `.modules.yaml` for rebuild and follow-up installs to consume. Pipeline changes under hoisted: - `CreateVirtualStore` skips `CreateVirtualDirBySnapshot` for both warm and cold batches, collects each snapshot's CAS file index keyed by `PkgIdWithPatchHash` into a new `cas_paths_by_pkg_id` output field. - `InstallPackageBySnapshot::run` returns the per-package CAS map unconditionally and skips the virtual-store-slot write when its new `node_linker` field is `Hoisted`. - `InstallFrozenLockfile::run` skips `SymlinkDirectDependencies`, `LinkVirtualStoreBins`, the isolated hoist pass, and `BuildModules` under hoisted; runs `lockfile_to_hoisted_dep_graph` + `link_hoisted_modules` in their place. Folds the walker's augmented skip set back into the install-time `SkippedSnapshots` so `.modules.yaml.skipped` reflects the union. - `Install::run`'s `build_modules_manifest` now takes the walker's `hoisted_locations` and writes it through `Modules.hoisted_locations` (only when non-empty so the isolated path doesn't produce a hoisted-only key). The build phase under hoisted (rebuild over `hoistedLocations` with ancestor-`.bin` lookup, `MISSING_HOISTED_LOCATIONS`) is slice 7's scope and is intentionally left as a no-op here. Workspace and `hoistingLimits` / `externalDependencies` knobs are slices 9-10. Mirrors upstream's hoisted branch in `headlessInstall` at https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L369-L425. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(package-manager): docs intra-link disambiguation + skip-set merge fix (pnpm#438 slice 6) Docs CI was failing because `[`crate::link_hoisted_modules`]` is ambiguous between the function and the module of the same name. Disambiguate by switching to the function form `[`crate::link_hoisted_modules()`]` everywhere. Slice 6's hoisted-walker skip-set merge previously folded every entry in `walker_result.skipped` into `SkippedSnapshots::insert_installability`, which would promote pre-existing transient skips (`fetch_failed` / `optional_excluded`) into the persisted-on-disk `.modules.yaml.skipped` set. Diff against the input `walker_skipped` so only walker-discovered (genuinely-new) installability skips flow into the persisted subset.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
… slice 7) (pnpm#520) Re-enables `BuildModules` for `nodeLinker: hoisted` installs. Slice 6 landed the hoisted install pipeline but skipped the build phase entirely because `BuildModules` walked virtual-store slot directories that don't exist under hoisted. Slice 7 routes the build phase through the slice 4 walker's per-node `dir` so postinstall scripts can run against the on-disk hoisted tree. Changes: - `BuildModules` gains two fields: `pkg_root_by_key: Option<&HashMap<PackageKey, PathBuf>>` overrides the per-snapshot pkg_root lookup with the slice 4 walker's `DependenciesGraphNode::dir` values; `gather_ancestor_bin_paths: bool` switches `extra_bin_paths` to the new `bin_dirs_in_all_parent_dirs` helper, a port of upstream's `binDirsInAllParentDirs` at https://github.com/pnpm/pnpm/blob/94240bc046/building/after-install/src/index.ts#L476-L487. - `pkg_root_for_key` helper: routes between the layout-based slot computation (isolated) and the override map (hoisted). Hoisted snapshots absent from the map (walker-dropped) take the same exit as the isolated `!pkg_dir.exists()` skip. - `InstallFrozenLockfile::run` now builds a snapshot-key → first-recorded-dir map from `walker_result.graph.values()` and threads it (plus `gather_ancestor_bin_paths: true`) into `BuildModules` instead of skipping the phase. Multiple graph nodes with the same dep_path collapse to the first entry, matching upstream's `pkgRoots[0]` pick at https://github.com/pnpm/pnpm/blob/94240bc046/building/after-install/src/index.ts#L348. - New private `HoistedLinkerOutput` struct bundles `hoisted_locations` and `pkg_root_by_key` so the hoisted-branch return doesn't trip `clippy::type_complexity`. Side-effects-cache key shape is unchanged — it's keyed by `pkg_id_with_patch_hash` + dep-graph hash, both layout-independent (`crates/graph-hasher/src/dep_state.rs`). `MISSING_HOISTED_LOCATIONS` is intentionally deferred — pacquet has no `rebuild` command, so the install path always re-runs the walker and never reads `.modules.yaml.hoisted_locations`. Tracked as a follow-up for when `pacquet rebuild` lands. Workspace-aware hoisting (slice 9) and `hoistingLimits` / `externalDependencies` (slice 10) remain. Tests: - Three new helper tests pin `bin_dirs_in_all_parent_dirs` against top-level, conflict-nested, and scoped-package shapes. - Three new helper tests pin `pkg_root_for_key` for the isolated pass-through, the hoisted override hit, and the hoisted-missing short-circuit.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…slice 9) (pnpm#521) Enables `nodeLinker: hoisted` for multi-importer (workspace) lockfiles. Previously the hoister rejected any lockfile with importers beyond `.` with `UnsupportedWorkspace`; now the whole workspace shares one hoist plan so conflicting versions across projects dedupe, and each importer's project-tree node_modules is materialized per-project. real-hoist: - Drop the upfront UnsupportedWorkspace guard in `hoist()`. The wrapper already constructed non-root importers as Workspace-kind children of the virtual `.` root; only the guard needed to go. - Add `HoistOpts::hoist_workspace_packages: bool` (default true). When false, non-root importers stay out of the shared tree (matches upstream's `hoistWorkspacePackages: false` mode for the Bit CLI). Removed the corresponding error variant. hoisted_dep_graph walker: - Replace the `workspace:`-prefix skip with a recurse-into branch: for each Workspace-kind hoister child, walk its post-hoist children under `<lockfile_dir>/<importer_id>/node_modules`. The workspace node itself is NOT added to the graph or to the parent's hierarchy — it has no contents to import. - Add per-importer `hierarchy` and `direct_dependencies_by_importer_id` entries. Per-importer direct deps are computed from the lockfile (not from the workspace node's post-hoist children) because hoisted-up siblings don't show up in the workspace node's tree. First-recorded location wins, matching upstream's `pkgLocationsByDepPath[depPath][0]` pick. - Add `LockfileToHoistedDepGraphOptions::hoist_workspace_packages` (default true) and plumb to HoistOpts. Config: - Add `Config::hoist_workspace_packages: bool` (#[default = true]). Read from `pnpm-workspace.yaml` via `WorkspaceSettings`. SymlinkDirectDependencies: - Add `link_only: bool` flag. When true, skip every Regular dep and only materialize Link entries (workspace siblings). Used by the hoisted branch in InstallFrozenLockfile::run so workspace-sibling symlinks land under each importer's `node_modules/<alias>` even though the regular deps now live as real directories from the hoisted linker. InstallFrozenLockfile::run: - Plumb `config.hoist_workspace_packages` into the walker. - After link_hoisted_modules, run SymlinkDirectDependencies with `link_only: true` so workspace siblings get their per-project symlinks. Mirrors upstream's hoisted branch at https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L411-L440. Tests: - real-hoist: replace the now-removed `UnsupportedWorkspace` test with one that pins multi-importer hoister output (Workspace children with percent-encoded names + `workspace:<id>` references) and one that pins the `hoist_workspace_packages: false` opt-out. - walker: three new tests cover per-importer direct_deps emission, per-importer hierarchy entries, the `hoist_workspace_packages: false` root-only mode, and version-conflict-across-importers nesting.
github-actions Bot
pushed a commit
to Eyalm321/pnpm
that referenced
this pull request
May 18, 2026
…nobs (pnpm#438 slice 10) (pnpm#522) Plumbs the two programmatic-only hoister knobs from `pnpm-workspace.yaml` through to the slice 4 walker and the slice 3 hoister. Both fields already existed on `HoistOpts`; this slice wires them end-to-end. - `Config::hoisting_limits: BTreeMap<String, BTreeSet<String>>` — per-importer block-list, locator-keyed (`'.@'` for the root). Reads `hoistingLimits: { ".@": [foo, bar] }` from yaml. Mirrors upstream's https://github.com/pnpm/pnpm/blob/94240bc046/installing/linking/real-hoist/src/index.ts#L10 programmatic-only knob, exposed as yaml for parity since the ergonomics of the locator-keyed map don't translate to a CLI flag. - `Config::external_dependencies: BTreeSet<String>` — name slots reserved at the root for an external linker (the Bit CLI is the only known consumer upstream). Reads `externalDependencies: [...]` from yaml. - `LockfileToHoistedDepGraphOptions` gains both fields and forwards them to `HoistOpts` in `build_dep_graph`. - `InstallFrozenLockfile::run` clones the two `Config` fields into the walker opts. Both knobs default to empty (no limits, no externals), matching upstream's default. Neither has any effect under `nodeLinker: isolated` — the isolated linker keeps per-importer subtrees by construction and doesn't consult the hoister. Tests: - `parses_hoisting_limits_from_yaml_and_applies` — yaml round-trip + apply_to. - `parses_external_dependencies_from_yaml_and_applies` — same. - `omitting_hoisting_limits_and_external_dependencies_keeps_defaults` — pins the apply_to skip-on-None branch so a yaml without these keys doesn't accidentally overwrite Config defaults. - `walker_forwards_external_dependencies_to_hoister` — end-to-end: the walker observes an empty graph for an externalised alias because the hoister stripped it. Pins the slice 10 plumbing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hello lovely humans,
is-ci just published its new version 1.0.10.
This version is not covered by your current version range.
Without accepting this pull request your project will work just like it did before. There might be a bunch of new features, fixes and perf improvements that the maintainers worked on for you though.
I recommend you look into these changes and try to get onto the latest version of is-ci.
Given that you have a decent test suite, a passing build is a strong indicator that you can take advantage of these changes by merging the proposed change into your project. Otherwise this branch is a great starting point for you to work on the update.
Do you have any ideas how I could improve these pull requests? Did I report anything you think isn’t right?
Are you unsure about how things are supposed to work?
There is a collection of frequently asked questions and while I’m just a bot, there is a group of people who are happy to teach me new things. Let them know.
Good luck with your project ✨
You rock!
🌴
The new version differs by 7 commits .
248e7a51.0.10b78b211Depend on ci-infoe5317eeSemaphore CI is just called Semaphore6a1dac8Don't list Visual Studio Online CIbcdaee1Test against Node.js v5 and v6f6d34e1Improve installation instructions in README.mdf3d8232Update copyright yearSee the full diff.
This pull request was created by greenkeeper.io.
Tired of seeing this sponsor message? ⚡
greenkeeper upgrade