Skip to content
This repository was archived by the owner on May 14, 2026. It is now read-only.

feat(package-manager): prev_graph diff from current lockfile (#438 slice 4d)#494

Merged
zkochan merged 3 commits into
mainfrom
feat/438-slice-4d
May 13, 2026
Merged

feat(package-manager): prev_graph diff from current lockfile (#438 slice 4d)#494
zkochan merged 3 commits into
mainfrom
feat/438-slice-4d

Conversation

@zkochan

@zkochan zkochan commented May 13, 2026

Copy link
Copy Markdown
Member

Summary

Sub-slice 4d of #438. Closes out Slice 4 by adding the prev_graph diff — the input Slice 5's linker will subtract from graph to identify orphaned directories that need removing.

What lands

lockfile_to_hoisted_dep_graph now takes an optional current_lockfile: Option<&Lockfile> and runs a second walk over it (when present and non-empty) with force: true, skipped: BTreeSet::new() to populate result.prev_graph. Ports upstream's wrapper at installing/deps-restorer/src/lockfileToHoistedDepGraph.ts:70-79.

let prev_graph = match current_lockfile {
    Some(current) if current.packages.is_some() => {
        let prev_opts = LockfileToHoistedDepGraphOptions {
            force: true,
            skipped: BTreeSet::new(),
            ..opts.clone()
        };
        Some(build_dep_graph(current, &prev_opts)?.graph)
    }
    _ => None,
};

Why force + empty skipped

Both overrides matter — neither is paranoia:

  • force: true — an orphan that landed under the previous install but would now fail installability (host platform changed, dependency was downgraded out of engine range, etc.) still has a directory on disk. Without force, the prev-walk would silently filter it out and the linker would leave the stale dir in place. Pinned by prev_graph_includes_orphan_even_when_now_incompatible.
  • skipped: empty — the previous install's filter set says which packages it didn't install, which is unrelated to which directories currently exist. Inheriting the wanted-walk's skipped would underreport prev_graph for unchanged-but-now-incompatible packages.

API change

lockfile_to_hoisted_dep_graph gains a middle current_lockfile: Option<&Lockfile> argument. The previous body splits into a private build_dep_graph helper that the public wrapper calls once or twice depending on whether a current lockfile is present. Existing tests updated (the only callers — None for the no-prev case).

prev_graph shape

current_lockfile result.prev_graph
None None
Some(lf) with lf.packages: None None
Some(lf) with lf.packages: Some(_) Some(graph)

Upstream uses prevGraph = {} for both null-current cases; pacquet uses None. The linker treats them the same (no orphans to remove on a fresh install).

Tests

Four new prev_graph tests; the 15 walker tests from 4a-4c carry over with the new signature.

  • prev_graph_none_when_current_lockfile_absent — no current lockfile → None, wanted graph still produced.
  • prev_graph_none_when_current_lockfile_has_no_packages — current lockfile with packages: NoneNone.
  • prev_graph_contains_orphan_from_current_only_lockfile — package in current but not wanted appears in prev_graph, not in graph.
  • prev_graph_includes_orphan_even_when_now_incompatible — darwin-only orphan in current, host is linux, wanted is empty: prev_graph still contains it (proves force: true works), and the wanted-walk's skipped stays empty (proves the prev-walk's skipped set is independent).

Test plan

  • All 19 walker tests pass (15 carry-over + 4 new).
  • just ready (874 → 878 tests, all green).
  • cargo doc --document-private-items, taplo format --check, just dylint clean.
  • Sabotage check: changing the prev-walk's options from force: true, skipped: empty to ..opts.clone() re-fails prev_graph_includes_orphan_even_when_now_incompatible (orphan disappears from prev_graph because it would now fail installability).

What's next

Slice 4 is done. Next on the critical path: Slice 5 — linkHoistedModules. Consumes LockfileToDepGraphResult to produce the on-disk tree: walks prev_graph minus graph to rimraf orphans, then imports each graph node from the CAS via Slice 1's import_indexed_dir primitive, then runs linkBins per node_modules/ directory.


Written by an agent (Claude Code, claude-opus-4-7).

Summary by CodeRabbit

  • Improvements

    • Added support for computing a previous dependency graph when a current lockfile is provided, improving detection of changes between lockfile states and handling of current-only (orphan) packages.
  • Tests

    • Expanded tests for lockfile-comparison behavior, ensuring prev-graph semantics and orphaned package detection (including installability edge cases).

Review Change Stack

…ice 4d)

`lockfile_to_hoisted_dep_graph` now takes an optional
`current_lockfile: Option<&Lockfile>` and populates
`result.prev_graph` from a second walk over that lockfile.
Ports upstream's wrapper at
installing/deps-restorer/src/lockfileToHoistedDepGraph.ts.

When the current lockfile is `Some(lf)` with a non-empty
`packages:` map, the second walk runs with `force: true` and
`skipped: BTreeSet::new()`. Force matters: an orphan that
landed under the previous install but would now fail
installability (e.g., the host changed platforms) still surfaces
in `prev_graph` so Slice 5's linker can find and rimraf the
stale directory. Empty skipped matters: the previous install's
own filter set is unrelated to "which directories still exist
on disk."

API change

- The public `lockfile_to_hoisted_dep_graph(lockfile, opts)`
  signature gains a middle `current_lockfile: Option<&Lockfile>`
  argument. Single existing caller (the tests) updated to pass
  `None`. Splits the previous body into a private
  `build_dep_graph` helper that the wrapper calls once or twice
  depending on whether a current lockfile is present.

prev_graph shape

- `None` when no current lockfile is supplied, or when the
  supplied lockfile has no `packages:` map (a brand-new install
  in progress). Mirrors upstream's `prevGraph = {}` fallback —
  pacquet uses `None` rather than an empty map, but the linker
  treats both the same.
- `Some(graph)` otherwise, keyed by directory just like the
  wanted-lockfile graph.

Tests

- `prev_graph_none_when_current_lockfile_absent` — no current
  lockfile → `prev_graph` is `None`, wanted graph still produced.
- `prev_graph_none_when_current_lockfile_has_no_packages` —
  current lockfile with `packages: None` → `prev_graph` is
  `None`.
- `prev_graph_contains_orphan_from_current_only_lockfile` —
  package in current but not wanted appears in `prev_graph`,
  not in `graph`.
- `prev_graph_includes_orphan_even_when_now_incompatible` —
  darwin-only orphan in the current lockfile, host is linux,
  wanted lockfile is empty: `prev_graph` still contains it
  (proves `force: true` overrides the installability check),
  and the wanted-walk's `skipped` stays empty (proves the
  prev-walk's skipped set is independent).

All 4 new tests pass alongside the 15 walker tests from
4a-4c.
Copilot AI review requested due to automatic review settings May 13, 2026 21:12
@coderabbitai

coderabbitai Bot commented May 13, 2026

Copy link
Copy Markdown

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR extends the hoisted dep-graph walker to accept an optional current lockfile and compute a previous-state dependency graph (prev_graph). When a current lockfile is provided with a packages map, the code builds the previous graph using forced hoist and an empty skipped set, enabling orphan directory detection. All existing tests are updated to the new three-parameter API, and comprehensive tests validate the new orphan-detection behavior.

Changes

Two-Lockfile Diff for Orphan Detection

Layer / File(s) Summary
Wrapper signature and prev_graph computation
crates/package-manager/src/hoisted_dep_graph.rs
Module documentation updated to describe the two-lockfile diff flow. New wrapper function accepts optional current_lockfile parameter and implements prev_graph computation: when current_lockfile has a packages map, the function builds a dependency graph for the current state using forced hoist with empty skipped set, then returns the wanted graph with prev_graph set accordingly.
Existing test migration to three-parameter API
crates/package-manager/src/hoisted_dep_graph.rs
Existing walker tests updated to call lockfile_to_hoisted_dep_graph with the new three-parameter signature, passing None for current_lockfile to preserve their original test intent.
New prev_graph and orphan detection validation tests
crates/package-manager/src/hoisted_dep_graph.rs
New test section validates prev_graph semantics: prev_graph is None when current_lockfile is absent or lacks packages; orphan directories present in current-only layout appear in prev_graph but are absent from the wanted graph; orphans remain in prev_graph even if installability-incompatible for the wanted install and are not added to the wanted walk’s skipped.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • pnpm/pacquet#486: Initial walker implementation for lockfile_to_hoisted_dep_graph that this PR extends with current_lockfile support.
  • pnpm/pacquet#491: Related changes to force and skipped handling in the same walker logic that the prev_graph computation leverages.
  • pnpm/pacquet#478: Introduces the LockfileToDepGraphResult and options types that this PR extends to populate prev_graph.

Poem

🐰 I hopped through locks of past and new,
Rebuilt the graph to find the true,
Orphans found where once they hid,
Now prev_graph tells what earlier did,
A carrot-coded trace for me and you.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: adding prev_graph diff computation from current lockfile, which is the primary feature of this PR.
Description check ✅ Passed The description thoroughly covers the Summary, What lands, API changes, test coverage, and next steps. All critical sections from the template are addressed with substantial detail.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/438-slice-4d

Comment @coderabbitai help to get the list of available commands and usage tips.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/package-manager/src/hoisted_dep_graph.rs`:
- Around line 309-318: The branch that sets prev_graph uses
current.packages.is_some(), which still matches Some({}) and causes an
unnecessary second walk; change the guard in the match on current_lockfile to
require a non-empty packages map (e.g. check current.packages is Some and not
empty via as_ref().map(|m| !m.is_empty()).unwrap_or(false) or is_some_and(|m|
!m.is_empty())), then call build_dep_graph only when the map is non-empty; also
add a regression test passing packages: Some(HashMap::new()) to ensure
prev_graph remains None. Use the symbols current_lockfile, current.packages,
LockfileToHoistedDepGraphOptions, build_dep_graph, prev_graph, and opts to
locate the code to modify.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fdd03a8d-6fdc-4ce1-a1be-9a80124c285d

📥 Commits

Reviewing files that changed from the base of the PR and between c359dd7 and c0d0b9b.

📒 Files selected for processing (1)
  • crates/package-manager/src/hoisted_dep_graph.rs
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
  • GitHub Check: copilot-pull-request-reviewer
  • GitHub Check: Lint and Test (ubuntu-latest)
  • GitHub Check: Lint and Test (windows-latest)
  • GitHub Check: Run benchmark on ubuntu-latest
  • GitHub Check: Run benchmark on ubuntu-latest
  • GitHub Check: Lint and Test (macos-latest)
  • GitHub Check: Code Coverage
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Preserve existing method chains and pipe-trait chains; do not break them into intermediate let bindings unless there is a concrete justification such as a compilation failure, borrow checker rejection, meaningful performance improvement, or other technical necessity. Refactoring for style alone is not sufficient justification.
Choose owned vs. borrowed parameters to minimize copies; prefer borrowed types (&Path over &PathBuf, &str over &String) when it does not force extra copies.
Prefer Arc::clone(&x) and Rc::clone(&x) over x.clone() for reference-counted types to make the cost visible at the call site.
Do not use star imports inside module bodies. Write use super::{Foo, bar} instead of use super::*; for any glob whose target is a module you control. External-crate preludes (e.g., use rayon::prelude::*;) and root-of-module re-exports (e.g., pub use submodule::*; in lib.rs) are exceptions.
Follow Rust API Guidelines for naming, as documented in https://rust-lang.github.io/api-guidelines/naming.html.
Declare a newtype wrapper for any branded string type being ported from TypeScript pnpm. Do not collapse the brand into a plain String or &str; give the type its own struct so misuse is a type error.
When porting branded string types where upstream TypeScript always validates before construction, validate in the Rust port too. Construct the wrapper only via TryFrom<String> and/or FromStr; do not provide an infallible public constructor that takes an arbitrary string.
For branded string types where upstream TypeScript never validates (used purely for type-safety to prevent confusion between string slots), expose an infallible From<String> and From<&str> constructor in the Rust wrapper.
When upstream TypeScript occasionally constructs a branded type without validation (via bare as assertion), add a from_str_unchecked (or similarly named) constructor on the Rust side. Keep the validating constructor as well; `from_str_u...

Files:

  • crates/package-manager/src/hoisted_dep_graph.rs
🧠 Learnings (3)
📚 Learning: 2026-05-07T23:19:08.272Z
Learnt from: KSXGitHub
Repo: pnpm/pacquet PR: 401
File: tasks/integrated-benchmark/src/work_env.rs:343-344
Timestamp: 2026-05-07T23:19:08.272Z
Learning: When reviewing Rust code in pnpm/pacquet for deprecated API usage, do not automatically treat `serde_saphyr::to_string` as deprecated. In `serde-saphyr` v0.0.25, `serde_saphyr::to_string` has no `#[deprecated]` attribute (the `#[deprecated]` later in `serde-saphyr-0.0.25/src/lib.rs` applies to a different function). Only flag `serde_saphyr::to_string` as deprecated if the resolved dependency version’s source shows `#[deprecated]` on that specific function.

Applied to files:

  • crates/package-manager/src/hoisted_dep_graph.rs
📚 Learning: 2026-05-13T19:22:48.951Z
Learnt from: zkochan
Repo: pnpm/pacquet PR: 478
File: crates/package-manager/src/hoisted_dep_graph.rs:51-55
Timestamp: 2026-05-13T19:22:48.951Z
Learning: When reviewing this Rust codebase, avoid introducing/using a newtype like `PkgIdWithPatchHash` in only one module (e.g., `hoisted_dep_graph.rs`) if other related pacquet modules still represent the upstream pnpm “pkg id with patch hash” as a plain `String` (e.g., `virtual_store_layout`). For type consistency, either keep `pkg_id_with_patch_hash` as `String` here, or require a workspace-wide sweep that defines/extracts the `PkgIdWithPatchHash` newtype once and updates all call sites together (otherwise defer the refactor to a coordinated follow-up PR).

Applied to files:

  • crates/package-manager/src/hoisted_dep_graph.rs
📚 Learning: 2026-05-13T20:09:22.171Z
Learnt from: zkochan
Repo: pnpm/pacquet PR: 486
File: crates/package-manager/src/hoisted_dep_graph.rs:472-476
Timestamp: 2026-05-13T20:09:22.171Z
Learning: In pnpm/pacquet, when generating serialized `hoisted_locations` strings (used for `.modules.yaml`), normalize path separators to forward slashes (e.g., replace `\\` with `/`). This preserves cross-platform portability: the output must not use OS-native separators because upstream pnpm’s `path.relative` can return platform-specific separators, which would break consistency with pnpm/pacquet’s other serialized formats on Windows.

Applied to files:

  • crates/package-manager/src/hoisted_dep_graph.rs

Comment thread crates/package-manager/src/hoisted_dep_graph.rs
@github-actions

github-actions Bot commented May 13, 2026

Copy link
Copy Markdown

Micro-Benchmark Results

Linux

group                          main                                   pr
-----                          ----                                   --
tarball/download_dependency    1.00     12.1±0.47ms   357.9 KB/sec    1.00     12.1±0.24ms   357.2 KB/sec

@codecov

codecov Bot commented May 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.59%. Comparing base (ef38a48) to head (6e8ed11).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #494      +/-   ##
==========================================
- Coverage   88.99%   88.59%   -0.41%     
==========================================
  Files         121      121              
  Lines       12760    13125     +365     
==========================================
+ Hits        11356    11628     +272     
- Misses       1404     1497      +93     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…v_graph (#494 review)

`current.packages.is_some()` matched `Some(empty_map)` too,
causing 4d to do an unnecessary second walk and return
`Some(empty_graph)` for a case the doc-comment described as
`None`. Tighten the guard to require a non-empty `packages` map.

Pacquet uses `Option<DependenciesGraph>` for `prev_graph`
(upstream uses an always-present `DependenciesGraph` with `{}`
for the no-current case). The no-packages and empty-packages
cases both produce the same observable behavior — "no orphans
to consider" — so they should share one representation rather
than be inconsistent. The doc-comment's stated contract was
`None`; the code now matches.

Added `prev_graph_none_when_current_lockfile_has_empty_packages`
to pin the empty-map case.

Caught by Coderabbit on #494.
Copilot AI review requested due to automatic review settings May 13, 2026 21:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@github-actions

Copy link
Copy Markdown

Integrated-Benchmark Report (Linux)

Scenario: Frozen Lockfile

Command Mean [s] Min [s] Max [s] Relative
pacquet@HEAD 2.695 ± 0.104 2.539 2.838 1.02 ± 0.04
pacquet@main 2.630 ± 0.041 2.542 2.673 1.00
pnpm 6.242 ± 0.081 6.153 6.407 2.37 ± 0.05
BENCHMARK_REPORT.json
{
  "results": [
    {
      "command": "pacquet@HEAD",
      "mean": 2.6952387252,
      "stddev": 0.10363875523982484,
      "median": 2.6742342772,
      "user": 2.5650369200000003,
      "system": 3.7075707799999997,
      "min": 2.5386509021999997,
      "max": 2.8381254721999998,
      "times": [
        2.8295376392,
        2.8381254721999998,
        2.7895934652,
        2.5996557252,
        2.6728467132,
        2.6756218412,
        2.5386509021999997,
        2.7616731762,
        2.6264354761999997,
        2.6202468411999997
      ]
    },
    {
      "command": "pacquet@main",
      "mean": 2.6299002116,
      "stddev": 0.041169580202623186,
      "median": 2.6355064382,
      "user": 2.5717664200000003,
      "system": 3.7399090799999994,
      "min": 2.5420211121999996,
      "max": 2.6734573862,
      "times": [
        2.6432166942,
        2.6685221562,
        2.6573307032,
        2.5929807972,
        2.6120221492,
        2.6661960031999996,
        2.5420211121999996,
        2.6734573862,
        2.6154589322,
        2.6277961822
      ]
    },
    {
      "command": "pnpm",
      "mean": 6.2421680428999995,
      "stddev": 0.0807744741554781,
      "median": 6.2352704717,
      "user": 9.14831162,
      "system": 4.48692288,
      "min": 6.1527093831999995,
      "max": 6.4073979682,
      "times": [
        6.1713484492,
        6.2694754942,
        6.4073979682,
        6.2471230991999995,
        6.2234178442,
        6.1899135242,
        6.2901262302,
        6.1571459632,
        6.1527093831999995,
        6.3130224732
      ]
    }
  ]
}

Scenario: Frozen Lockfile (Hot Cache)

Command Mean [ms] Min [ms] Max [ms] Relative
pacquet@HEAD 761.8 ± 61.5 715.5 929.5 1.00
pacquet@main 812.3 ± 56.9 749.6 916.7 1.07 ± 0.11
pnpm 2686.3 ± 102.4 2574.6 2902.7 3.53 ± 0.31
BENCHMARK_REPORT.json
{
  "results": [
    {
      "command": "pacquet@HEAD",
      "mean": 0.7617755819800001,
      "stddev": 0.061494705560108255,
      "median": 0.74939361378,
      "user": 0.40366559999999996,
      "system": 1.60018992,
      "min": 0.7155361342800001,
      "max": 0.92953663028,
      "times": [
        0.92953663028,
        0.74959667628,
        0.75560101828,
        0.7463682272800001,
        0.77105477028,
        0.7293925372800001,
        0.7541761112800001,
        0.71730316328,
        0.74919055128,
        0.7155361342800001
      ]
    },
    {
      "command": "pacquet@main",
      "mean": 0.8122597558800001,
      "stddev": 0.05691932316680746,
      "median": 0.78996634578,
      "user": 0.39049329999999993,
      "system": 1.63869262,
      "min": 0.74962893228,
      "max": 0.91670827728,
      "times": [
        0.8600896882800001,
        0.78821737428,
        0.8898229712800001,
        0.79171531728,
        0.74962893228,
        0.76821123128,
        0.91670827728,
        0.8117146672800001,
        0.7775667782800001,
        0.76892232128
      ]
    },
    {
      "command": "pnpm",
      "mean": 2.6863350904800005,
      "stddev": 0.10241019380751822,
      "median": 2.6638178387800004,
      "user": 3.2082362999999994,
      "system": 2.2539271199999997,
      "min": 2.5746103212800002,
      "max": 2.90267468628,
      "times": [
        2.6257164542800004,
        2.6148898232800004,
        2.90267468628,
        2.5803238912800004,
        2.5746103212800002,
        2.6241975512800004,
        2.7357525812800003,
        2.7489633752800002,
        2.7019192232800004,
        2.7543029972800004
      ]
    }
  ]
}

@zkochan

zkochan commented May 13, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented May 13, 2026

Copy link
Copy Markdown
✅ Actions performed

Full review triggered.

@zkochan zkochan merged commit 4c062a8 into main May 13, 2026
14 of 17 checks passed
@zkochan zkochan deleted the feat/438-slice-4d branch May 13, 2026 22:02
@zkochan zkochan mentioned this pull request May 14, 2026
59 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants