Skip to content

perf(pacquet): thread the prior lockfile through resolve_node to skip child re-derivation on unchanged packages #11845

Description

@zkochan

Summary

Pacquet's fresh-lockfile install uses the prior pnpm-lock.yaml only for preferred-version seeding (install_with_fresh_lockfile.rs:416). Once that's done, the tree walker (resolve_dependency_tree.rs) goes through pick_package + extract_children(&result) for every single node — re-deriving the children edges from a freshly-fetched manifest even when the prior lockfile's snapshots: map already pinned the version and named every child by (alias, depPath).

Upstream pnpm threads a rich payload through getDepsToResolve and resolveDependencyinfoFromLockfile, dependencyLockfile, current resolution, current deps, hasBin, peer metadata, and the updated flag — and uses it to avoid or narrow child traversal for unchanged packages. See resolveDependencies.ts:1090.

This is the third pacquet hot-cache resolver gap, alongside #11843 (peekManifestFromStore) and #11844 (leaf-NodeId collapse). It's the biggest of the three by scope, and the one with the widest payoff on real hot-cache installs.

Why this matters

resolve_importer is ~3.1 s of the ~5.03 s warm-cache wall on alotta-files (after the wins on #11837). The wins compose cleanly:

Together they collapse the resolver on a hot install into "copy the prior lockfile's graph, only consult the network/store for entries that changed." That's the shape pnpm has today; pacquet's resolver re-derives everything every install.

What pnpm threads through

From resolveDependencies.ts:1090-style call sites, per dep:

  1. infoFromLockfile — the prior lockfile's pin for this (alias, range) edge: resolved version, integrity, tarball URL.
  2. dependencyLockfile — the matching snapshots: entry: dependencies:, optionalDependencies:, peerDependencies:, transitivePeerDependencies:, patched, optional.
  3. hasBin — from the prior packages: entry, so the bin linker's gate is authoritative without re-reading manifests.
  4. updated flag — set when this package's pin must change (the spec moved, a transitive forced a different version, an override applies). Drives the "re-fetch packument" branch.
  5. peer metadatapeerDependencies and peerDependenciesMeta straight from packages:.

When updated is false and the prior pin still satisfies the spec, pnpm:

  • Skips the packument fetch (covered by perf(pacquet): port peekManifestFromStore fast path to skip the picker on hot-cache resolves #11843's peekManifestFromStore for the manifest-substitute case; this issue covers the children-edges case).
  • Constructs a ResolveResult-equivalent straight from infoFromLockfile + dependencyLockfile.
  • Uses the lockfile's dependencies: / optionalDependencies: maps as the children spec instead of extract_children(manifest).
  • Recurses on each child with that child's own infoFromLockfile / dependencyLockfile pre-looked-up from the same snapshots: map.

So the tree walk for the unchanged subgraph becomes a lockfile graph copy with zero manifest parsing.

What changes on the pacquet side

Contract additions

WantedDependency (or a sibling WantedDependencyContext) needs an optional payload, populated by the fresh-install dispatch when a prior lockfile is in scope:

pub struct PriorLockfileInfo<'a> {
    /// The matched importer-level pin: `(alias, version)` from the
    /// prior lockfile's `importers:<root>.dependencies` /
    /// `devDependencies` / `optionalDependencies`. `None` when this
    /// edge is new (not in the prior lockfile) or the spec changed.
    pub pin: Option<&'a ImporterPin>,
    /// The matched `snapshots:` entry, indexed by the pin's
    /// `name@version[(peer)...]` depPath. Carries `dependencies`,
    /// `optionalDependencies`, `transitivePeerDependencies`,
    /// `patched`, `optional`.
    pub snapshot: Option<&'a SnapshotEntry>,
    /// The matched `packages:` row by `name@version`. Carries
    /// `resolution` (integrity + tarball), `hasBin`,
    /// `peerDependencies`, `peerDependenciesMeta`, `engines`, `cpu`,
    /// `os`, `libc`, `deprecated`.
    pub package: Option<&'a PackageMetadata>,
    /// `true` when this dep's pin must change — the spec moved, a
    /// transitive forced a different version, an override applies,
    /// or the prior pin is no longer in `packages:`. Pacquet's
    /// dispatch sets this; `resolve_node` short-circuits when
    /// `false`.
    pub updated: bool,
}

The fresh-install dispatch builds a HashMap<(importer_dir, alias, range), PriorLockfileInfo> from wanted_lockfile once at install start and threads it into each direct dep's WantedDependency. Per-child lookups happen inside resolve_node: when recursing on a child, lift the matching SnapshotEntry.dependencies[child_alias] (a SnapshotDepRef), resolve it to a name@version depPath, look up the next snapshot + package entry, hand it down.

Resolver short-circuit

In resolve_node:

if let Some(info) = wanted.prior_lockfile_info()
    && !info.updated
    && let Some(snapshot) = info.snapshot
    && let Some(package) = info.package
{
    // Build a ResolveResult from the lockfile entries — no picker
    // call, no manifest parse. Children come from
    // `snapshot.dependencies + snapshot.optional_dependencies`.
    let result = build_resolve_result_from_lockfile(snapshot, package, info.pin)?;
    let child_specs = lockfile_child_specs(snapshot);
    // ... existing recursion, but each child carries its own
    // pre-looked-up PriorLockfileInfo from the same `snapshots:`
    // map.
    ...
}

When the gate misses (no prior entry, updated = true, or the spec moved), fall through to today's pick_package flow unchanged.

Gating conditions

Fast path is safe iff all of:

  1. A prior lockfile is in scope (InstallWithFreshLockfile::wanted_lockfile is Some).
  2. The importer's WantedDependency.bare_specifier matches what the prior lockfile's importer block recorded for the same (alias, dependency_group) — i.e. the user did not change the spec.
  3. No --update / update == UpdateBehavior::Latest for this dep.
  4. No published_by / minimum_release_age policy in play (the lockfile entries don't carry publish-time; same gate as perf(pacquet): port peekManifestFromStore fast path to skip the picker on hot-cache resolves #11843).
  5. No override / catalog re-resolution applies to the dep.
  6. The matched snapshots: and packages: entries exist (defensive — the lockfile shouldn't reference a snapshot it doesn't have, but pacquet has seen broken lockfiles in the wild).
  7. The pin's integrity matches whatever store-index row would have been consulted (only relevant if we also want to skip the existence check — otherwise the install path will catch a missing tarball later).

Any miss → fall through to the regular picker. The updated flag itself is the AND of the spec/override/transitive gates; in the simplest pacquet port it's just !spec_unchanged_in_prior_lockfile.

Interaction with #11843 and #11844

Expected impact

Per-phase trace on the warm-cache alotta-files benchmark today: resolve_importer is ~3.1 s of 5.03 s. With this issue plus #11843 plus #11844 the resolver should land in the few-hundred-ms range — a lockfile graph copy plus a handful of pick_package calls for any updated pins. That puts pacquet under pnpm's 4.16 s on the same fixture.

Standalone (this issue alone, picker still firing for the manifest-substitute): a meaningful chunk of resolve_importer's wall time goes to extract_children + recursion scheduling rather than the picker itself, so this would still cut a substantial slice independent of #11843.

References


Written by an agent (Claude Code, claude-opus-4-7).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions