Skip to content

perf(pacquet/install): no-op short-circuit fires after verify_lockfile_resolutions, defeating the fast path on cache wipes #11940

Description

@zkochan

Summary

Pacquet's no-op short-circuit runs after verify_lockfile_resolutions. When ~/.cache/pnpm/lockfile-verified.jsonl is hot, the verifier short-circuits in milliseconds and the no-op completes in ~125 ms — faster than pnpm. When that cache is wiped (e.g. between vlt benchmark iterations via clean-helpers.sh::clean_pnpm_cache's safe_remove "$HOME/.cache/pnpm"), the verifier re-fans-out over every lockfile entry: ~770 ms on svelte, multiple seconds on larger trees.

The state at that point is by construction "the previous install already finished and wrote the wanted lockfile, the current lockfile under the virtual store, and a consistent modules.yaml." The previous install's verifier passed against those bytes. Re-verifying isn't doing safety work; it's recomputing the same answer.

Found while investigating #11902 — see investigation comment.

Repro (svelte fixture, ~25 packages)

cd /tmp/repro-svelte           # vlt's fixtures/svelte
pacquet install >/dev/null      # warm: writes lockfile-verified.jsonl
{ time pacquet install; }       # warm: ~125 ms
rm -rf ~/.cache/pnpm            # what vlt's clean_pnpm_cache does
{ time pacquet install; }       # cold: ~770 ms

ndjson trace pins the gap inside the verifier:

T0     pnpm:package-manifest/debug:
+   2ms pnpm:lockfile-verification/debug: (Started)
+ 767ms pnpm:lockfile-verification/debug: (Done)
+   0ms pnpm:context/debug:
+   0ms pnpm/info: Lockfile is up to date, resolution step is skipped

Impact on the vlt chart

Every lockfile+node_modules cell pays the verifier cost on every measured iteration because the bench's prepare wipes ~/.cache/pnpm. Pnpm appears to have an earlier shortcut and is unaffected (~580 ms regardless).

Variation next astro svelte vue large babylon
lockfile+node_modules 5.89× 3.08× 1.64× 5.90× 7.20× (panic, #11939)

Numbers from #11902 verification.

Two ways forward

A. Reorder — cheap win

Move the no-op check before verify_lockfile_resolutions. The contract: when take_frozen_path && wanted_lockfile == current_lockfile && modules.yaml is consistent, the install state is byte-identical to the last completed install. The verifier already passed against those bytes on the previous run.

Caveat: policy fingerprint. If the user tightens minimumReleaseAge between installs, a previously-verified lockfile may no longer pass. The existing lockfile-verified.jsonl stat shortcut already encodes the policy snapshot at verification time and provides exactly the equivalence check (every_verifier_trusts_cached_run) needed at the new call site.

Sketch:

// 1. read modules.yaml + current_lockfile
// 2. compute take_frozen_path
// 3. if take_frozen_path && wanted_lockfile == current_lockfile && is_modules_yaml_consistent
//    && every_verifier_trusts_cached_run(<cached record for this lockfile>, &verifiers):
//        emit "Lockfile is up to date", return Ok(())
// 4. otherwise run verify_lockfile_resolutions + the rest of the install path

The lockup against the cached record adds one stat + one disk read — orders of magnitude faster than the per-candidate fan-out.

B. Port pnpm's earlier shortcut directly

Pnpm at ~580 ms regardless of cache state suggests a path inside installing/deps-installer/src/install/index.ts that exits before verifyLockfileResolutions runs. Couldn't fully trace it in the time I had — pnpm's lockfile-verified.jsonl doesn't get touched for our svelte scenario either, so its bypass isn't via the same cache. Worth a targeted read of the dispatch flow at index.ts:332-407.

Option A is the cheaper of the two and would put pacquet at ~125 ms on every lockfile+node_modules cell. Option B keeps the code closer to upstream.


Written by an agent (Claude Code, claude-opus-4-7).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions