Skip to content

Offload lockfile verification to pnpr (and cache it server-side) #12139

Description

@zkochan

Motivation

Lockfile-resolution verification (the minimumReleaseAge / trustPolicy / tarball-URL-binding checks in verifyLockfileResolutions — see #12134) is expensive and connectivity-dependent on the client: a cache miss means packument round-trips to the registry, and the tarball-URL binding is fail-closed, so a re-verified lockfile now requires registry reachability.

pnpr already does the work that makes verification cheap: it resolves server-side and caches the packuments (<storage>/<pkg>/package.json) — which is exactly the data verification needs (dist.tarball, time, attestation evidence). So pnpr is the natural place to verify, and it can amortize the cost across all clients/projects.

Goal: when a pnpr server is configured (pnprServer / --pnpr-server / PNPR_SERVER), the client should never run verifyLockfileResolutions locally — pnpr verifies instead. This is faster (pnpr's packument cache is warm and shared) and removes the client's own registry-reachability requirement. It adds no new trust: the client already trusts pnpr to resolve and to serve the package bytes (/v1/files by digest), so delegating verification is within that boundary.

What gets verified: the input lockfile, before resolution

Locally, verify_lockfile_resolutions runs on the loaded on-disk lockfile, before the resolve/frozen dispatch (install.rs ~line 630) — regardless of frozen vs non-frozen. The thing verified is the input lockfile, before any updates. That must be preserved when offloading to pnpr.

This matters because the existing lockfile is an input in the non-frozen / partial case too (pnpm add, pnpm update, pnpm install after a manifest change): it's the resolution reuse seed. Entries reused from it are carried over, not freshly resolved by pnpr, so they are not inherently verified — they must be checked just like in a frozen install.

So the unified model:

Whenever the client has an on-disk lockfile, it sends it to /v1/install (needed both for resolution reuse and as the verification target). pnpr verifies that input lockfile first — under the client's policy, against its warm packument cache — before resolving. Then it resolves: frozen → use as-is (already verified); non-frozen → reuse the verified entries + resolve new/changed ones. Freshly-resolved output entries are inherently verified (pnpr wrote their tarball from the packument it read). A true first install with no lockfile has nothing to verify.

frozenLockfile governs resolution behavior (freeze vs reuse-and-update), not whether the lockfile is sent or verified.

Prerequisite: send the existing lockfile to pnpr

Today the request (pacquet/crates/pnpr-client/src/lib.rs:164-174) sends only projects (manifest deps), storeIntegrities, registry, namedRegistries, overrides, minimumReleaseAgenot the existing lockfile, so pnpr can neither reuse nor verify it. Sending the on-disk lockfile (when present) is a prerequisite of this work — and it's also what enables pnpr to match pnpm's resolution-reuse behavior on partial installs rather than re-resolving from scratch.

Protocol — single endpoint

Keep one endpoint (POST /v1/install); add to the request:

  • the client's existing on-disk lockfile (when present),
  • a frozenLockfile: boolean flag (governs resolution behavior),
  • the full client policy the local verifier uses: minimumReleaseAge (already sent), minimumReleaseAgeExclude, ignoreMissingTimeField, trustPolicy, trustPolicyExclude, trustPolicyIgnoreAfter.

No new endpoint and no capability/version negotiation — pnpr is pre-production and client + server move in lockstep.

pnpr verifies the input lockfile under the client's policy (not its own) before resolving; on failure it streams a violations line carrying the existing codes/reasons (ERR_PNPM_TARBALL_URL_MISMATCH, MINIMUM_RELEASE_AGE_VIOLATION, TRUST_DOWNGRADE) so the client aborts identically to local verification.

Caching on pnpr

Two layers, both reusing what already exists:

  1. The packument cache is already the verification cache. It holds the per-version dist.tarball / time / attestation data, so verification is a local compute over warm data — no extra registry hits. This eliminates the expensive part, so even a cache miss is just N cheap local computes (semver + URL compare).

  2. Whole-lockfile verdict cache — same model as local. Cache the entire lockfile verification result, keyed by (lockfile content hash, policy snapshot) exactly like the local lockfile-verified.jsonl (CacheRecord + canTrustPastCheck). A hit is O(1) — one lookup says "this lockfile already passed under a compatible policy," skip the whole pass. This is the dominant win for a shared pnpr: repeat installs of the same lockfile (CI re-runs, a fleet building the same repo) hit instantly.

    This deliberately is not a per-entry verdict cache. Per-entry keying would be O(N) lookups per install even on a full hit, for a negligible payoff (the expensive fetches are already covered by the packument cache, so the only thing per-entry saves is a cheap recompute). Whole-lockfile keying also sidesteps any "now"-staleness: only passes of a fully-verified lockfile are cached, an age-pass is monotonic (versions only get older), and the lockfile hash pins the exact versions — which is why the local cache is already time-correct via canTrustPastCheck without storing a cutoff timestamp. pnpr reuses that logic verbatim (policy snapshot incl. the tarballUrlBinding marker).

Client routing

pacquet install already detours to the pnpr fast path before the local verify gate when config.pnpr_server is set (pacquet/crates/cli/src/cli_args/install.rs, around the install_via_pnpr branch, vs the local verify_lockfile_resolutions call in install.rs). The change: treat pnpr's result as authoritative and skip verifyLockfileResolutions (and the local lockfile-verified.jsonl cache) entirely on the pnpr path. The TS CLI's pnpr path should do the same.

SQLite scope

  • pnpr: yes — as a concurrency-safe store for the same whole-lockfile records (CacheRecord), so many clients can read/write and the server can do TTL/LRU eviction without the JSONL append/compaction races. Not for per-entry keying. pnpr already uses SQLite for its token store.
  • Local CLI cache: out of scope here. lockfile-verified.jsonl is single-machine and deliberately shared byte-for-byte between the pnpm CLI and pacquet; migrating it to SQLite would be a separate, lockstep pnpm+pacquet decision with a small payoff.

Suggested phasing

  1. Send the existing lockfile + full policy on /v1/install; pnpr verifies the input lockfile before resolving (and gains reuse from it); client skips local verification whenever pnpr is configured.
  2. Add the frozenLockfile flag so frozen/CI installs use the committed lockfile as-is (already verified by step 1) rather than reusing-and-updating.
  3. Whole-lockfile verdict cache on pnpr (SQLite-backed CacheRecord + canTrustPastCheck, keyed by lockfile hash + policy) so repeat installs of the same lockfile short-circuit in O(1).

Relevant code

  • pnpr server / accelerator: pnpr/crates/pnpr/src/install_accelerator.rs, server.rs, cache.rs
  • client + request payload: pacquet/crates/pnpr-client/src/lib.rs (request at ~:164-174)
  • local verification + the pre-dispatch gate: installing/deps-installer/src/install/verifyLockfileResolutions.ts (+ verifyLockfileResolutionsCache.ts); pacquet pacquet/crates/lockfile-verification/, gate at pacquet/crates/package-manager/src/install.rs (~:630)
  • routing/config: pacquet/crates/cli/src/cli_args/install.rs, pacquet/crates/config/src/lib.rs (pnpr_server)

Related: #12134, #12122.


Written by an agent (Claude Code, claude-opus-4-8).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions