Skip to content

perf(npm-resolver): layer abbreviated meta + attestation before full metadata in the minimumReleaseAge gate#11704

Merged
zkochan merged 4 commits into
mainfrom
11687-attestation
May 17, 2026
Merged

perf(npm-resolver): layer abbreviated meta + attestation before full metadata in the minimumReleaseAge gate#11704
zkochan merged 4 commits into
mainfrom
11687-attestation

Conversation

@zkochan

@zkochan zkochan commented May 17, 2026

Copy link
Copy Markdown
Member

Follow-up to #11691 — item 2 from #11687, plus a related shortcut.

What

When the minimumReleaseAge lockfile verification gate needs to know when a version was published, it used to fetch a multi-MB full metadata document per package just to read one timestamp. This PR replaces that single-step path with a four-layer lookup that pays the cheapest viable source first:

  1. Abbreviated metadata's modified field — the resolver already fetches this for resolution. If the package as a whole hasn't been modified within the policy cutoff, every version it contains is at least that old; return modified as a conservative upper-bound and skip the rest of the chain.
  2. Local FULL_META_DIR mirror — exact per-version times if a previous verification populated it.
  3. npm attestation endpoint (/-/npm/v1/attestations/<name>@<version>) — a tens-of-KB Sigstore bundle whose Rekor inclusion time stands in for publish time. Wins on cold cache when the package was published with provenance.
  4. Full metadata fetch — last resort.

Why

The verification cache from #11691 made repeat installs against an unchanged lockfile effectively free. The remaining cost is the first verification on a fresh CI runner with no restored cache — particularly pnpm install --frozen-lockfile, where every locked package's publish timestamp has to be confirmed. Fetching the full metadata document for each package is wasteful when:

  • The resolver has usually already cached abbreviated metadata, whose modified field alone is enough to clear stable packages (the common case).
  • For recently-modified packages, the per-version attestation endpoint is orders of magnitude smaller than full metadata.

How

Abbreviated modified shortcut

fetchFullMetadataCached is refactored to share an internal helper with a new fetchAbbreviatedMetadataCached. Both do conditional GETs against their respective on-disk mirrors. On a non-frozen install the abbreviated mirror is already populated by the resolver, so the shortcut hits the local cache at headers-only cost. On --frozen-lockfile the fetch is still cheaper than full metadata.

If Date.parse(modified) < cutoff, return modified — it's an upper bound on every version's publish time in this package, and the verifier's published < cutoff check passes trivially.

Attestation endpoint

fetchAttestationPublishedAt (new module) hits /-/npm/v1/attestations/<name>@<version>, parses the response, and reads the earliest bundle.verificationMaterial.tlogEntries[].integratedTime across the attestation bundles. That's the Rekor inclusion time — a couple of seconds after publish, well within tolerance for a policy that operates in minutes/hours/days. Returns undefined on 404 / network error / malformed body so the caller falls back.

Per-install dedup

The lookup carries a PublishedAtLookupContext with four memo maps: abbreviated meta per (registry, name), local full meta per (registry, name), full meta network fetch per (registry, name), final published-at per (registry, name, version). Verifying many versions of the same package only pays the disk/network costs once.

Trust model

No Sigstore signature verification on the attestation path. The trust model is identical to reading the registry's time field on the full metadata document — we already trust the registry to serve correct publish timestamps for the gate's purpose. The win is purely bandwidth.

Full Sigstore verification (Fulcio cert chain + Rekor inclusion proof) would harden the timestamp against a compromised registry. It pulls in the sigstore npm package and the TUF root — a separate dependency-surface discussion, parked as future work.

Tests

  • 13 unit tests in resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts: ISO timestamp extraction, URL construction (scoped + unscoped), 404 / 5xx / network error / malformed JSON / missing fields → undefined, earliest-of-multiple-attestations, defensive number-as-integratedTime, auth header forwarding, trailing-slash normalization.
  • Existing minimumReleaseAge + verifyLockfileResolutions integration suites (45 tests) still pass — the fallback chain preserves end-to-end behavior when the new shortcuts don't apply.

Written by an agent (Claude Code, claude-opus-4-7).

…l metadata

The minimumReleaseAge gate currently fetches a package's full metadata
document just to read one version's publish timestamp — that's
multi-MB per package, paid on cold-cache + \`--frozen-lockfile\`
installs (e.g. fresh CI runners without a restored verification
cache).

For provenance-published packages, npm exposes a smaller per-version
endpoint at \`/-/npm/v1/attestations/<name>@<version>\` that returns
the Sigstore attestation bundle. The bundle's
\`verificationMaterial.tlogEntries[].integratedTime\` is the Rekor
inclusion time — a couple of seconds after publish, well within
tolerance for a policy that operates in minutes/hours/days.

Wire it in as a step between the on-disk metadata mirror and the
full-metadata network fetch:

  1. Local FULL_META_DIR mirror — cheapest. If a previous resolution
     populated it, take the timestamp from there.
  2. Attestation endpoint — small payload per version. Earliest
     integratedTime across attestations is the conservative pick.
  3. Full metadata fetch — fallback for unattested packages or when
     the attestation response is malformed/missing. Dedup'd per
     (registry, name) so verifying many versions of the same package
     only triggers one full fetch.

Per-(registry, name, version) results memoize for the duration of one
install. Sigstore signature verification is deliberately not done —
the trust model matches reading the registry's \`time\` field on the
full metadata document. The win is purely bandwidth.

Closes the issue-11687 follow-up.
@coderabbitai

coderabbitai Bot commented May 17, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

This PR introduces an attestation-first optimization to pnpm's release-age verification. The verifier now attempts a smaller npm attestation endpoint before fetching full metadata, deriving publish timestamps from Rekor inclusion times, with multi-step fallbacks to cached mirrors and full metadata when attestation data is unavailable.

Changes

Attestation-first published-at lookup

Layer / File(s) Summary
Attestation endpoint fetch function and tests
resolving/npm-resolver/src/fetchAttestationPublishedAt.ts, resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts
fetchAttestationPublishedAt queries npm's /-/npm/v1/attestations/<name>@<version> endpoint, extracts the earliest integratedTime from Rekor tlogEntries, and converts it to an ISO timestamp. Returns undefined on network failures or missing/malformed data. Comprehensive test suite covers URL construction, scoped packages, registry normalization, auth header forwarding, and edge cases (404, 5xx, JSON errors, missing attestation data).
Metadata caching helper refactor
resolving/npm-resolver/src/fetchFullMetadataCached.ts
Refactors cached metadata fetching into a shared internal helper supporting both full and abbreviated registry metadata. Introduces generalized FetchMetadataCachedOptions interface and exports new fetchAbbreviatedMetadataCached function that reuses the full-metadata cache logic with ABBREVIATED_META_DIR.
Verifier multi-step published-at lookup pipeline
resolving/npm-resolver/src/createNpmResolutionVerifier.ts
Refactors the verifier to implement a multi-step fetchPublishedAt pipeline: abbreviated metadata modified shortcut → local FULL_META_DIR mirror → attestation endpoint → full metadata fetch. Replaces prior in-flight time-map memoization with PublishedAtLookupContext containing dedicated caches. Converts any lookup/parse failures into fail-closed MINIMUM_RELEASE_AGE_VIOLATION with error capture instead of silent pass-through.
Changeset and spell check
.changeset/attestation-first-min-release-age.md, cspell.json
Changeset documents the attestation-first optimization and multi-step fallback behavior for cold-cache installs; registers spell-check words Rekor, SLSA, and tlog.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

  • pnpm/pnpm#11687: This PR implements the attestation-first published-time lookup and per-verifier instance memoization requested in the issue, including trying the attestation endpoint before full metadata fetch and caching verification results.

Possibly related PRs

  • pnpm/pnpm#11691: Both PRs modify createNpmResolutionVerifier for minimumReleaseAge semantics; this PR refactors published-at lookup and fail-closed behavior, while the related PR adjusts verifier object shape with policy/canTrustPastCheck.
  • pnpm/pnpm#11622: Both PRs address published-time enforcement in npm-resolver; the related PR upgrades abbreviated metadata to include full time field, while this PR refactors the verifier to prefer abbreviated metadata then fall back to full-metadata/attestation sources.

Suggested labels

area: lockfile, area: supply chain security

Poem

A rabbit bounds through metadata feeds,
Trying smaller paths to find what's old,
When attestations bloom with Rekor seeds,
Cold caches warm with timestamps told,
Fallbacks catch what small probes can't hold. 🐰⛓️

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main performance optimization: adding an attestation-first lookup path before full metadata in the minimumReleaseAge verification gate.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 11687-attestation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

zkochan added 3 commits May 17, 2026 14:03
…nitions

- Hoist \`fetchPublishedAt\`/\`readLocalMetaTime\`/\`fetchFullMetaTime\`
  out of \`createNpmResolutionVerifier\`'s body into module-scope
  helpers. The state they used to close over (\`opts\`, the three
  cache Maps) moves into a \`PublishedAtLookupContext\` object passed
  as the first argument.
- Eliminate the IIFEs the helpers used for atomic-set-before-await:
  the work now lives in a separate \`resolvePublishedAt\` /
  \`loadLocalMetaTime\` function, called expression-style so the
  Promise is set into the cache before the first await runs.
- Rename one/two-letter locals: \`op\` → \`retryOperation\`,
  \`p\` (cached Promise) → \`cachedPromise\`, \`att\` → \`attestation\`,
  \`raw\` extracted into \`parseIntegratedTimeSeconds\` for clarity.
…n lookups

The resolver already fetches abbreviated metadata for every package by
default — it's a smaller document than full metadata, with a
package-level \`modified\` timestamp but no per-version \`time\`
field. For packages whose \`modified\` predates the policy cutoff, we
know every version they contain is also older than the cutoff: no
per-version lookup needed.

Wire it in as the first step of the published-at chain:

  1. Abbreviated metadata \`modified\` shortcut — cheap, usually
     already on disk from resolution. If modified < cutoff, return
     it as a conservative upper-bound publish time.
  2. Local FULL_META_DIR for exact per-version times.
  3. npm attestation endpoint.
  4. Full metadata fetch.

Factored \`fetchFullMetadataCached\` to share a private helper with
the new \`fetchAbbreviatedMetadataCached\`; both do conditional GETs
against their respective on-disk mirrors. The abbreviated fetch is
dedup'd per (registry, name), and on a non-frozen install hits the
disk mirror at headers-only cost because the resolver already populated
it. On a frozen-lockfile install with cold caches, the fetch is still
cheaper than full metadata.
@zkochan zkochan marked this pull request as ready for review May 17, 2026 12:23
Copilot AI review requested due to automatic review settings May 17, 2026 12:23
@coderabbitai coderabbitai Bot added area: lockfile area: supply chain security Issues related to minimumReleaseAge, blockExoticSubdeps, build script safety, and trust policies. labels May 17, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (3)
resolving/npm-resolver/src/fetchAttestationPublishedAt.ts (1)

35-40: ⚡ Quick win

Reduce positional parameters in fetchAttestationPublishedAt

This signature currently uses 4 positional args; please collapse pkgName + version into a single options object to match repo conventions and make call sites less error-prone.

♻️ Proposed refactor
 export async function fetchAttestationPublishedAt (
   fetchOpts: FetchMetadataFromFromRegistryOptions,
-  pkgName: string,
-  version: string,
+  spec: { pkgName: string, version: string },
   opts: FetchAttestationOptions
 ): Promise<string | undefined> {
-  const url = `${opts.registry.replace(/\/$/, '')}/-/npm/v1/attestations/${pkgName}@${version}`
+  const url = `${opts.registry.replace(/\/$/, '')}/-/npm/v1/attestations/${spec.pkgName}@${spec.version}`
   const retryOperation = retry.operation(fetchOpts.retry)
   return new Promise<string | undefined>((resolve) => {

As per coding guidelines: "Functions should have no more than two or three arguments. If a function needs more parameters, use a single options object instead."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@resolving/npm-resolver/src/fetchAttestationPublishedAt.ts` around lines 35 -
40, The function fetchAttestationPublishedAt currently takes four positional
args; change its signature to accept (fetchOpts:
FetchMetadataFromFromRegistryOptions, pkg: { name: string; version: string } |
PackageIdentifier, opts: FetchAttestationOptions) so pkgName and version are
collapsed into a single package identifier object (or define a PackageIdentifier
type), then update the implementation to read pkg.name and pkg.version and
update all call sites to pass the new object instead of two separate strings;
also update any related type definitions/imports (and tests) that reference
fetchAttestationPublishedAt to use the new parameter shape.
resolving/npm-resolver/src/createNpmResolutionVerifier.ts (1)

245-250: ⚡ Quick win

Collapse (registry, name, version) into one lookup object in helper signatures

The new helper pair repeats 4 positional parameters; wrapping the package lookup fields into a single object will align with project conventions and reduce call-site churn.

♻️ Proposed refactor
+interface PublishedAtLookupKey {
+  registry: string
+  name: string
+  version: string
+}
+
 async function fetchPublishedAt (
   context: PublishedAtLookupContext,
-  registry: string,
-  name: string,
-  version: string
+  key: PublishedAtLookupKey
 ): Promise<string | undefined> {
-  const cacheKey = `${registry}\x00${name}\x00${version}`
+  const cacheKey = `${key.registry}\x00${key.name}\x00${key.version}`
   let cachedPromise = context.publishedAtCache.get(cacheKey)
   if (cachedPromise == null) {
-    cachedPromise = resolvePublishedAt(context, registry, name, version)
+    cachedPromise = resolvePublishedAt(context, key)
     context.publishedAtCache.set(cacheKey, cachedPromise)
   }
   return cachedPromise
 }

 async function resolvePublishedAt (
   context: PublishedAtLookupContext,
-  registry: string,
-  name: string,
-  version: string
+  key: PublishedAtLookupKey
 ): Promise<string | undefined> {
-  const abbreviatedShortcut = await tryAbbreviatedModifiedShortcut(context, registry, name)
+  const abbreviatedShortcut = await tryAbbreviatedModifiedShortcut(context, key.registry, key.name)
   if (abbreviatedShortcut != null) return abbreviatedShortcut

-  const localTime = await readLocalMetaTime(context, registry, name)
-  if (localTime?.[version]) return localTime[version]
+  const localTime = await readLocalMetaTime(context, key.registry, key.name)
+  if (localTime?.[key.version]) return localTime[key.version]

As per coding guidelines: "Functions should have no more than two or three arguments. If a function needs more parameters, use a single options object instead."

Also applies to: 260-265

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@resolving/npm-resolver/src/createNpmResolutionVerifier.ts` around lines 245 -
250, Collapse the repeated positional parameters (registry, name, version) into
a single lookup object in the helper signatures: change
fetchPublishedAt(context: PublishedAtLookupContext, registry: string, name:
string, version: string) to accept a single lookup parameter (e.g., lookup: {
registry: string; name: string; version: string }) plus the context, update the
function body to use lookup.registry/lookup.name/lookup.version, and update all
call sites accordingly; apply the same refactor to the other helper in the file
that accepts the same three parameters (the helper around lines 260-265) so both
signatures and callers use the unified lookup object.
.changeset/attestation-first-min-release-age.md (1)

8-8: ⚡ Quick win

Clarify the fallback flow description.

The current wording suggests that both "local mirror has timestamp" and "attestation 404s/errors" trigger a fallback, but per the stack context, the local mirror is checked before the attestation endpoint is tried. If the local mirror has the timestamp, attestation is never attempted (no fallback needed). The actual fallback is: when the local mirror is cold and the attestation endpoint fails (404/errors), then the full metadata is fetched.

Consider rewording to: "The publish time comes from bundle.verificationMaterial.tlogEntries[].integratedTime (the Rekor inclusion time, a couple of seconds after the actual publish — close enough for a policy that operates in minutes/hours/days). The attestation endpoint is tried when the local full-metadata mirror is cold; if the endpoint returns 404 or errors, the verifier fetches the full metadata document. Sigstore signature verification is not performed; the trust model is unchanged versus reading the registry's time field #11687."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.changeset/attestation-first-min-release-age.md at line 8, The paragraph
should be rewritten to clarify order: state that publish time is taken from
bundle.verificationMaterial.tlogEntries[].integratedTime, that the verifier
checks the local full-metadata mirror first and only tries the attestation
endpoint when the local mirror is cold, and that if the attestation endpoint
returns 404 or errors the verifier falls back to the existing
fetchFullMetadataCached path; also keep the note that Sigstore signature
verification is not performed and the trust model is unchanged versus reading
the registry's time field.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.changeset/attestation-first-min-release-age.md:
- Line 8: The paragraph should be rewritten to clarify order: state that publish
time is taken from bundle.verificationMaterial.tlogEntries[].integratedTime,
that the verifier checks the local full-metadata mirror first and only tries the
attestation endpoint when the local mirror is cold, and that if the attestation
endpoint returns 404 or errors the verifier falls back to the existing
fetchFullMetadataCached path; also keep the note that Sigstore signature
verification is not performed and the trust model is unchanged versus reading
the registry's time field.

In `@resolving/npm-resolver/src/createNpmResolutionVerifier.ts`:
- Around line 245-250: Collapse the repeated positional parameters (registry,
name, version) into a single lookup object in the helper signatures: change
fetchPublishedAt(context: PublishedAtLookupContext, registry: string, name:
string, version: string) to accept a single lookup parameter (e.g., lookup: {
registry: string; name: string; version: string }) plus the context, update the
function body to use lookup.registry/lookup.name/lookup.version, and update all
call sites accordingly; apply the same refactor to the other helper in the file
that accepts the same three parameters (the helper around lines 260-265) so both
signatures and callers use the unified lookup object.

In `@resolving/npm-resolver/src/fetchAttestationPublishedAt.ts`:
- Around line 35-40: The function fetchAttestationPublishedAt currently takes
four positional args; change its signature to accept (fetchOpts:
FetchMetadataFromFromRegistryOptions, pkg: { name: string; version: string } |
PackageIdentifier, opts: FetchAttestationOptions) so pkgName and version are
collapsed into a single package identifier object (or define a PackageIdentifier
type), then update the implementation to read pkg.name and pkg.version and
update all call sites to pass the new object instead of two separate strings;
also update any related type definitions/imports (and tests) that reference
fetchAttestationPublishedAt to use the new parameter shape.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7a2db0e7-fffd-44a0-b869-8e01511b19ba

📥 Commits

Reviewing files that changed from the base of the PR and between 5dc8be8 and 9d7d66c.

📒 Files selected for processing (6)
  • .changeset/attestation-first-min-release-age.md
  • cspell.json
  • resolving/npm-resolver/src/createNpmResolutionVerifier.ts
  • resolving/npm-resolver/src/fetchAttestationPublishedAt.ts
  • resolving/npm-resolver/src/fetchFullMetadataCached.ts
  • resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Agent
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{ts,tsx,js,jsx}: Use Standard Style with modifications: trailing commas are used, functions are preferred over classes, functions are declared after they are used (hoisting is relied upon).
Functions should have no more than two or three arguments. If a function needs more parameters, use a single options object instead.
Maintain import order: (1) Standard libraries, (2) External dependencies (sorted alphabetically), (3) Relative imports.

Files:

  • resolving/npm-resolver/src/fetchAttestationPublishedAt.ts
  • resolving/npm-resolver/src/createNpmResolutionVerifier.ts
  • resolving/npm-resolver/src/fetchFullMetadataCached.ts
  • resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts
**/*.test.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Do not use instanceof Error for error type checking in Jest tests. Use util.types.isNativeError() instead, as Jest runs tests in a VM context where instanceof checks can fail across realms.

Files:

  • resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts
🧠 Learnings (1)
📚 Learning: 2026-05-14T09:04:00.133Z
Learnt from: zkochan
Repo: pnpm/pnpm PR: 11622
File: resolving/npm-resolver/test/publishedBy.test.ts:350-354
Timestamp: 2026-05-14T09:04:00.133Z
Learning: In the pnpm/pnpm repository, ESLint is the authoritative style linter. Do not raise review findings for missing trailing commas in multiline function calls (e.g., `fs.writeFileSync(...)`) when this repo’s ESLint configuration does not report them and lint passes. Prefer deferring to the ESLint results for this specific trailing-comma rule rather than enforcing it manually in code review.

Applied to files:

  • resolving/npm-resolver/src/fetchAttestationPublishedAt.ts
  • resolving/npm-resolver/src/createNpmResolutionVerifier.ts
  • resolving/npm-resolver/src/fetchFullMetadataCached.ts
  • resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts
🔇 Additional comments (4)
resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts (1)

1-213: LGTM!

resolving/npm-resolver/src/fetchFullMetadataCached.ts (1)

1-99: LGTM!

.changeset/attestation-first-min-release-age.md (1)

1-6: LGTM!

cspell.json (1)

280-280: LGTM!

Also applies to: 308-308, 340-340

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: lockfile area: supply chain security Issues related to minimumReleaseAge, blockExoticSubdeps, build script safety, and trust policies.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants