• Seed the registry mock with real npm packages that tests dist-tag/publish over.
• Route fixtures and published test names by exact package name to avoid scope collisions.
• Restore default unpublish ACL behavior to match prior registry-mock contract.

graph TD
  A["Jest globalSetup.js"] --> B["pnpr-prepare --mirror"] --> F[("Registry storage")]
  B --> C["upstream_mirror.rs"] --> D{{"registry.npmjs.org"}}
  C --> E[("upstream-cache/")]
  I["upstream-mirror.yaml"] --> C
  H["pnpr config.yaml"] --> G["pnpr server"]
  F --> G

  subgraph Legend
    direction LR
    _p["Process/Code"] ~~~ _s[("Storage")] ~~~ _e{{"External"}}
  end

Loading

The following are alternative approaches to this PR:

1. Reintroduce overlay/fall-through behavior for upstream-routed writes

• ➕ Avoids downloading/caching upstream tarballs during test setup
• ➕ Keeps pnpr-prepare simpler; fewer moving parts
• ➖ Violates the declared-provenance/no-fallthrough mounts model this PR explicitly preserves
• ➖ Harder to reason about origin correctness; can mask routing mistakes
• ➖ Still needs careful handling for integrity-locked fixtures and real-code execution expectations

2. Rewrite tests to avoid dist-tag/publish-over real npm package names

• ➕ Eliminates the need for mirroring upstream packages
• ➕ Reduces coupling to npm registry content and availability
• ➖ Large, invasive test refactor across many suites
• ➖ Loses coverage of real-world behaviors that currently depend on known package names/graphs
• ➖ Doesn’t address scope-route collisions unless routes are still tightened

3. Vendor minimal local fixture packages instead of downloading real npm bytes

• ➕ Fully offline/deterministic test setup
• ➕ No external HTTP dependency in pnpr-prepare
• ➖ Incompatible with tests that assert real tarball integrity, execute real code, or rely on real dependency graphs (including circular ones)
• ➖ Ongoing maintenance burden to keep fixtures equivalent to upstream behavior

Recommendation: Keep the PR’s approach: mirror only the specific upstream package names that tests must write to, and route them by exact name to the hosted mount. This preserves the mounts model (single declared origin, no fall-through), avoids wildcard route footguns, and satisfies tests that require real upstream bytes/integrities while keeping the blast radius limited to the with-registry harness.

Enhancement (3) +252 / -1

pnpr-prepare.rsAdd --mirror flag to pnpr-prepare +9/-1

Add --mirror flag to pnpr-prepare

• Extends the CLI with an optional --mirror manifest path. When provided, pnpr-prepare seeds upstream-mirrored packages into the generated storage alongside local fixtures.

pnpr/crates/pnpr-fixtures/src/bin/pnpr-prepare.rs

lib.rsExpose upstream mirror materialization and cache directory helper +12/-0

Expose upstream mirror materialization and cache directory helper

• Wires in the new upstream_mirror module, re-exporting the materializer and adding upstream_cache_dir() under target/pnpr-fixtures. Keeps cache lifecycle aligned with cargo target cleanup.

pnpr/crates/pnpr-fixtures/src/lib.rs

upstream_mirror.rsImplement upstream packument/tarball mirroring with caching and integrity checks +231/-0

Implement upstream packument/tarball mirroring with caching and integrity checks

• Adds a materializer that downloads upstream packuments and all version tarballs for mirrored package names, caches them under target/, and writes them into verdaccio-shaped hosted storage. Validates required versions exist and verifies sha512 dist.integrity when present; uses bounded concurrency and write-then-rename caching for safety.

pnpr/crates/pnpr-fixtures/src/upstream_mirror.rs

Bug fix (2) +101 / -7

globalSetup.jsSeed pnpr registry storage with upstream mirror during Jest setup +9/-2

Seed pnpr registry storage with upstream mirror during Jest setup

• Extends the with-registry Jest global setup to pass a new --mirror manifest into pnpr-prepare when building the temp storage. Documents that mirroring is specific to this harness and not used by pacquet’s in-process registry.

pnpm11/utils/jest-config/with-registry/globalSetup.js

config.yamlRoute hosted names by exact match and restore default unpublish ACL +92/-5

Route hosted names by exact match and restore default unpublish ACL

• Replaces broad scope routing for @pnpm/* and @zkochan/* with explicit fixture package names, and enumerates upstream-mirrored and test-published package names to ensure they are hosted. Restores a '**' ACL rule to allow authenticated unpublish for non-tightened packages, matching historical registry-mock expectations.

pnpr/crates/pnpr/config.yaml

Tests (3) +25 / -10

lockfile-verified.jsonlAdd lockfile verification records for git-protocol fixture +3/-0

Add lockfile verification records for git-protocol fixture

• Introduces cached lockfile verification entries used by the pnpm11 fixture suite. Captures verification policy and timestamps for multiple lockfile states.

pnpm11/fixtures/with-git-protocol-dep/cache/lockfile-verified.jsonl

pnpm-lock.yamlUpdate git-protocol fixture lockfile for new dependency/integrity shape +11/-3

Update git-protocol fixture lockfile for new dependency/integrity shape

• Adds the is-number dependency and updates the git-hosted resolution metadata for is-negative to include integrity and gitHosted markers. Adjusts snapshots accordingly to match the new lockfile format.

pnpm11/fixtures/with-git-protocol-dep/pnpm-lock.yaml

tests.rsUpdate config test to assert hosted routing for mirrored/published names +11/-7

Update config test to assert hosted routing for mirrored/published names

• Expands the bundled-config parsing test to verify that fixture scopes, mirrored packages (e.g. lodash), and published test names resolve to the local hosted mount, while unrelated packages (e.g. react) still proxy to npmjs.

pnpr/crates/pnpr/src/config/tests.rs

Other (2) +75 / -8

upstream-mirror.yamlDefine upstream packages and versions required by tests +64/-0

Define upstream packages and versions required by tests

• Adds a manifest listing real npm package names and versions that tests dist-tag/publish over or assert by integrity. Documents mirroring semantics (all versions mirrored; listed versions are existence assertions) and cache behavior.

pnpr/.fixtures/upstream-mirror.yaml

Cargo.tomlAdd async HTTP + YAML deps for upstream mirroring +11/-8

Add async HTTP + YAML deps for upstream mirroring

• Introduces reqwest, tokio, and serde-saphyr dependencies needed to download and parse the upstream mirror manifest and fetch packuments/tarballs.

pnpr/crates/pnpr-fixtures/Cargo.toml

Walkthrough

The PR tightens pnpr routing and ACL rules, adds new fixture packages, updates pnpm tests to use those fixtures, and changes registry-mock package metadata lookup to scan namespaced public cache paths.

Changes

pnpr routing tightening and fixture-based test migration

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

• test: failures on many unrelated PRs - registry-mock regression from #12747 #12771: The pnpr routing and fixture changes align with the same registry-mock route breakage described in this issue.

Possibly related PRs

• pnpm/pnpm#12747: The main PR refines the RFC #13 mount-model routing by tightening mounts.main package-pattern allowlists and adjusting per-package ACL catch-all behavior in pnpr/crates/pnpr/config.yaml and related routing code.
• pnpm/pnpm#12205: Related proxy-cache metadata lookup changes touched the same registry-mock path resolution area.

Suggested labels: product: pnpm@11, product: pnpr

• X
• Mastodon
• Reddit
• LinkedIn

Description

pnpr’s bundled config.yaml was updated to stop routing whole real npm scopes (@pnpm/*,
@zkochan/*) to the hosted store, but Config::proxy() still uses REGISTRY_MOCK_LOCAL_PATTERNS
with those broad scope wildcards. Any code path that uses Config::proxy() (instead of parsing the
bundled YAML) will still resolve packages like @pnpm/error to the hosted mount rather than the npm
upstream, diverging from the fixed behavior and risking the same 404 routing break in those
environments.

Code

pnpr/crates/pnpr/config.yaml[R100-125]

+          # Scopes seeded from the in-repo fixture sources
+          # (`pnpr/.fixtures/packages/`). Scopes shared with real, active npm
+          # scopes (`@pnpm`, `@zkochan`) are routed by exact name instead —
+          # see below — so proxied dependency trees can still pull the scope's
+          # other real packages from npm.
       - '@foo/*'
       - '@having/*'
       - '@jsr/*'
-          - '@pnpm/*'
       - '@pnpm.e2e/*'
       - '@private/*'
       - '@scoped/*'
-          - '@zkochan/*'
       - 'create-touch-file-one-bin'
+          # The fixture packages in the real `@pnpm` and `@zkochan` scopes,
+          # each routed by exact name.
+          - '@pnpm/plugin-pnpmfile'
+          - '@pnpm/postinstall-modifies-source'
+          - '@pnpm/x'
+          - '@pnpm/xyz'
+          - '@pnpm/xyz-parent'
+          - '@pnpm/xyz-parent-parent'
+          - '@pnpm/xyz-parent-parent-parent'
+          - '@pnpm/xyz-parent-parent-parent-parent'
+          - '@pnpm/xyz-parent-parent-with-xyz'
+          - '@pnpm/y'
+          - '@pnpm/z'
+          - '@zkochan/test-pnpm-issue219'

Evidence

The PR’s bundled YAML routes only exact @pnpm/* fixture names to local, but the Rust proxy
constructor still declares @pnpm/* and @zkochan/* as locally hosted patterns. Because pnpr’s
pattern parser treats @scope/* as a scope-wide matcher, Config::proxy() will still route any
@pnpm/ (including @pnpm/error) to local, contradicting the new bundled-config test that
expects @pnpm/error to proxy to npmjs.

pnpr/crates/pnpr/config.yaml[99-125]
pnpr/crates/pnpr/src/config.rs[1148-1162]
pnpr/crates/pnpr/src/mount.rs[48-96]
pnpr/crates/pnpr/src/config/tests.rs[433-456]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The bundled `pnpr` default YAML config now routes only specific fixture packages in real npm scopes (e.g. `@pnpm/y`) to the hosted mount and proxies the rest of the scope to npmjs. However, the Rust `Config::proxy()` constructor still hard-codes broad patterns (`@pnpm/*`, `@zkochan/*`) that route the entire scopes to the hosted mount, which reintroduces the original breakage (real packages under those scopes won’t proxy).
### Issue Context
- `Config::proxy()` is explicitly documented as mirroring the bundled `config.yaml` router; after this PR, it no longer does.
- Routing uses first-match semantics and `@scope/*` matches any well-formed `@scope/name`.
### Fix Focus Areas
- pnpr/crates/pnpr/src/config.rs[1148-1162]
- pnpr/crates/pnpr/src/config.rs[1190-1206]
- pnpr/crates/pnpr/src/config/tests.rs[397-417]
### Suggested fix
1. Update `REGISTRY_MOCK_LOCAL_PATTERNS` to match the new bundled `config.yaml` intent:
- Remove `@pnpm/*` and `@zkochan/*`.
- Add exact-name patterns for the fixture packages that live in those real scopes (the same set as in `config.yaml`).
- If `Config::proxy()` must also support test-published unscoped names, include those patterns too, or explicitly document that `Config::proxy()` is *not* identical to the bundled YAML routing.
2. Strengthen `proxy_constructor_serves_fixtures_locally_and_proxies_the_rest` to assert at least one real-scope package proxies (e.g. `@pnpm/error` -> upstream) and one fixture in the scope is hosted (e.g. `@pnpm/y` -> hosted), so this doesn’t regress again.
(Alternative) Consider making `Config::proxy()` derive its router patterns by parsing `DEFAULT_CONFIG_YAML` (and then overriding listen/storage), eliminating the dual-source-of-truth pattern list.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

getIntegrity() now calls fs.readdirSync() to enumerate .pnpr-cache/~public namespaces on every
call and on every retry attempt, even when the hosted packument path exists and would succeed
immediately. This adds avoidable synchronous filesystem work on a heavily-used test helper (e.g.
assertStore() calls getIntegrity() for store index keys), increasing test runtime and variance.

Code

pnpm11/testing/registry-mock/src/index.ts[R105-112]

+    // Rebuilt on every attempt: the proxy-cache namespace directory itself is
+    // created lazily along with the first cached packument, so it may not
+    // exist yet on the first attempt.
+    const candidatePaths = [
+      path.join(storage, pkgName, 'package.json'),
+      ...listPublicProxyNamespaces(path.join(storage, PROXY_CACHE_DIR, '~public'))
+        .map((namespace) => path.join(namespace, pkgName, 'package.json')),
+    ]

Evidence

The new implementation unconditionally calls listPublicProxyNamespaces() (which uses
fs.readdirSync) to build candidatePaths inside the retry loop. assertStore() (used by many
tests) calls getIntegrity() per package/version when computing store index keys, multiplying the
added synchronous directory scan cost across the suite.

pnpm11/testing/registry-mock/src/index.ts[94-136]
pnpm11/utils/assert-store/src/index.ts[17-33]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`getIntegrity()` rebuilds `candidatePaths` each retry by always enumerating public proxy namespaces via `fs.readdirSync()`. For the common case (hosted fixture packuments), this directory scan is unnecessary overhead and is paid per `getIntegrity()` call.
### Issue Context
`getIntegrity()` is used broadly in tests (directly and via helpers like `assertStore()`), so even small per-call synchronous filesystem overhead can noticeably slow CI.
### Fix Focus Areas
- pnpm11/testing/registry-mock/src/index.ts[94-136]
- pnpm11/__utils__/assert-store/src/index.ts[17-33]
### Suggested fix
Implement a cheap fast path and/or memoization:
1. Try reading the hosted packument first without scanning namespaces:
- Attempt `readPackument([path.join(storage, pkgName, 'package.json')])`.
- Only if that returns `undefined`, then enumerate namespaces and try the proxy candidates.
2. Optionally cache the computed namespace directories after the first successful `readdirSync()` (or cache only once non-empty) to avoid repeated scans.
3. If you keep enumeration, sort the `readdirSync()` results to keep deterministic search order.
This preserves the retry behavior for lazy-created namespaces while eliminating the unconditional per-call directory scan in the common case.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

getIntegrity() enumerates every entry under .pnpr-cache/~public without filtering to
directories, but readPackument() only treats ENOENT as retryable. If ~public contains a stray
file or non-directory entry, reads can throw ENOTDIR/EISDIR and fail the registry-mock test
harness instead of retrying.

Code

pnpm11/testing/registry-mock/src/index.ts[R124-133]

+function listPublicProxyNamespaces (publicCacheDir: string): string[] {
+  let entries: string[]
+  try {
+    entries = fs.readdirSync(publicCacheDir)
+  } catch (err: unknown) {
+    if ((err as NodeJS.ErrnoException).code === 'ENOENT') return []
+    throw err
+  }
+  return entries.map((entry) => path.join(publicCacheDir, entry))
+}

Evidence

The new helper returns unfiltered readdirSync() entries as namespaces, and the packument reader
only skips ENOENT, so a non-directory namespace entry can cause a hard failure. The pnpr Rust
tests for the same cache layout explicitly filter to directories and mention stray files, indicating
this is a real concern for this directory.

pnpm11/testing/registry-mock/src/index.ts[102-146]
pnpr/crates/pnpr/tests/server.rs[46-55]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`listPublicProxyNamespaces()` returns every `readdirSync()` entry as a candidate namespace. If `.pnpr-cache/~public` contains a non-directory entry, subsequent reads of `<entry>/<pkg>/package.json` can raise `ENOTDIR`/`EISDIR`, and `readPackument()` will rethrow (it only treats `ENOENT` as retryable).
### Issue Context
The pnpr Rust tests already treat “stray files” in this directory as plausible and explicitly filter to directories + sort.
### Fix Focus Areas
- pnpm11/testing/registry-mock/src/index.ts[102-146]
- pnpm11/testing/registry-mock/src/index.ts[121-133]
### Suggested fix
- Use `fs.readdirSync(publicCacheDir, { withFileTypes: true })` and keep only `dirent.isDirectory()` entries.
- Optionally sort entries for determinism.
- Optionally treat `ENOTDIR`/`EISDIR` like `ENOENT` in `readPackument()` (skip and continue) to be resilient to unexpected filesystem state.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

The bundled pnpr default config now grants unpublish: $authenticated for **, so any
authenticated user can unpublish any hosted package. Because pnpr falls back to this bundled config
when no config file exists, this broad destructive permission becomes the out-of-box server
behavior, increasing the blast radius of a compromised token/user.

Code

pnpr/crates/pnpr/config.yaml[R195-198]

+  '**':
+    access: $all
+    publish: $authenticated
+    unpublish: $authenticated

Evidence

The diff adds a catch-all rule granting authenticated unpublish. The pnpr CLI docs and config code
show the bundled YAML is used as the default when no config file exists, so this is a
default-behavior permission expansion rather than only a test-local change.

pnpr/crates/pnpr/config.yaml[179-199]
pnpr/crates/pnpr/src/main.rs[9-13]
pnpr/crates/pnpr/src/config.rs[20-40]
pnpr/crates/pnpr/src/config.rs[1961-1965]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The bundled default `config.yaml` now includes a catch-all `packages: '**'` rule with `unpublish: $authenticated`, making destructive writes broadly available to any authenticated account when pnpr is run with default config resolution.
### Issue Context
pnpr uses `DEFAULT_CONFIG_YAML` as a fallback when no config file is found, so changes here affect out-of-box behavior, not just the test harness.
### Fix Focus Areas
- pnpr/crates/pnpr/config.yaml[179-199]
- pnpr/crates/pnpr/src/main.rs[9-13]
- pnpr/crates/pnpr/src/config.rs[20-40]
### Suggested fix options
1) Keep bundled defaults safer: remove `unpublish` from the `**` rule (or remove the `**` rule entirely) and have the *test harness* inject the permissive unpublish rule when generating its test config (similar to how it already rewrites `max_users`).
2) If tests require unpublish broadly, scope it to the explicit hosted test-published names and `@pnpmtest/*` (and any other known local patterns), leaving `**` without unpublish to avoid granting global destructive permission by default.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

download_tarballs() spawns one Tokio task per tarball job and holds them all in a JoinSet; for
packages with many versions this adds avoidable scheduling/memory overhead even though concurrency
is already limited by a semaphore. This can increase fixture build latency/variance in CI when
mirroring a large version set.

Code

pnpr/crates/pnpr-fixtures/src/upstream_mirror.rs[R175-189]

+async fn download_tarballs(client: &reqwest::Client, jobs: Vec<TarballJob>) {
+    let semaphore = Arc::new(tokio::sync::Semaphore::new(CONCURRENT_DOWNLOADS));
+    let mut tasks = tokio::task::JoinSet::new();
+    for job in jobs {
+        let client = client.clone();
+        let semaphore = Arc::clone(&semaphore);
+        tasks.spawn(async move {
+            let _permit = semaphore.acquire().await.expect("semaphore is never closed");
+            job.run(client).await;
+        });
+    }
+    while let Some(result) = tasks.join_next().await {
+        result.expect("mirror a tarball");
+    }
+}

Evidence

The code enqueues all jobs into a JoinSet via spawn, while separately using a semaphore for
concurrency, which means the per-job task overhead is not necessary to achieve the concurrency cap.

pnpr/crates/pnpr-fixtures/src/upstream_mirror.rs[175-189]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`download_tarballs()` spawns a task per job and relies on a semaphore to bound concurrency. For large `jobs` lists, task creation and JoinSet bookkeeping become extra overhead.
### Issue Context
Concurrency is already explicitly capped by `CONCURRENT_DOWNLOADS`, so you can process jobs with bounded concurrency without spawning N tasks.
### Fix Focus Areas
- pnpr/crates/pnpr-fixtures/src/upstream_mirror.rs[175-189]
### What to change
- Replace “spawn everything + semaphore” with a bounded-concurrency loop/stream pattern (e.g., `for_each_concurrent(CONCURRENT_DOWNLOADS, ...)`) so only ~N in-flight futures exist at once.
- Keep the current failure semantics (a single tarball failure should fail the mirror run).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.