feat(pnpr): separate the proxied upstream cache from published packages#12195
Merged
Conversation
pnpr stored proxied upstream packuments/tarballs and locally-published packages in the same on-disk tree, with no way to tell them apart. That made the proxy cache impossible to clear without risking published packages, and forced every backup/upgrade to treat the disposable mirror as precious data. Split storage into two physically separate roots: - `storage` — authoritative source of truth: packages published to this server and content served in static mode. Served as-is and never overwritten by an upstream refresh, so published versions can't be masked or lost. - `cache` — disposable mirror of upstream registries plus the install-accelerator store. Safe to wipe at any time; self-heals on the next request. Defaults to a `.pnpr-cache` subdirectory of `storage`; set the YAML `cache:` key or `--cache` to put it on a separate, ephemeral volume. Reads prefer the authoritative store; publish, unpublish, packument updates and dist-tag changes write to it, while upstream refreshes write only to the cache. Closes #12194.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR implements dual-root cache architecture for pnpr, separating authoritative hosted packages from a disposable proxy cache. It adds a ChangesHosted storage and proxy cache separation
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review Summary by QodoSplit pnpr storage into separate authoritative and disposable cache roots
WalkthroughsDescription• Split on-disk storage into two separate roots: storage (authoritative published packages) and cache (disposable proxy mirror) • Authoritative packages are never overwritten by upstream refreshes; cache is safe to wipe anytime • Added cache: config key and --cache CLI flag to override default .pnpr-cache subdirectory location • Updated all read/write operations to route through appropriate store based on content origin • Published packages survive full cache wipes; install-accelerator store moved under cache root Diagramflowchart LR
A["Published Packages<br/>Authoritative"] -->|"Never overwritten"| B["storage root<br/>Source of Truth"]
C["Proxied Upstream<br/>Disposable"] -->|"Safe to wipe"| D["cache root<br/>Ephemeral"]
E["Config"] -->|"storage: path"| B
E -->|"cache: path<br/>default: .pnpr-cache"| D
F["Read Operations"] -->|"Prefer published"| B
F -->|"Fallback to cache"| D
G["Write Operations"] -->|"Publish/Unpublish"| B
G -->|"Upstream refresh"| D
File Changes1. pnpr/crates/pnpr/src/cache.rs
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #12195 +/- ##
==========================================
+ Coverage 87.56% 87.70% +0.13%
==========================================
Files 269 271 +2
Lines 30817 31157 +340
==========================================
+ Hits 26984 27325 +341
+ Misses 3833 3832 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Rename the source-of-truth store's vocabulary from "published" to "hosted" — matching the registry-server convention (Nexus hosted/proxy) and covering both API-published and static-served content. Internal only: the `Cache` methods, the search root accessor, and doc comments. The user-facing `storage:` / `cache:` YAML keys are unchanged.
Contributor
Integrated-Benchmark Report (Linux)Each scenario has pacquet rows (direct install) and pnpr rows (the same client through the pnpr install accelerator), so pnpr@HEAD vs pacquet@HEAD is the pnpr-vs-direct ratio. Cold-store scenarios wipe the client store between runs (warm server); hot-store scenarios keep it warm. The pacquet@HEAD rows feed the pacquet Bencher testbed; the pnpr@HEAD rows feed the pnpr testbed. Scenario: Isolated linker: fresh restore, cold cache + cold store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 4.790404828200001,
"stddev": 0.06125856696633137,
"median": 4.7868212831,
"user": 2.4032157599999997,
"system": 3.7893475199999997,
"min": 4.7094633641,
"max": 4.8997309351,
"times": [
4.862281746100001,
4.7889661201000004,
4.8168202721,
4.8997309351,
4.7371939311,
4.7309558011,
4.748367716100001,
4.7094633641,
4.784676446100001,
4.825591950100001
]
},
{
"command": "pacquet@main",
"mean": 4.788193398900001,
"stddev": 0.045811918766629774,
"median": 4.771975490100001,
"user": 2.37808566,
"system": 3.78785272,
"min": 4.7292577581,
"max": 4.882857378100001,
"times": [
4.7672048651,
4.826690468100001,
4.8225644221,
4.7292577581,
4.7662832121,
4.7419967521,
4.8011281531000005,
4.7765832451,
4.767367735100001,
4.882857378100001
]
},
{
"command": "pnpr@HEAD",
"mean": 2.0380261821000003,
"stddev": 0.07171756088158213,
"median": 2.0413213491,
"user": 2.5321917599999995,
"system": 3.3167824200000005,
"min": 1.8970105011,
"max": 2.1210501931,
"times": [
2.0988661980999996,
2.0977986951,
1.9539491571,
1.8970105011,
2.1001976660999997,
2.1210501931,
2.0345798651,
2.0203835111,
2.0083632011,
2.0480628331
]
},
{
"command": "pnpr@main",
"mean": 2.0207473113,
"stddev": 0.08192538931670618,
"median": 1.9893146886000002,
"user": 2.5861324599999995,
"system": 3.30867742,
"min": 1.9530258471,
"max": 2.2083289621,
"times": [
1.9940076021000002,
1.9530258471,
2.2083289621,
1.9557189581,
1.9717904751,
2.0460193551,
1.9846217751000002,
1.9948727011,
2.1167028191,
1.9823846181
]
}
]
}Scenario: Isolated linker: fresh restore, hot cache + hot store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 0.67079707924,
"stddev": 0.030042404707773585,
"median": 0.6642141695399999,
"user": 0.37160288,
"system": 1.3170284200000002,
"min": 0.6449645415399999,
"max": 0.75346251154,
"times": [
0.75346251154,
0.6672178505399999,
0.67124593954,
0.66327793854,
0.66034868054,
0.66857589354,
0.6449645415399999,
0.65987974554,
0.65384729054,
0.66515040054
]
},
{
"command": "pacquet@main",
"mean": 0.65454617984,
"stddev": 0.018755412248005914,
"median": 0.6517743095399999,
"user": 0.35981118000000006,
"system": 1.3184646199999999,
"min": 0.63947405254,
"max": 0.70500110454,
"times": [
0.70500110454,
0.65202643054,
0.63947405254,
0.65168602454,
0.64119806254,
0.64611747454,
0.66036364354,
0.65230911754,
0.65186259454,
0.6454232935399999
]
},
{
"command": "pnpr@HEAD",
"mean": 0.6466108123400001,
"stddev": 0.009110556840642148,
"median": 0.6455309415399999,
"user": 0.35694948000000004,
"system": 1.31315802,
"min": 0.62681177954,
"max": 0.66236449454,
"times": [
0.66236449454,
0.65594125954,
0.64808641754,
0.64809190454,
0.6460974175399999,
0.64441793854,
0.64496446554,
0.64448437354,
0.64484807254,
0.62681177954
]
},
{
"command": "pnpr@main",
"mean": 0.7058149218399998,
"stddev": 0.04384845346630356,
"median": 0.7105775110399999,
"user": 0.35798978,
"system": 1.31168662,
"min": 0.64649906654,
"max": 0.78564451054,
"times": [
0.78564451054,
0.73061420254,
0.69664716554,
0.7245078565399999,
0.64904752654,
0.66726901154,
0.64649906654,
0.73540414654,
0.6922648125399999,
0.73025091954
]
}
]
}Scenario: Isolated linker: fresh install, cold cache + cold store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 2.1270757773200004,
"stddev": 0.03291976187454829,
"median": 2.1434218841200003,
"user": 3.3967876599999998,
"system": 2.99405296,
"min": 2.07504178612,
"max": 2.1637412511200003,
"times": [
2.07504178612,
2.14617611312,
2.14066765512,
2.1637412511200003,
2.0917622751200002,
2.11145392412,
2.08557077212,
2.15785059812,
2.14724657612,
2.15124682212
]
},
{
"command": "pacquet@main",
"mean": 2.13132798362,
"stddev": 0.03389451474585595,
"median": 2.12878486412,
"user": 3.356614859999999,
"system": 3.02464826,
"min": 2.0951755591200003,
"max": 2.2013235621200002,
"times": [
2.15105416612,
2.11008868912,
2.15223857012,
2.2013235621200002,
2.14750210412,
2.0982326271200002,
2.0951755591200003,
2.10009483012,
2.11281094412,
2.14475878412
]
},
{
"command": "pnpr@HEAD",
"mean": 2.14252100172,
"stddev": 0.08072459442752962,
"median": 2.12752724562,
"user": 3.35903516,
"system": 3.0034234599999996,
"min": 2.0715321811200003,
"max": 2.34385524612,
"times": [
2.09679782812,
2.09190414012,
2.34385524612,
2.0724044581200003,
2.13732146612,
2.14475553312,
2.1946825001200003,
2.0715321811200003,
2.15422363912,
2.11773302512
]
},
{
"command": "pnpr@main",
"mean": 2.12429574192,
"stddev": 0.032303140764434427,
"median": 2.11160263462,
"user": 3.396480859999999,
"system": 3.00246796,
"min": 2.0907444501200003,
"max": 2.19508650712,
"times": [
2.19508650712,
2.14422300912,
2.10386434912,
2.11605875412,
2.1136763851200002,
2.15991540012,
2.10827360812,
2.10158607212,
2.10952888412,
2.0907444501200003
]
}
]
}Scenario: Isolated linker: fresh install, hot cache + hot store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 1.24941705376,
"stddev": 0.013728215031570734,
"median": 1.25167529266,
"user": 1.36844616,
"system": 1.67459542,
"min": 1.22735341766,
"max": 1.2683485566600001,
"times": [
1.24544914566,
1.2314453256600002,
1.22735341766,
1.24963250366,
1.25465307266,
1.2683485566600001,
1.2384428306600002,
1.2537180816600002,
1.26226955766,
1.26285804566
]
},
{
"command": "pacquet@main",
"mean": 1.27348231026,
"stddev": 0.058221142119793866,
"median": 1.2556322891600002,
"user": 1.3822258600000001,
"system": 1.6882150200000001,
"min": 1.23318074266,
"max": 1.43297379666,
"times": [
1.27174958566,
1.23318074266,
1.2564209216600002,
1.2462617576600001,
1.2804293366600001,
1.43297379666,
1.24164010466,
1.24190426466,
1.27541893566,
1.25484365666
]
},
{
"command": "pnpr@HEAD",
"mean": 1.25638581146,
"stddev": 0.05074727955531733,
"median": 1.25124657366,
"user": 1.3632949599999997,
"system": 1.68032322,
"min": 1.20901628966,
"max": 1.39321964266,
"times": [
1.20901628966,
1.2558484116600002,
1.25626096166,
1.2539189476600001,
1.22275285066,
1.39321964266,
1.22769654866,
1.25326013966,
1.24923300766,
1.24265131466
]
},
{
"command": "pnpr@main",
"mean": 1.29124557466,
"stddev": 0.0777892854832095,
"median": 1.27275661866,
"user": 1.3831566599999998,
"system": 1.68853302,
"min": 1.22430708866,
"max": 1.5036370416600002,
"times": [
1.24808047666,
1.27604625766,
1.26067705066,
1.2632782846600001,
1.22430708866,
1.5036370416600002,
1.2827914746600002,
1.30812483466,
1.26969498866,
1.27581824866
]
}
]
} |
Contributor
|
Bencher Report| Branch | pr/12195 |
| Testbed | pacquet |
🚨 1 Alert
| Benchmark | Measure Units | View | Benchmark Result (Result Δ%) | Upper Boundary (Limit %) |
|---|---|---|---|---|
| isolated-linker.fresh-restore.cold-cache.cold-store | Latency seconds (s) | 📈 plot 🚷 threshold 🚨 alert (🔔) | 4.79 s(+83.72%)Baseline: 2.61 s | 3.13 s (153.10%) |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| isolated-linker.fresh-install.cold-cache.cold-store | 📈 view plot 🚷 view threshold | 2,127.08 ms(-7.26%)Baseline: 2,293.70 ms | 2,752.45 ms (77.28%) |
| isolated-linker.fresh-install.hot-cache.hot-store | 📈 view plot 🚷 view threshold | 1,249.42 ms(-14.03%)Baseline: 1,453.34 ms | 1,744.00 ms (71.64%) |
| isolated-linker.fresh-restore.cold-cache.cold-store | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 4,790.40 ms(+83.72%)Baseline: 2,607.40 ms | 3,128.87 ms (153.10%) |
| isolated-linker.fresh-restore.hot-cache.hot-store | 📈 view plot 🚷 view threshold | 670.80 ms(+0.66%)Baseline: 666.42 ms | 799.70 ms (83.88%) |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pnpr/crates/pnpr/src/cache.rs`:
- Around line 131-136: The partial-unpublish flow only calls
remove_hosted_tarball (pnpr::cache::remove_hosted_tarball) so a stale
proxy/cached copy can be served by open_tarball's fallback to self.cache; update
the delete_tarball handler in pnpr/src/server.rs to call the composed remover
that clears both hosted and cached/proxied copies (instead of hosted-only
remove_hosted_tarball) — locate the delete_tarball handler and replace the call
to remove_hosted_tarball with the cache-composed remover (e.g., the cache-level
remove_tarball / composed removal method on the same Cache/Store type) so the
tarball is removed from hosted storage and any cache/proxy layer.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: f6fe548f-05c6-469e-98b9-1b93684743a9
📒 Files selected for processing (2)
pnpr/crates/pnpr/src/cache.rspnpr/crates/pnpr/src/server.rs
🚧 Files skipped from review as they are similar to previous changes (1)
- pnpr/crates/pnpr/src/server.rs
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: Code Coverage
- GitHub Check: Run benchmark on ubuntu-latest
- GitHub Check: Lint and Test (windows-latest)
- GitHub Check: Lint and Test (macos-latest)
- GitHub Check: Lint and Test (ubuntu-latest)
- GitHub Check: Compile & Lint
🧰 Additional context used
📓 Path-based instructions (1)
pnpr/**/pnpr/**/*.rs
📄 CodeRabbit inference engine (pnpr/AGENTS.md)
pnpr/**/pnpr/**/*.rs: Follow the pacquet code-style guide (../pacquet/CODE_STYLE_GUIDE.md) for Rust-level conventions including imports, naming, ownership, and error handling
Follow the pacquet contributing guide (../pacquet/CONTRIBUTING.md) for test layout and Rust conventions
Files:
pnpr/crates/pnpr/src/cache.rs
🧠 Learnings (3)
📓 Common learnings
Learnt from: CR
Repo: pnpm/pnpm PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-25T12:36:42.202Z
Learning: User-visible changes (CLI flags, defaults, environment variables, lockfile/manifest/state-file formats, error codes/messages, log emissions, store layout, hook semantics) in pnpm must be mirrored to pacquet in the same PR
Learnt from: zkochan
Repo: pnpm/pnpm PR: 12189
File: pacquet/crates/package-manager/src/install_with_fresh_lockfile.rs:435-439
Timestamp: 2026-06-04T14:40:25.306Z
Learning: In `pacquet/crates/package-manager/src/install_with_fresh_lockfile.rs` (pnpm/pnpm repo), the pnpr install accelerator always invokes `Install` with `lockfile_only: true` (hard-coded in `pnpr/crates/pnpr/src/install_accelerator/resolve.rs`). Under `lockfile_only: true`:
1. The `PrefetchingResolver` wrapper is skipped — the bare `inner_resolver` is used instead, so `PrefetchContext { config }` is never constructed.
2. The function returns before `CreateVirtualStore` is reached, so `install_package_by_snapshot` and its `config.auth_headers` fetch path are never hit.
pnpr's tarball fetch is handled separately in `resolve::fetch_uncached`, which independently receives the request-scoped `auth_headers`. Therefore, `auth_override` only needs to be threaded into the resolver-side components (NpmResolver, TarballResolver, NamedRegistryResolver) — not into PrefetchingResolver or CreateVirtualStore — on the pnpr path. For local installs (`lockfile_only: false`), `auth_override` is always `None` a...
Learnt from: zkochan
Repo: pnpm/pnpm PR: 11878
File: resolving/npm-resolver/src/createNpmResolutionVerifier.ts:381-418
Timestamp: 2026-05-23T17:30:06.849Z
Learning: In `resolving/npm-resolver/src/pickPackage.ts` (pnpm/pnpm), the resolver's `PackageMetaCache` keys by `name` (abbreviated) and `name:full` (full metadata) only — no registry component is included. This is a pre-existing limitation meaning that if two different registries serve packages of the same name in one install, the cache will only hold the first fetched entry. The `createNpmResolutionVerifier.ts` shares this same cache and inherits the limitation; a `validateSharedMeta` name-check guards against cross-package contamination but cannot distinguish same-named packages from different registries. Tightening to a registry-qualified key would require a coordinated change to the resolver's cache key shape. The Pacquet/Rust side is already registry-qualified (`{registry}\x00{name}:full`).
Learnt from: zkochan
Repo: pnpm/pnpm PR: 12189
File: pacquet/crates/cli/src/cli_args/install.rs:0-0
Timestamp: 2026-06-04T14:55:48.516Z
Learning: In `pacquet/crates/cli/src/cli_args/install.rs` (pnpm/pnpm repo), the `install_via_pnpr` function intentionally forwards the **full** `state.config.auth_headers` map to the pnpr server (not filtered to only the declared default/named registries). This is required for correctness: transitive dependencies can be scope-routed to registries not in the explicit registry list, or pinned to tarball URLs on hosts present in `.npmrc` but not a declared registry. Filtering to the declared registries silently drops tokens those sub-dependencies need, causing 401s on the server. pnpr uses the forwarded map to attach the right token per fetched URL exactly as a local install does (`AuthHeaders::for_url`). The pnpr-server's own credential is sent separately in the `Authorization` header and is excluded from the body map. Do NOT flag this as a credential-leakage issue — the rationale is documented in a comment at both call sites.
Learnt from: CR
Repo: pnpm/pnpm PR: 0
File: pnpr/AGENTS.md:0-0
Timestamp: 2026-05-29T18:03:24.797Z
Learning: Use Conventional Commits with 'pnpr' as the scope in commit messages (e.g., feat(pnpr): ..., fix(pnpr): ...)
Learnt from: CR
Repo: pnpm/pnpm PR: 0
File: pnpr/AGENTS.md:0-0
Timestamp: 2026-05-29T18:03:24.797Z
Learning: Applies to pnpr/**/pnpr/crates/**/Cargo.toml : New registry-only crates must be placed under pnpr/crates/<short-name>/ and named pnpr-<short-name> in Cargo.toml, never using the pacquet- prefix
Learnt from: zkochan
Repo: pnpm/pnpm PR: 11915
File: pacquet/crates/resolving-deps-resolver/src/resolve_dependency_tree.rs:553-617
Timestamp: 2026-05-24T21:11:04.272Z
Learning: In the pacquet Rust port (pnpm/pnpm repo), the `ResolvedPackage.optional` AND-folding on revisit intentionally mirrors pnpm's `resolveDependencies.ts:1627-1648` behavior: only the directly-revisited package's `optional` flag is updated; transitive descendants are not re-walked. pnpm CLI corrects stale optional flags downstream via `copyDependencySubGraph` BFS in `lockfile/pruner/src/index.ts:160-205`, which tracks a `nonOptional` set and re-stamps any package reachable by an all-non-optional path. Pacquet does not yet have this pruner equivalent, so the stale flags flow directly through `dependencies_graph_to_lockfile.rs:409` → `create_virtual_store.rs:762` → `installability.rs:394`. A follow-up to port `copyDependencySubGraph` is planned; until then, do not flag the resolver-layer optional propagation gap as a bug in pacquet PRs — it is intentional parity with pnpm's resolver layer.
Learnt from: zkochan
Repo: pnpm/pnpm PR: 11931
File: pacquet/crates/resolving-npm-resolver/src/create_npm_resolution_verifier.rs:560-589
Timestamp: 2026-05-25T14:58:11.105Z
Learning: In `pacquet/crates/resolving-npm-resolver/src/create_npm_resolution_verifier.rs`, all per-`(registry, name[, version])` caches in `NpmResolutionVerifier` (`published_at`, `full_meta`, `full_meta_for_trust`, `abbreviated_meta`, `local_meta`) intentionally use the same pattern: lock → miss-check → release lock → await fetch/load → re-acquire lock → insert. This uniform pattern is deliberate; do not flag individual caches for using it. The known follow-up improvement (replacing the pattern with `tokio::sync::OnceCell` per key inside a `Mutex<HashMap<…>>`) is tracked as a future structural change to cover all five caches simultaneously.
Learnt from: zkochan
Repo: pnpm/pnpm PR: 12181
File: worker/src/start.ts:504-520
Timestamp: 2026-06-04T06:04:01.216Z
Learning: In pnpm/pnpm's pnpr install accelerator, the `/v1/install` response has a two-level framing structure:
1. **Outer layer** (full HTTP body): `[u32 outer header length][outer header JSON][files payload]` — `fetchFromPnpmRegistry` (pnpr/client/src/fetchFromPnpmRegistry.ts) strips the outer layer with `body.subarray(4 + headerLength)` and passes the remaining bytes to `writeCafsFiles`.
2. **Inner layer** (files payload): the files payload itself starts with its own `[u32 inner json length][inner header JSON]` prefix (built by the server's `build_files_payload` / `empty_files_payload_prefix`), followed by `[64-byte digest][u32 size][1-byte exec][content]` frames and a 64-zero-byte end marker.
`writeCafsFiles` in `worker/src/start.ts` is correct to read `jsonLen = payload.readUInt32BE(0)` and start frames at `offset = 4 + jsonLen` — this skips the inner header. The same two-level structure is mirrored in the Rust reference client (`parse_inline_response` + `write_files_payload`). Do not fla...
Learnt from: zkochan
Repo: pnpm/pnpm PR: 11784
File: pacquet/crates/resolving-deps-resolver/src/hoist_peers.rs:120-133
Timestamp: 2026-05-20T23:08:06.093Z
Learning: Pacquet (pnpm's Rust port) has a cardinal rule: "match pnpm exactly — do not fix pnpm quirks unless the same fix has landed in pnpm first." Review comments should not suggest behavioral deviations from upstream pnpm, even when the upstream behavior appears buggy. If a real bug is identified, it must be fixed upstream first.
📚 Learning: 2026-05-25T14:58:11.105Z
Learnt from: zkochan
Repo: pnpm/pnpm PR: 11931
File: pacquet/crates/resolving-npm-resolver/src/create_npm_resolution_verifier.rs:560-589
Timestamp: 2026-05-25T14:58:11.105Z
Learning: In `pacquet/crates/resolving-npm-resolver/src/create_npm_resolution_verifier.rs`, all per-`(registry, name[, version])` caches in `NpmResolutionVerifier` (`published_at`, `full_meta`, `full_meta_for_trust`, `abbreviated_meta`, `local_meta`) intentionally use the same pattern: lock → miss-check → release lock → await fetch/load → re-acquire lock → insert. This uniform pattern is deliberate; do not flag individual caches for using it. The known follow-up improvement (replacing the pattern with `tokio::sync::OnceCell` per key inside a `Mutex<HashMap<…>>`) is tracked as a future structural change to cover all five caches simultaneously.
Applied to files:
pnpr/crates/pnpr/src/cache.rs
📚 Learning: 2026-05-29T18:03:24.797Z
Learnt from: CR
Repo: pnpm/pnpm PR: 0
File: pnpr/AGENTS.md:0-0
Timestamp: 2026-05-29T18:03:24.797Z
Learning: Applies to pnpr/**/pnpr/crates/**/Cargo.toml : New registry-only crates must be placed under pnpr/crates/<short-name>/ and named pnpr-<short-name> in Cargo.toml, never using the pacquet- prefix
Applied to files:
pnpr/crates/pnpr/src/cache.rs
🔇 Additional comments (1)
pnpr/crates/pnpr/src/cache.rs (1)
101-105: Confirm local search skips.pnpr-cache—hosted_root()is passed into the local search (run_local_search(state.inner.cache.hosted_root(), ...)), andcollect_packument_paths()ignores any top-level entries whose name starts with.(if name_str.starts_with('.') { continue; }), so the default<storage>/.pnpr-cachedirectory won’t be walked or indexed in local search results.
`open_tarball` falls back to the cache store, so deleting only the hosted tarball left a stale proxied copy of the same filename servable on the next GET. Make tarball removal purge both stores, mirroring how package removal already does, and cover it with a test that plants a proxied copy and asserts the version is gone after the delete.
Pair it with `hosted` so the two `Cache` stores read consistently (`hosted` / `cached`), matching the cached-flavoured method names. Kept "cached" rather than "proxied" to avoid overloading the `proxy:` package routing key, which already means "which uplink to proxy from".
The type now owns two stores — an authoritative `hosted` root and a disposable `cached` one — so "Cache" no longer describes it. Rename the struct (and its `cache` module) to `Storage`, and the `AppInner` field to `storage`, leaving the per-store and YAML/CLI cache vocabulary intact.
Close two gaps in the storage-split coverage: - `hosted_packument_is_never_overwritten_by_upstream`: with a proxy upstream, a divergent upstream packument and a zero TTL, the hosted copy is still served and the upstream is never contacted — the core "published versions can't be masked or lost" invariant. - `hosted_tarball_is_preferred_over_a_cached_copy`: when the same filename sits in both stores, `open_tarball` serves the hosted one.
zkochan
added a commit
that referenced
this pull request
Jun 4, 2026
… dir (#12205) #12195 separated pnpr's proxied upstream cache from hosted packages, moving proxied packuments to a `.pnpr-cache` subdirectory of the storage root. The `getIntegrity` test helper still only read the hosted `<storage>/<pkg>/package.json` path, so tests that resolve integrity for packages uplinked from the real npm registry (e.g. store add express, store prune is-negative) failed with ENOENT. Try the hosted location first, then `<storage>/.pnpr-cache/<pkg>/package.json`, keeping the existing retry for the lazy/in-flight cache write.
This was referenced Jun 17, 2026
This was referenced Jun 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Splits pnpr's on-disk storage into two physically separate roots so the disposable proxy cache and the authoritative hosted packages no longer share a lifecycle.
Closes #12194.
Before
Proxied upstream packuments/tarballs and locally-published packages were written to the same
<storage>/<pkg>/tree through a singleCacheabstraction, with no marker distinguishing them. Consequences:After
storage(hosted)cache(proxy)./storage<storage>/.pnpr-cachestorage:/--storagecache:/--cache(point at a separate ephemeral volume)Storagetype wraps twoStoreroots —hosted(authoritative) andcached(disposable). Reads prefer the hosted store; a hosted/static packument is served as-is and never refreshed over.Naming
hosted/cachedfields of theStoragetype (hosted/proxyis the registry-server convention, e.g. Sonatype Nexus).storage:/cache:(nouns that name directories, the clearest register for a setting), which also keeps verdaccio-shaped configs working.Server / deployment
storageon a durable, backed-up volume andcacheon scratch/ephemeral disk (or just leave the default subdir).storage; the cache can start cold.storage.Migration note
Existing proxy deployments have proxied packuments sitting in the old
storageroot. After upgrading they are treated as hosted (never refreshed). Operators who want them to resume refreshing should clear the old storage dir or move cached content out; new proxied content caches correctly into.pnpr-cache. The separation is forward-looking.Tests
New tests:
hosted_packument_is_never_overwritten_by_upstream— with a proxy upstream, a divergent upstream packument and a zero TTL, the hosted copy is still served and the upstream is never contacted (the "published versions can't be masked or lost" invariant).hosted_tarball_is_preferred_over_a_cached_copy—open_tarballserves the hosted copy when both stores hold the same filename.published_package_survives_wiping_the_proxy_cache— a hosted package survives a full.pnpr-cachewipe and is never written into it.unpublish_tarball_also_clears_the_proxied_copy— partial unpublish removes the proxied copy too.cache:key, relative-path resolution.Plus: updated the existing proxy-cache path assertions to the new cache root. Full pnpr suite green (229 tests),
cargo check --workspaceclean, clippy + rustfmt + typos clean.pnpr-only change — no pacquet port or changeset needed.
Written by an agent (Claude Code, claude-opus-4-8).
Summary by CodeRabbit
Release Notes
New Features
.pnpr-cachesubdirectory by default, keeping published packages durable.--cacheCLI flag to configure cache location independently from package storage.Tests