feat(pnpr): make resolver cache authorization-aware#12700
Conversation
…rfcs#11) Lay the foundation for making pnpr's resolution cache authorization-aware instead of disabling it whenever a request carries any upstream auth. This is Part 1 of the RFC (classify by fetch route): route classification, per-fetch footprint recording, caller-identity threading, and inline-URL-auth rejection. Descriptor-keyed caching of private resolutions, descriptor-scoped metadata mirrors, and gateway tarball-URL rewriting are Part 2. What changed: - pacquet-network: add an `UpstreamRouteHook` seam on `AuthHeaders`. When a hook is attached, every `for_url*` lookup delegates the decision to it, so a server can own auth selection at the single point every metadata/tarball fetch already flows through. The client-forwarded credentials are ignored (including inline `user:pass@`), and `None` (the CLI case) keeps lookup behavior identical. - pnpr route module: `RouteClass` (public / pnpr-hosted / proxied-alias / unknown), a `PrivateAccessDescriptor`, a `Footprint` recorder, and a pure `classify()` following the RFC precedence (public suppresses auth, then hosted, then an authorized upstream alias, otherwise fail-closed). The footprint digest is an HMAC-SHA256 (built over the workspace `sha2`, so no new dependency) verified against an RFC 4231 vector. - pnpr config: a route policy (built-in unscoped-npmjs-public, operator- disablable, plus declared public routes), pnpr-managed upstream credential aliases (route + resolved credential + access list + rotation generation), and the HMAC secret (reusing the YAML `secret:` key, else a per-process CSPRNG value). - pnpr resolver/server: thread the caller `Identity` into the resolve and verify endpoints, install the route hook on the request's `AuthHeaders`, reject inline URL credentials before any fetch, and gate the shared cache on the resolution's footprint being public rather than on the absence of client auth. An authenticated caller's fully-public install now populates and reuses the shared cache; a private resolution is never stored under the auth-excluded key. Client-forwarded upstream credentials are no longer sent to third-party registries (RFC-strict): private routes use a pnpr-managed alias or fail closed. No existing tests relied on the old forwarding behavior.
…ayload With server-side route classification in place, the pnpr resolve/verify endpoints no longer use client-forwarded upstream credentials, so the clients should not send them at all. Remove `authHeaders` from the `/-/pnpr/v0/resolve` and `/-/pnpr/v0/verify-lockfile` request bodies on both clients: - pacquet pnpr-client: drop the `auth_headers` field from `ResolveOptions` and `VerifyLockfileOptions` and stop serializing `authHeaders`. The CLI install path no longer attaches `config.auth_headers.to_by_scope()`; it still sends the `authorization` header that identifies the caller to pnpr. - TypeScript `@pnpm/pnpr.client` + `@pnpm/installing.deps-installer`: drop the `authHeaders` option and stop building the forwarded credential map; keep the pnpr identity `Authorization` header. The server stays tolerant of the field (`#[serde(default)]`) so an older client still interoperates; its forwarded creds are simply ignored. Rework the pnpr-client integration tests to the new model: a wire-level test asserts the request carries the identity header but no `authHeaders` body, and the private-package and verify-lockfile cases now exercise a pnpr-managed upstream credential alias instead of a forwarded client credential. Export `RoutePolicy`, `PublicRoute`, and `UpstreamAlias` from the pnpr crate so tests (and embedders) can configure aliases.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughRemoves client-forwarded upstream registry credentials from pnpr resolve and verify requests. Adds server-side route classification, scope-aware metadata caching, uplink gateway endpoints, group-based access policies, per-uplink namespaced storage, and route-aware resolution caching. Changespnpr server-side upstream auth and route-scoped caching
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related issues
Possibly related PRs
Suggested labels
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Micro-Benchmark ResultsLinux |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #12700 +/- ##
==========================================
- Coverage 85.47% 85.21% -0.27%
==========================================
Files 393 397 +4
Lines 59808 61268 +1460
==========================================
+ Hits 51120 52208 +1088
- Misses 8688 9060 +372 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The authorization-aware resolution cache classifies every metadata and tarball fetch through the route hook installed on `AuthHeaders`, so a private resolve is keyed by the access descriptor that produced it. The hook only fires at the auth-selection point (`for_url_with_package`), which the metadata fast paths in `pick_package` bypass: an in-memory cache hit, an offline/preferOffline disk read, a version-spec exact match, and the publishedBy mtime shortcut all answer a pick straight from cache without ever selecting auth. The on-disk metadata mirror is shared across pnpr requests, so a lockfile-seeded private resolve could be served pinned-version metadata from the shared mirror with no route recorded — leaving the footprint incomplete and the resolution mis-classified as public, which would let one caller's private resolution be served to another. Add `AuthHeaders::record_route`, which drives the route hook (classify + record) without sending a request, and call it up front in `pick_package` so every layer — cache hits, all disk fast paths, and the network fetch — contributes to the footprint regardless of which one serves the metadata. It is a no-op when no hook is installed (the CLI) and idempotent on the hook, so the network path re-recording the same route is harmless. Closes the open metadata-fetch and fast-path items under section 4 of #12699.
Integrated-Benchmark Report (Linux)Commit: Each scenario reports direct installs and pnpr installs. Bencher consumes pacquet@HEAD and pnpr@HEAD. Scenario: Isolated linker: fresh restore, cold cache + cold store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 4.41857844234,
"stddev": 0.14531705689570895,
"median": 4.37661511674,
"user": 3.6805848599999997,
"system": 3.4490136199999997,
"min": 4.26731180924,
"max": 4.75992036624,
"times": [
4.75992036624,
4.3138183122400005,
4.42033426124,
4.5446336272400005,
4.34533323424,
4.26731180924,
4.3100613042400004,
4.47114127524,
4.39011178224,
4.36311845124
]
},
{
"command": "pacquet@main",
"mean": 4.41746976274,
"stddev": 0.09927051462847562,
"median": 4.406707984740001,
"user": 3.6652658600000003,
"system": 3.424288219999999,
"min": 4.30592750124,
"max": 4.66947376524,
"times": [
4.394132502240001,
4.35497566824,
4.46412010524,
4.411523675240001,
4.66947376524,
4.30592750124,
4.34123917524,
4.40260407724,
4.41988926524,
4.41081189224
]
},
{
"command": "pnpr@HEAD",
"mean": 3.02263819784,
"stddev": 0.1651929192599902,
"median": 3.0348120522400004,
"user": 2.7686182599999998,
"system": 3.0347249199999995,
"min": 2.80524382924,
"max": 3.2375882982400004,
"times": [
2.8456632562400004,
3.09384744324,
2.9757766612400003,
3.1775380552400003,
2.85813845524,
3.1961661132400003,
3.1453757112400003,
3.2375882982400004,
2.8910441552400004,
2.80524382924
]
},
{
"command": "pnpr@main",
"mean": 2.95611285734,
"stddev": 0.1245447453823802,
"median": 2.94032633274,
"user": 2.7448232599999995,
"system": 3.0485421199999996,
"min": 2.8237049792400004,
"max": 3.18686498024,
"times": [
2.84176305224,
2.83479818724,
3.13931023024,
2.9909388052400003,
2.95700406024,
3.18686498024,
2.9236486052400004,
2.9762429172400005,
2.8237049792400004,
2.88685275624
]
}
]
}Scenario: Isolated linker: fresh restore, hot cache + hot store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 0.6485145474000001,
"stddev": 0.011149034494470631,
"median": 0.6444516565,
"user": 0.3857186,
"system": 1.3373159000000001,
"min": 0.6356376420000001,
"max": 0.667193173,
"times": [
0.640873685,
0.641319016,
0.667193173,
0.638309621,
0.664192486,
0.643161483,
0.658789117,
0.649927421,
0.6356376420000001,
0.64574183
]
},
{
"command": "pacquet@main",
"mean": 0.668572266,
"stddev": 0.04826314623906114,
"median": 0.6520007245,
"user": 0.3879493,
"system": 1.3286980000000002,
"min": 0.631285273,
"max": 0.8002192730000001,
"times": [
0.684076245,
0.651305178,
0.6428500030000001,
0.631285273,
0.663502522,
0.652224593,
0.6595696990000001,
0.648913018,
0.8002192730000001,
0.651776856
]
},
{
"command": "pnpr@HEAD",
"mean": 0.6830531302,
"stddev": 0.008977738356496155,
"median": 0.6831290875,
"user": 0.3884786,
"system": 1.3523863999999999,
"min": 0.669613679,
"max": 0.697645515,
"times": [
0.692195754,
0.683969958,
0.673051287,
0.682288217,
0.689125644,
0.669613679,
0.68884945,
0.677393739,
0.676398059,
0.697645515
]
},
{
"command": "pnpr@main",
"mean": 0.6998371741,
"stddev": 0.011440705615603595,
"median": 0.6992799265,
"user": 0.39498160000000004,
"system": 1.3619539,
"min": 0.685469734,
"max": 0.715884889,
"times": [
0.715399903,
0.687298498,
0.702939175,
0.715884889,
0.707016818,
0.706752887,
0.685469734,
0.694961928,
0.695620678,
0.687027231
]
}
]
}Scenario: Isolated linker: fresh install, cold cache + cold store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 4.7767115134600004,
"stddev": 0.041869188330370684,
"median": 4.79506385016,
"user": 3.8630231999999993,
"system": 3.4545200800000004,
"min": 4.71622639816,
"max": 4.838667385160001,
"times": [
4.838667385160001,
4.7986142461600005,
4.79151345416,
4.80708057916,
4.71622639816,
4.75810153116,
4.80241665516,
4.72923893716,
4.723886843160001,
4.80136910516
]
},
{
"command": "pacquet@main",
"mean": 4.706858843559999,
"stddev": 0.033006801694330026,
"median": 4.70422529066,
"user": 3.7768653999999997,
"system": 3.4348886800000002,
"min": 4.65746299016,
"max": 4.75067059916,
"times": [
4.7496242661600006,
4.68995562316,
4.69059851116,
4.75067059916,
4.723938284160001,
4.65746299016,
4.66264477516,
4.70372026216,
4.70473031916,
4.73524280516
]
},
{
"command": "pnpr@HEAD",
"mean": 2.95092474246,
"stddev": 0.17676286694593374,
"median": 2.8985990456599997,
"user": 2.5944671999999995,
"system": 2.90965968,
"min": 2.75738507716,
"max": 3.1725259441599998,
"times": [
2.82235905816,
2.8501816991599997,
2.9470163921599997,
3.14440508016,
2.7780680951599996,
3.1725259441599998,
3.1345866181599997,
2.75738507716,
2.76846411816,
3.13425534216
]
},
{
"command": "pnpr@main",
"mean": 3.1202824971599994,
"stddev": 0.06898614612577031,
"median": 3.1121033856599998,
"user": 2.7839628000000003,
"system": 3.17815108,
"min": 2.99664290216,
"max": 3.2476511371599996,
"times": [
3.08670018816,
3.15325313116,
2.99664290216,
3.08820986816,
3.2476511371599996,
3.18739851516,
3.14193585716,
3.08076636116,
3.0842701081599997,
3.1359969031599997
]
}
]
}Scenario: Isolated linker: fresh install, hot cache + hot store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 1.32676855936,
"stddev": 0.013975053965299316,
"median": 1.3283815594600001,
"user": 1.29660028,
"system": 1.6963368000000003,
"min": 1.3053670504600001,
"max": 1.3454051124600002,
"times": [
1.31282020846,
1.3053670504600001,
1.3272551644600001,
1.3295079544600001,
1.3151513144600002,
1.3154677354600002,
1.3454051124600002,
1.3441571174600002,
1.3377304674600001,
1.33482346846
]
},
{
"command": "pacquet@main",
"mean": 1.3754803588600004,
"stddev": 0.06148369265066731,
"median": 1.35976702396,
"user": 1.32603998,
"system": 1.7364448,
"min": 1.3249037594600002,
"max": 1.5415447994600002,
"times": [
1.3744575254600002,
1.3801198304600002,
1.5415447994600002,
1.3352302144600001,
1.3843793504600002,
1.36608743746,
1.3249037594600002,
1.34574969446,
1.34888436646,
1.35344661046
]
},
{
"command": "pnpr@HEAD",
"mean": 0.6790738891599999,
"stddev": 0.03653169404415321,
"median": 0.66848906596,
"user": 0.33351398,
"system": 1.2998717,
"min": 0.6523031704600001,
"max": 0.77809138046,
"times": [
0.65972403846,
0.68661912546,
0.66719673246,
0.66978139946,
0.6793151294600001,
0.77809138046,
0.68019918946,
0.65732402246,
0.66018470346,
0.6523031704600001
]
},
{
"command": "pnpr@main",
"mean": 1.4020825657600002,
"stddev": 0.04137962416278824,
"median": 1.3861387899600002,
"user": 0.5700085799999999,
"system": 1.5539192,
"min": 1.37148932646,
"max": 1.50609922046,
"times": [
1.43726892146,
1.39739423046,
1.3836043904600002,
1.37148932646,
1.37853885846,
1.50609922046,
1.4045147044600002,
1.3795344734600001,
1.3886731894600002,
1.37370834246
]
}
]
}Scenario: Isolated linker: fresh install, cold cache + hot store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 3.0388583318200006,
"stddev": 0.0378394607343635,
"median": 3.02836551362,
"user": 1.7757322399999995,
"system": 1.9898816200000002,
"min": 2.9993207776200004,
"max": 3.11612258362,
"times": [
3.02905467762,
3.0330759626200003,
3.06203597962,
3.01648705762,
3.01194331162,
2.9993207776200004,
3.0062960446200004,
3.11612258362,
3.0865705736200004,
3.02767634962
]
},
{
"command": "pacquet@main",
"mean": 3.00270789732,
"stddev": 0.04117069068733261,
"median": 3.0002204981200005,
"user": 1.7670572399999998,
"system": 1.9690548199999998,
"min": 2.9455334316200004,
"max": 3.10289861162,
"times": [
3.0017438496200004,
2.9860084596200003,
2.9744220206200005,
2.9986971466200005,
3.01129372262,
3.00617591562,
2.9455334316200004,
3.01956110362,
3.10289861162,
2.9807447116200003
]
},
{
"command": "pnpr@HEAD",
"mean": 0.66887437112,
"stddev": 0.012767954579822926,
"median": 0.6639889371200001,
"user": 0.33732804,
"system": 1.31170692,
"min": 0.65824062662,
"max": 0.7012016426200001,
"times": [
0.7012016426200001,
0.67141935862,
0.66488297162,
0.67037773262,
0.66309490262,
0.66179784762,
0.65824062662,
0.6622037216200001,
0.6589892296200001,
0.67653567762
]
},
{
"command": "pnpr@main",
"mean": 1.4596758199199997,
"stddev": 0.017380518653954843,
"median": 1.46207187912,
"user": 0.6006479400000001,
"system": 1.58766252,
"min": 1.42832388862,
"max": 1.48643868262,
"times": [
1.48643868262,
1.44331084462,
1.45792116662,
1.47497304062,
1.47159474762,
1.46100124362,
1.44246469162,
1.4631425146200001,
1.46758737862,
1.42832388862
]
}
]
} |
|
| Branch | pr/12700 |
| Testbed | pacquet |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| isolated-linker.fresh-install.cold-cache.cold-store | 📈 view plot 🚷 view threshold | 4,776.71 ms(+0.59%)Baseline: 4,748.84 ms | 5,698.61 ms (83.82%) |
| isolated-linker.fresh-install.cold-cache.hot-store | 📈 view plot 🚷 view threshold | 3,038.86 ms(-0.15%)Baseline: 3,043.48 ms | 3,652.18 ms (83.21%) |
| isolated-linker.fresh-install.hot-cache.hot-store | 📈 view plot 🚷 view threshold | 1,326.77 ms(-1.97%)Baseline: 1,353.37 ms | 1,624.05 ms (81.70%) |
| isolated-linker.fresh-restore.cold-cache.cold-store | 📈 view plot 🚷 view threshold | 4,418.58 ms(-8.82%)Baseline: 4,846.12 ms | 5,815.35 ms (75.98%) |
| isolated-linker.fresh-restore.hot-cache.hot-store | 📈 view plot 🚷 view threshold | 648.51 ms(-0.14%)Baseline: 649.42 ms | 779.31 ms (83.22%) |
|
| Branch | pr/12700 |
| Testbed | pnpr |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Latency | milliseconds (ms) |
|---|---|---|
| isolated-linker.fresh-install.cold-cache.cold-store | 📈 view plot | 2,950.92 ms |
| isolated-linker.fresh-install.cold-cache.hot-store | 📈 view plot | 668.87 ms |
| isolated-linker.fresh-install.hot-cache.hot-store | 📈 view plot | 679.07 ms |
| isolated-linker.fresh-restore.cold-cache.cold-store | 📈 view plot | 3,022.64 ms |
| isolated-linker.fresh-restore.hot-cache.hot-store | 📈 view plot | 683.05 ms |
Implements the "Private metadata cache (lower mirror)" work item from #12699: the npm resolver's metadata mirror is now namespaced per fetch route so a pnpr server resolving on behalf of many callers never lets one caller's private packument land in (or be read from) the global mirror every other caller shares. A new `MetadataCacheScope` (`pacquet-network`) classifies each `(registry, package)` fetch as `Public`, `Private { descriptor_id }`, or `Bypass`. The pnpm CLI installs no route hook, so every fetch is `Public` and the global mirror behaves exactly as before. Threaded through the npm resolver: - `mirror::scoped_meta_dir` relocates a private route's mirror to `v11/metadata-private/<descriptor-id>/<suffix>` and returns `None` for a `Bypass` route (no shared mirror at all). - `pick_package` computes the scope once and applies it to the mirror path, the in-memory cache key, and the fetch-lock key; a `Bypass` route stays out of the shared in-memory cache. - `fetch_full_metadata_cached` self-scopes its mirror path, covering conditional headers, 304 reads, and write-back. - The verifier's `read_local_meta_time` reads the scoped mirror, closing the private-metadata oracle (its other fetches self-scope via the cached fetcher). - A private/bypass route fails closed on `401`/`403`/`404` (`FetchMetadataError::is_access_denied`): it never falls back to a cached mirror on a denial, only on transport failures and only within its own namespace. Public routes keep their existing fallback. On the pnpr side, `RouteHook` carries the server secret and maps each `RouteClass` to a scope, HMACing the private access descriptor's key input into the namespace id so the mirror path is not correlatable offline; `Unknown` routes map to `Bypass`.
First step of aligning the resolver with the auth-aware resolution cache RFC (pnpm/rfcs#11): an uplink can now carry an `access:` policy and a `generation`, folding the separate `upstreamAliases` concept into the existing `uplinks` config. An uplink that declares `access:` is eligible to back a proxied private route and be exposed at its `/~<name>/` registry endpoint; an uplink without `access:` stays registry-proxy only. No consumer reads the new fields yet; route classification and endpoint serving are wired up in follow-up commits.
Route classification now sources proxied-route credentials from `uplinks:` entries that declare an `access:` policy, in addition to the legacy `upstreamAliases` block. Uplink-sourced credentials are matched by registry origin (no package glob) and carry the uplink's generation; a plain proxy uplink without `access:` is never offered as a private-route credential. Part of aligning the resolver with the auth-aware resolution cache RFC (pnpm/rfcs#11).
A fetch to pnpr's own `/~<uplink>/` registry endpoint is now classified as a proxied route through that uplink, using the uplink's current generation (the URL carries none). Since a package name can never begin with `~`, such a URL is unambiguously an uplink endpoint, not a hosted package: an unauthorized caller or an unknown uplink fails closed rather than falling through to the hosted-package policy. This is what lets a `/resolve` request whose scope is configured at a `/~<uplink>/` endpoint classify and credential the route through the backing uplink instead of treating its own endpoint as a third-party upstream. Part of aligning the resolver with the auth-aware resolution cache RFC (pnpm/rfcs#11).
Expose each access-bearing uplink as a read-only registry endpoint at `/~<uplink>/`: packument (scoped and unscoped) and tarball reads route through the named uplink instead of the `packages.proxy` chain, gated by the uplink's own `access:` policy. Served packuments have their `dist.tarball` rewritten back onto the same endpoint, so a client that points a scope at `https://<pnpr>/~<uplink>/` gets canonical (integrity-only) lockfile entries. The endpoint is fetch-through: it reads the uplink's packument fresh for the version integrity and streams the tarball through a temp file verified against it, writing nothing to the shared proxy mirror. A private uplink's packuments and tarballs therefore never persist where the public path or another uplink could read them — closing the metadata/tarball leak that a shared, package-keyed mirror would open. Integration tests cover the endpoint rewrite, fail-closed access gating, and the no-persistence property. Part of aligning the resolver with the auth-aware resolution cache RFC (pnpm/rfcs#11).
Replace the opaque per-tarball gateway scheme with the uplink registry endpoints. The resolver now rewrites a proxied route's tarball to its `/~<uplink>/<package>/-/<file>` endpoint URL (canonical for a client whose scope is configured there, so the lockfile entry collapses to integrity-only), keeps a public/unknown route's upstream URL untouched, and reverses endpoint URLs back to upstream when verifying an input lockfile. Deleted: - the `/-/pnpr/v0/tarballs/alias|unknown/...` routes and their handlers; - `TarballGatewayRoutes` (the in-memory HMAC key -> URL map) and the unknown-route gateway machinery; - `GatewayAlias`/`gateway_alias` (replaced by `RouteContext::uplink_registry`); - the `upstreamAliases` config block and `UpstreamAlias` type — proxied-route credentials now come solely from access-bearing `uplinks:` entries, matched by registry origin. Tests across the route, resolver, config, and server suites are migrated from `upstreamAliases` to access-bearing uplinks and from gateway URLs to endpoint URLs; 518 pnpr tests pass. Part of aligning the resolver with the auth-aware resolution cache RFC (pnpm/rfcs#11).
Update the sample `config.yaml` to show an access-bearing uplink (exposed at `/~<name>/`) in place of the removed `upstreamAliases:` block, and refresh the resolver/pnpr-client doc comments that still referred to the read-only tarball gateway to describe the `/~<uplink>/` registry endpoints.
Port the pnpr-client integration tests from `upstreamAliases` to access-bearing uplinks: the private-route fixtures register a `uplinks:` entry (exposed at `/~test-registry/`) instead of an `UpstreamAlias`, the unknown-route test now asserts the resolution stays integrity-only (no gateway URL is minted), and the verify-lockfile test shares a `public_url` across its resolve and verify instances so the `/~<uplink>/` endpoint URL reverses back to upstream the way a single-pnpr deployment does.
A registry-resolved package classified Public emitted its upstream dist.tarball verbatim, so a signed/tokenized URL (?X-Amz-Signature=…, ?token=…) from a presigned-CDN registry would leak the upstream token to callers and into the shared cache. route_registry_url's Public arm now runs the URL through sanitize_registry_tarball_url, which drops inline userinfo and any query/fragment. A genuinely public tarball is fetched by its bare path (verified by SRI), so the sanitized URL still works; a tarball that truly needs a token is not a public route and must be configured as an uplink (whose tarballs route through /~<uplink>/, keeping the token server-side). A client-supplied direct tarball dependency keeps its own query (the caller's intent); only the untrusted upstream dist.tarball is fully sanitized.
|
Code review by qodo was updated up to the latest commit dd030ae |
1 similar comment
|
Code review by qodo was updated up to the latest commit dd030ae |
…lic toggle The npmjsPublic boolean was doing two things the public-route machinery already does: allowlisting registry.npmjs.org and classifying it Public. With the default npmjs uplink it was a no-op, and its only remaining effects (allowlisting npmjs when no uplink is configured, and "public wins over an npmjs uplink credential") are exactly what a built-in public route provides. RouteContext::from_config now prepends a host-level built-in route for registry.npmjs.org to public_routes, so npmjs is allowlisted and classified Public through the same path as operator-declared routes — and the per-call npmjs special cases in allows_registry/is_public_route, the RoutePolicy field, and the npmjsPublic config key are gone. The one behavior dropped is the "conservative deployment" mode that disabled the built-in; nothing is configurable to refuse a direct npmjs resolve anymore.
|
Code review by qodo was updated up to the latest commit 45f5ad3 |
|
Code review by qodo was updated up to the latest commit 45f5ad3 |
BlockedRedirect's Display printed the full redirect target URL, so blocking an off-allowlist redirect to a presigned URL (e.g. S3 with ?X-Amz-Signature=…) leaked the signature/token through the propagated error string, which pnpr surfaces to clients. It now names only scheme://host[:port] — never the path, query, fragment, or userinfo.
|
Code review by qodo was updated up to the latest commit ab29c62 |
1 similar comment
|
Code review by qodo was updated up to the latest commit ab29c62 |
…op generation The private cache (resolution cache, metadata mirror, per-uplink packument/ tarball namespace) was epoch-stamped by a manual `generation: u64` counter the operator bumped on credential rotation. That had a footgun: rotate the token but forget to bump it, and the cache keeps serving content fetched with the retired credential. A `PrivateAccessDescriptor::Alias` now carries `credential_digest` — a SHA-256 of the uplink's `Authorization`, computed once at config load — instead of `generation`. Rotating the credential changes the digest automatically, so it re-keys every private cache the uplink owns with no manual step. The raw token never reaches disk: the digest is one-way and then HMAC-keyed with the server secret. `allows_descriptor` now requires the caller's selected alias to hash to the same credential, so a cached resolution from a retired credential fails closed. The `generation` config knob and field are removed.
|
Code review by qodo was updated up to the latest commit 63f6233 |
|
Code review by qodo was updated up to the latest commit 63f6233 |
When a public registry is also configured as an uplink (a caching mirror), announce resolved tarballs as pnpr's own `<pnpr>/<pkg>/-/<file>` endpoint instead of the upstream registry/CDN URL, so clients fetch tarballs from pnpr's warm, near cache rather than across the slow client↔registry link — the install-accelerator win for a cold client store. This reverses the #12700 choice to always emit upstream URLs for public routes; the upstream presigned token still never reaches the client (it stays server-side, where pnpr fetches the tarball). A registry with no uplink keeps its upstream URL.
…ve path (#12709) #12570 made every tarball serve — warm cache hits included — both re-hash the cached bytes against dist.integrity and load+parse the packument to bind the request to a version. The benchmark mock is pnpr and a cold-store install pulls ~1300 tarballs through it, so both costs are paid per tarball: together the cold-store regression from ~2.15s to ~3.15s on the Bencher pnpr testbed (#12700 recovered only the resolution-cache part). Both are redundant on a cache hit. A tarball only enters the proxy cache via download_verified_to_cache, which resolves the version and verifies the bytes against dist.integrity as they are written, so the cache only ever holds correct bytes for a (name, filename); the install client re-verifies whatever it receives anyway. Serve cached bytes directly and defer the packument load to the cache-miss download path, which still verifies freshly-fetched bytes before caching them. The OSV screen on the filename version still runs first. Write-time verification is preserved, so GHSA-5f9g-98vq-2jxw stays mitigated — the bytes that enter the cache are still verified, which the pre-regression serve path did not even do. Drops the now-dead per-serve integrity sidecar and helpers.
…e resolver (#832) Reflects the pnpr redesign in pnpm/pnpm#12700 (authorization-aware resolver cache, default-deny fetch allowlist, no forwarded upstream credentials) and pnpm/pnpm#12747 (registry mounts as the only routing model): mounts:/defaultTarget: replace uplinks: and packages: proxy:, packages: is ACL-only, and every mount is served at /~<mount>/.
Summary
Related to #12699 and pnpm/rfcs#11.
This branch makes the pnpr resolver cache authorization-aware and routes private dependencies through per-uplink registry endpoints rather than opaque tarball gateways. It implements the route-classification foundation, a default-deny fetch allowlist that closes the resolver's SSRF surface, the auth-aware resolution cache (footprint + descriptor-keyed candidates), the descriptor-scoped private metadata mirror, the
/~<uplink>/endpoint serving with a per-uplink cache, and integrity-only lockfile routing. The optional refinements in issue section 9 (lightweight revocation validation, operator metrics) remain follow-up work.Route-classification foundation
pacquet-networkexposes anUpstreamRouteHookseam so pnpr owns upstream auth selection at the single fetch/auth-selection point, while the CLI path stays unchanged (it installs no hook).Identity, records a per-resolve footprint, rejects inlineuser:pass@URL credentials before fetch, and never forwards the client's own upstream credentials.Fetch allowlist (default-deny) — closes SSRF, supersedes #12675
/~<uplink>/endpoints), and pnpr's own origin. A clientregistry/namedRegistrieswhose origin matches none of these is rejected at the request boundary, before any fetch, by both/resolveand/verify-lockfile.http(s)tarball spec orgit/git+…URL in a dependency, anoverridesleaf, or an input-lockfileresolution.tarballis rejected unless its origin is allowlisted (agit+prefix is stripped before the check). Semver ranges andnpm:/workspace:/file:aliases never hit the network and are ignored. A direct URL dependency therefore requires the operator to allowlist its origin, the same as any registry.registryURL orpackageglob drops the whole rule rather than collapsing to a match-all, so a misconfiguration can only narrow matching, never widen a scoped public route into one that classifies a private registry as public.registry.npmjs.org— scoped or unscoped — is allowlisted and public with no configuration (a private scoped package 404s on the anonymous fetch and is never resolved this way), ahead of any uplink credential for the same origin (public wins). There is nonpmjsPublic/npmjsUnscopedPublictoggle; npmjs flows through the same public-route machinery as operator-declared routes...segment: a./..in the nerf-darted key is rejected, so//host/base/../admincan't pass a//host/base/path-scoped allow.RouteClass::Unknown, the non-shareable footprint tier, and theMetadataCacheScope::Bypassmetadata-cache tier.Uplinks are the private-route credential (folds in
upstreamAliases)uplinks:entry that declares anaccess:policy becomes a pnpr-managed private-route credential. The separateupstreamAliasesblock (andUpstreamAliastype) is removed — proxied routes now source their credential from access-bearing uplinks, matched by registry origin (no per-credential package glob; an uplink credential covers every package on its registry). One upstream token per uplink is fanned out to a whole team by pnpr authorization.AccessListmodel, and the top-levelgroups:team map.uplink + credential-digestin the footprint — the digest is a SHA-256 of the uplink'sAuthorization, computed once at config load. Rotating the upstream credential changes the digest automatically, moving new private cache entries to a fresh namespace (old entries age out by TTL) with no manual epoch to bump.http://fetch never receives anhttps://uplink's token (nerf-darting strips the scheme), so a server-owned credential can't be sent in cleartext.Tarball routing via
/~<uplink>/registry endpoints (issue section 7)https://<pnpr>/~<uplink>/: packument and tarball reads route through that uplink (gated by its access policy), withdist.tarballrewritten back onto the same endpoint. The~prefix keeps endpoints out of the package-name path namespace, since a package name cannot begin with~./~<uplink>/endpoint URL; public routes keep their upstream URL (fetched directly from the registry/CDN, never through pnpr); pnpr-hosted packages use pnpr-hosted URLs. A registry-resolved package is classified by its registry route, not itsdist.tarballhost — so a split-domain private registry (packument and tarball on different hosts) still routes through/~<uplink>/instead of leaking the raw upstream tarball URL in a streamed frame.user:pass@userinfo is stripped everywhere, and a registry package's upstreamdist.tarballalso has its query/fragment dropped, so a presigned/tokenized upstream URL (?X-Amz-Signature=…) never reaches a client or the shared cache..npmrc, not the lockfile. A project then resolves to the same lockfile whether it goes through/resolveor a direct/proxied install, and public packages stay on the CDN purely because the default registry points there. This replaces the previous "rewrite every non-public tarball to a pnpr URL, with a special case for public" behavior.verification_lockfilereverses/~<uplink>/URLs back to upstream so an input lockfile verifies against the real registry, and route classification recognizes pnpr's own/~<uplink>/URLs so/resolveresolves a scope already pointed at its endpoint through the backing uplink./-/pnpr/v0/tarballs/alias|unknown/…routes and handlers), its in-memory HMACkey → URLmap, and theGatewayAliasmachinery.Per-uplink private cache
~uplinks/<digest>under the proxy cache root, where<digest>is an HMAC of(uplink, credential)keyed with the server secret): packuments are served within the freshness window and tarballs are verified once and served from cache thereafter, so a private install caches like a public one rather than re-fetching the packument on every tarball request.Resolution cache: footprint + descriptor-keyed candidates
hash_lockfile()digest rather than embedding serialized lockfile data.Footprint, last-used time, and descriptor HMAC. Public candidates match every caller; private candidates match only when the caller still satisfies every stored uplink-or-hosted-policy descriptor gate./~<uplink>/endpoint (integrity-only) URLs, never raw private upstream URLs.Footprint completeness on metadata fast paths (issue section 4)
AuthHeaders::for_url_with_package), which the npm resolver's metadata fast paths bypassed: an in-memory cache hit, an offline/preferOffline disk read, a version-spec exact match, and the publishedBy mtime shortcut each answer a pick straight from cache without selecting auth.AuthHeaders::record_routedrives the route hook (classify + record) without issuing a request;pick_packagecalls it up front so every layer — cache hits, all disk fast paths, and the network fetch — contributes to the footprint regardless of which one serves the metadata. It is a no-op when no hook is installed (the CLI) and idempotent on the hook.Private metadata cache / lower mirror (issue section 8)
MetadataCacheScopeinpacquet-network(Public/Private { descriptor_id }), with read-onlyUpstreamRouteHook::metadata_scopeandAuthHeaders::metadata_scope. The CLI installs no hook, so every fetch isPublicand the global mirror is unchanged.mirror::scoped_meta_dirrelocates a private route tov11/metadata-private/<descriptor-id>/<suffix>, applied per(registry, package)fetch rather than by swapping the wholecache_dir.pick_package(mirror path, in-memory cache key, fetch-lock key),fetch_full_metadata_cached(conditional headers, 304 reads, write-back), and the verifier'sread_local_meta_time, closing the private-metadata oracle.FetchMetadataError::is_access_denied(401/403/404). A private route never falls back to a cached mirror on a denial; only transport failures fall back, and only within the same namespace. Public routes keep their existing fallback.Squash Commit Body
Checklist
pacquet/port, or the description notes what still needs porting.pnpm changeset) if this PR changes any publishedpackage.
Written by agents (Codex, GPT-5; Claude Code, claude-opus-4-8).
Summary by CodeRabbit