perf(pnpr): shrink the abbreviated packuments served to clients#12163
Merged
Conversation
When pnpr proxies a registry, trim the `application/vnd.npm.install-v1+json` packument down to only the fields the pnpm and pacquet resolvers actually read, so clients download, parse, and cache less metadata. Dropped (never read during resolution): - top-level `readme`, `readmeFilename`, `_id`, `_rev` - per-version `funding`, `devDependencies`, `acceptDependencies`, `_hasShrinkwrap` - per-version `dist.shasum` when `dist.integrity` is present (the resolvers prefer SRI integrity and only fall back to the legacy sha1 shasum when integrity is absent, so shipping both is redundant) `shasum` is kept when `integrity` is absent (pre-2017 publishes) so pnpm's getIntegrity fallback still has a hash. Also forwards per-version `libc` alongside `os`/`cpu`, which was previously stripped — pnpm reads it for optional-dependency platform filtering (#9950), so omitting it produced wrong installs through pnpr.
Satisfies the perfectionist::bare-issue-reference dylint.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📜 Recent review details🧰 Additional context used📓 Path-based instructions (1)pnpr/**/pnpr/**/*.rs📄 CodeRabbit inference engine (pnpr/AGENTS.md)
Files:
🧠 Learnings (24)📓 Common learnings📚 Learning: 2026-05-29T18:03:15.354ZApplied to files:
📚 Learning: 2026-05-20T21:18:56.391ZApplied to files:
📚 Learning: 2026-05-29T18:03:15.354ZApplied to files:
📚 Learning: 2026-05-28T16:19:30.483ZApplied to files:
📚 Learning: 2026-05-20T01:52:55.764ZApplied to files:
📚 Learning: 2026-05-29T18:03:15.354ZApplied to files:
📚 Learning: 2026-05-29T18:03:15.354ZApplied to files:
📚 Learning: 2026-05-29T18:03:24.760ZApplied to files:
📚 Learning: 2026-06-01T08:59:42.161ZApplied to files:
📚 Learning: 2026-05-18T20:35:22.917ZApplied to files:
📚 Learning: 2026-05-20T13:36:20.653ZApplied to files:
📚 Learning: 2026-05-23T16:55:36.507ZApplied to files:
📚 Learning: 2026-06-02T13:18:26.437ZApplied to files:
📚 Learning: 2026-05-20T23:24:24.022ZApplied to files:
📚 Learning: 2026-05-25T12:36:42.202ZApplied to files:
📚 Learning: 2026-05-20T23:08:06.093ZApplied to files:
📚 Learning: 2026-05-24T21:11:04.272ZApplied to files:
📚 Learning: 2026-05-29T18:03:15.354ZApplied to files:
📚 Learning: 2026-05-23T17:30:06.849ZApplied to files:
📚 Learning: 2026-05-24T16:07:54.784ZApplied to files:
📚 Learning: 2026-05-15T11:37:17.491ZApplied to files:
📚 Learning: 2026-05-29T18:03:15.354ZApplied to files:
📚 Learning: 2026-05-24T21:11:04.272ZApplied to files:
🔇 Additional comments (6)
📝 WalkthroughWalkthroughThe PR narrows the abbreviated "install-v1" packument shape by reducing preserved top-level and per-version fields, adds a helper to prune ChangesPackument Abbreviation and Dist Trimming
Possibly Related PRs
🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #12163 +/- ##
==========================================
+ Coverage 87.56% 87.58% +0.01%
==========================================
Files 268 268
Lines 30761 30775 +14
==========================================
+ Hits 26935 26953 +18
+ Misses 3826 3822 -4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…ments `npm-signature` is npm's deprecated PGP detached signature; npm stopped populating it years ago and neither pnpm nor pacquet reads it. The ECDSA `dist.signatures` is kept — it binds `name@version:integrity` to the upstream registry key and survives pnpr's tarball-URL rewriting, so it remains available for a potential client-side install-time check.
Review Summary by QodoShrink abbreviated packuments by removing unused resolver fields
WalkthroughsDescription• Trim abbreviated packuments to resolver-read fields only • Drop unused top-level fields: readme, readmeFilename, _id, _rev • Drop unused per-version fields: funding, devDependencies, acceptDependencies, _hasShrinkwrap • Remove redundant dist.shasum when dist.integrity present; drop dist.npm-signature • Add missing libc field for optional-dependency platform filtering Diagramflowchart LR
A["Full Packument"] -->|abbreviate_packument| B["Trimmed Abbreviated Form"]
B -->|drop unused fields| C["Smaller JSON"]
B -->|keep resolver-read fields| D["Functional Metadata"]
E["trim_dist_fields"] -->|remove npm-signature| F["Legacy PGP gone"]
E -->|remove shasum if integrity present| G["No redundant hashes"]
E -->|keep dist.signatures| H["ECDSA signatures preserved"]
File Changes1. pnpr/crates/pnpr/src/upstream.rs
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes pnpr’s registry-proxy behavior by further shrinking abbreviated packuments (Accept: application/vnd.npm.install-v1+json) to only the metadata fields actually used by pnpm/pacquet during dependency resolution, reducing network transfer and client-side JSON parse/cache overhead while keeping full-metadata responses unchanged.
Changes:
- Drop unused top-level and per-version fields from abbreviated packuments and synthesize
modifiedfromtime.modified. - Preserve resolver-relevant platform fields (notably
libc) for correct optional-dependency filtering. - Trim redundant
distsubfields in abbreviated responses (e.g. removenpm-signature, and conditionally removeshasumwhenintegrityis present), with new unit/integration tests.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| pnpr/crates/pnpr/src/upstream.rs | Implements the abbreviated-packument field whitelist, adds libc, and trims dist subfields to reduce response size. |
| pnpr/crates/pnpr/src/upstream/tests.rs | Adds unit tests asserting dropped/kept fields and shasum retention when integrity is absent. |
| pnpr/crates/pnpr/tests/registry_mock.rs | Updates end-to-end assertions to ensure README-related fields are dropped in abbreviated responses. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…d packuments Neither is read during resolution or install. `fileCount` is read nowhere in pnpm or pacquet; `unpackedSize` is read only by `pnpm view`, which fetches the full metadata document (`fullMetadata: true`) that pnpr serves unstripped. On a large packument like react's the two sizes account for ~3.5% of the abbreviated payload. `dist.signatures` (the ECDSA registry signatures, ~18% of react's metadata) is kept: a future client-side install-time check on the pnpr path can verify against it.
Match pnpm's `getIntegrity` truthiness (`if (dist.integrity)`): an absent, empty, or non-string `integrity` must keep `shasum` so the sha1 fallback still works. The previous check only excluded null, so `integrity: ""` would wrongly strip the fallback hash.
Contributor
Integrated-Benchmark Report (Linux)Each scenario has pacquet rows (direct install) and pnpr rows (the same client through the pnpr install accelerator), so pnpr@HEAD vs pacquet@HEAD is the pnpr-vs-direct ratio. Cold-store scenarios wipe the client store between runs (warm server); hot-store scenarios keep it warm. The pacquet@HEAD rows feed the pacquet Bencher testbed; the pnpr@HEAD rows feed the pnpr testbed. Scenario: Isolated linker: fresh restore, cold cache + cold store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 2.02785176262,
"stddev": 0.08911990979935962,
"median": 2.0085909282200003,
"user": 2.6421887,
"system": 3.2200080199999994,
"min": 1.9278284042199998,
"max": 2.22600202622,
"times": [
2.22600202622,
1.9278284042199998,
1.95052519722,
2.01231524822,
1.98132578822,
2.02264172722,
2.1367264072200003,
2.02605353122,
1.99023268822,
2.00486660822
]
},
{
"command": "pacquet@main",
"mean": 2.00530014372,
"stddev": 0.07850624199254083,
"median": 1.9739370482199998,
"user": 2.6695900999999997,
"system": 3.2378142199999997,
"min": 1.9302020392199999,
"max": 2.14944397622,
"times": [
1.9302020392199999,
2.14944397622,
1.98578336022,
2.03369113822,
1.93147744222,
1.96209073622,
2.05267992522,
1.95735760922,
2.1123663932200003,
1.93790881722
]
},
{
"command": "pnpr@HEAD",
"mean": 1.9973875404199997,
"stddev": 0.0654365344914182,
"median": 1.9933188932199999,
"user": 2.6732291999999998,
"system": 3.2195297199999997,
"min": 1.91647492822,
"max": 2.14918201022,
"times": [
1.94597768622,
1.98024406522,
2.01214620622,
2.02464144022,
1.91647492822,
2.02597967722,
1.93259160422,
1.9922116432199999,
1.99442614322,
2.14918201022
]
},
{
"command": "pnpr@main",
"mean": 1.97164165322,
"stddev": 0.04609814527204395,
"median": 1.9566944017199999,
"user": 2.6613379,
"system": 3.2033963199999995,
"min": 1.92222467322,
"max": 2.07412839222,
"times": [
1.99685045622,
1.93988974322,
1.96476081422,
1.93993184722,
1.94587752222,
1.92222467322,
2.01936428022,
1.9497960002199999,
1.9635928032199998,
2.07412839222
]
}
]
}Scenario: Isolated linker: fresh restore, hot cache + hot store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 0.6582197401000001,
"stddev": 0.019701756983415964,
"median": 0.6551835519,
"user": 0.36495002,
"system": 1.3256836,
"min": 0.6381355924000001,
"max": 0.7101924194000001,
"times": [
0.7101924194000001,
0.6530940354000001,
0.6610878004,
0.6576040694,
0.6381355924000001,
0.6405183624,
0.6540048984000001,
0.6539060174000001,
0.6572920004,
0.6563622054
]
},
{
"command": "pacquet@main",
"mean": 0.6591196673,
"stddev": 0.010409308221694067,
"median": 0.6559924194,
"user": 0.35782561999999996,
"system": 1.3408412,
"min": 0.6475648744000001,
"max": 0.6772360604000001,
"times": [
0.6772360604000001,
0.6758455124,
0.6545982944000001,
0.6475648744000001,
0.6566872914,
0.6552975474,
0.6506521344,
0.6498155134000001,
0.6585369504,
0.6649624944000001
]
},
{
"command": "pnpr@HEAD",
"mean": 0.7017298776,
"stddev": 0.04335697607780875,
"median": 0.6885637294,
"user": 0.36296582000000005,
"system": 1.3346044999999997,
"min": 0.6628160974,
"max": 0.8113452694000001,
"times": [
0.8113452694000001,
0.7199880174000001,
0.7222638714,
0.6679473924,
0.6628160974,
0.6902668504,
0.6793066224000001,
0.6999517124000001,
0.6868606084000001,
0.6765523344000001
]
},
{
"command": "pnpr@main",
"mean": 0.735842348,
"stddev": 0.09161558889771305,
"median": 0.7104622414000001,
"user": 0.35928102,
"system": 1.3463068,
"min": 0.6552951474,
"max": 0.9702170954000001,
"times": [
0.7977422084000001,
0.7101865004000001,
0.6829657544000001,
0.7107379824000001,
0.7458161534000001,
0.6552951474,
0.6853880634,
0.7223562984,
0.6777182764,
0.9702170954000001
]
}
]
}Scenario: Isolated linker: fresh install, cold cache + cold store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 2.1294681658199996,
"stddev": 0.03436095636337518,
"median": 2.11910228442,
"user": 3.43033992,
"system": 2.9837288600000003,
"min": 2.08892597042,
"max": 2.18569493042,
"times": [
2.18569493042,
2.17157540942,
2.09176055042,
2.12301348342,
2.11519108542,
2.16715633942,
2.08892597042,
2.10728878942,
2.10866171742,
2.13541338242
]
},
{
"command": "pacquet@main",
"mean": 2.1182140830200002,
"stddev": 0.032748242604357276,
"median": 2.12321524742,
"user": 3.4343397199999997,
"system": 2.9349980600000003,
"min": 2.07062618542,
"max": 2.17674159342,
"times": [
2.17674159342,
2.13059643242,
2.08153119142,
2.0925615994199998,
2.09734439042,
2.13809485842,
2.13347413642,
2.14533638042,
2.11583406242,
2.07062618542
]
},
{
"command": "pnpr@HEAD",
"mean": 2.1167361031199996,
"stddev": 0.030530923510809115,
"median": 2.1276011434199997,
"user": 3.41606732,
"system": 2.95568886,
"min": 2.07366057242,
"max": 2.1648244874199998,
"times": [
2.13948813142,
2.07366057242,
2.12297728942,
2.08406618842,
2.13326046142,
2.08283970742,
2.13222499742,
2.09538512642,
2.13863406942,
2.1648244874199998
]
},
{
"command": "pnpr@main",
"mean": 2.12752536182,
"stddev": 0.021598616843611893,
"median": 2.12948828542,
"user": 3.4535176199999995,
"system": 2.9607284599999995,
"min": 2.09136178842,
"max": 2.15114027142,
"times": [
2.14996183442,
2.12364175942,
2.09425792542,
2.09136178842,
2.15114027142,
2.11803711042,
2.12949535342,
2.14991061442,
2.13796574342,
2.12948121742
]
}
]
}Scenario: Isolated linker: fresh install, hot cache + hot store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 1.26457336562,
"stddev": 0.01987202701182894,
"median": 1.25899676442,
"user": 1.3407974200000001,
"system": 1.7235582000000005,
"min": 1.24144986842,
"max": 1.30631639142,
"times": [
1.26893848642,
1.30631639142,
1.24371654142,
1.26078305542,
1.25721047342,
1.27887517342,
1.2540882954199999,
1.28177592842,
1.24144986842,
1.25257944242
]
},
{
"command": "pacquet@main",
"mean": 1.28017386812,
"stddev": 0.06229369044614951,
"median": 1.26410045092,
"user": 1.33852092,
"system": 1.7463478000000001,
"min": 1.24041513342,
"max": 1.45241937642,
"times": [
1.24512868142,
1.26871005842,
1.28232480542,
1.25949084342,
1.45241937642,
1.28188755742,
1.27063616442,
1.25281129642,
1.24791476442,
1.24041513342
]
},
{
"command": "pnpr@HEAD",
"mean": 1.2807651871199996,
"stddev": 0.03812883492638937,
"median": 1.27413463292,
"user": 1.3392362199999996,
"system": 1.7359016999999999,
"min": 1.24137168342,
"max": 1.37763571242,
"times": [
1.25271709242,
1.2845917894199999,
1.24137168342,
1.28405389142,
1.37763571242,
1.29304661442,
1.28922171142,
1.26269747342,
1.25810052842,
1.26421537442
]
},
{
"command": "pnpr@main",
"mean": 1.2934959095199998,
"stddev": 0.02807033283924353,
"median": 1.2861175569199998,
"user": 1.34582602,
"system": 1.7549021,
"min": 1.26549159342,
"max": 1.35795415242,
"times": [
1.27344190542,
1.26549159342,
1.30955987742,
1.29470800442,
1.35795415242,
1.30579693942,
1.26771384742,
1.3057135574199998,
1.27705210842,
1.27752710942
]
}
]
} |
Contributor
|
Bencher Report| Branch | pr/12163 |
| Testbed | pacquet |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| isolated-linker.fresh-install.cold-cache.cold-store | 📈 view plot 🚷 view threshold | 2,129.47 ms(-9.08%)Baseline: 2,342.23 ms | 2,810.68 ms (75.76%) |
| isolated-linker.fresh-install.hot-cache.hot-store | 📈 view plot 🚷 view threshold | 1,264.57 ms(-16.83%)Baseline: 1,520.53 ms | 1,824.64 ms (69.31%) |
| isolated-linker.fresh-restore.cold-cache.cold-store | 📈 view plot 🚷 view threshold | 2,027.85 ms(-1.08%)Baseline: 2,050.05 ms | 2,460.06 ms (82.43%) |
| isolated-linker.fresh-restore.hot-cache.hot-store | 📈 view plot 🚷 view threshold | 658.22 ms(+1.22%)Baseline: 650.28 ms | 780.34 ms (84.35%) |
Contributor
|
| Branch | pr/12163 |
| Testbed | pnpr |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Latency | milliseconds (ms) |
|---|---|---|
| isolated-linker.fresh-install.cold-cache.cold-store | 📈 view plot | 2,116.74 ms |
| isolated-linker.fresh-install.hot-cache.hot-store | 📈 view plot | 1,280.77 ms |
| isolated-linker.fresh-restore.cold-cache.cold-store | 📈 view plot | 1,997.39 ms |
| isolated-linker.fresh-restore.hot-cache.hot-store | 📈 view plot | 701.73 ms |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
When pnpr proxies a registry, trim the
application/vnd.npm.install-v1+jsonpackument down to only the fields the pnpm and pacquet resolvers actually read, so clients download, parse, and cache less metadata.Only the abbreviated path (
Accept: application/vnd.npm.install-v1+json) is affected; full-document clients are untouched.Dropped (never read during resolution)
readme,readmeFilename,_id,_rev—readmeis the dominant per-packument bloat (full README text) and npm's own abbreviated format never carried it.funding,devDependencies,acceptDependencies,_hasShrinkwrap. A dependency'sdevDependenciesare never installed, so the resolver has no use for them.dist["npm-signature"]— npm's deprecated PGP detached signature. npm stopped populating it years ago and nothing in pnpm or pacquet reads it.dist.fileCount— read nowhere in pnpm or pacquet.dist.unpackedSize— read only bypnpm view, which fetches the full metadata document (fullMetadata: true) that pnpr serves unstripped.dist.shasumwhendist.integrityis present. Both pnpm (getIntegrity) and pacquet prefer SRIintegrityand only fall back to the legacy sha1shasumwhenintegrityis absent, so shipping both is a redundant hash on every version.shasumis kept whenintegrityis absent (pre-2017 publishes) so thegetIntegrityfallback still has a hash.Deliberately kept
time(top-level publish timestamps). npm's own abbreviated form omits it, but pnpr retains it because pnpm/pacquet read it for theminimumReleaseAgecheck.dist.signatures(ECDSA registry signatures). It bindsname@version:integrityto the upstream registry's signing key and survives pnpr'sdist.tarballrewriting (the signature covers the triple, not the URL). Nothing verifies it at install time today —pnpm audit signaturesfetches its own full metadata — but keeping it leaves the door open to an optional client-side install-time registry-signature check, which is most valuable precisely on the pnpr path (an extra trust hop).dist.attestations— read by pacquet'strustPolicyverifier.Fixed along the way
Per-version
libcis now forwarded alongsideos/cpu— it was previously stripped. pnpm readslibcfor optional-dependency platform filtering (#9950), so omitting it produced wrong installs through pnpr.Impact (measured)
Compact JSON size of the abbreviated packument, before vs. after this PR, on real registry metadata:
Where the savings come from (react, % of the before size):
dist["npm-signature"]dist.shasum(integrity present)dist.unpackedSizedist.fileCountfunding/devDependencies/ etc.dist["npm-signature"]dominates. (dist.signatures, kept, is a further ~18% that this PR intentionally leaves in place.)Methodology note: the table simulates the per-version and
disttrims against npm's already-abbreviated metadata, which does not contain top-levelreadme/_id/_rev. pnpr fetches the full upstream document and abbreviates it, so the real reduction is at least the figures above plus the full README text that is additionally dropped at the top level.The win lands on the proxy/registry path (where the client still resolves locally); it does not affect the
/v1/installaccelerator path, where the client never receives a packument.Follow-up
Dropping
dist.tarball(a further ~8% on react) is tracked separately in #12164 — it first needs pnpm and pacquet to reconstruct the URL when it is absent.Safety
getIntegrityreturnsintegrityoutright when present;dist.signatures/npm-signature/unpackedSizeare consumed only by commands (audit signatures,view) that fetch their own full metadata, which pnpr serves unstripped.PackageVersionhas#[serde(default)]ondev_dependenciesand ignoresshasum/ signature / size fields, so a pacquet-as-client deserialization stays valid.Tests
upstream/tests.rs: one asserts every dropped field is gone (incl.npm-signature,fileCount,unpackedSize, andshasum-when-integrity-present) whileos/cpu/libc,time, anddist.signaturessurvive; another assertsshasumis kept whenintegrityis absent.registry_mock.rsto assertreadme/readmeFilenameare dropped.No changeset (pnpr is a Rust binary, not a published npm package).
Written by an agent (Claude Code, claude-opus-4-8).
Summary by CodeRabbit
Bug Fixes
Tests