feat(pnpr): per-uplink proxy settings#12375
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
📝 WalkthroughWalkthroughAdds Verdaccio-style uplink tuning fields ( ChangesVerdaccio Uplink Tuning Knobs and Circuit Breaker
Sequence Diagram(s)sequenceDiagram
participant Client
participant Server
participant Upstream
participant CircuitBreaker
participant RemoteRegistry
Client->>Server: GET /pkg or GET /pkg/-/pkg.tgz
Server->>Upstream: fetch_packument / fetch_tarball_response
Upstream->>CircuitBreaker: available()?
alt Circuit open
CircuitBreaker-->>Upstream: false
Upstream-->>Server: RegistryError::UpstreamUnavailable
Server-->>Client: 503 Service Unavailable
else Circuit closed
Upstream->>RemoteRegistry: HTTP request (timeout, conditional headers)
alt 404
RemoteRegistry-->>Upstream: 404
Upstream->>CircuitBreaker: record_success
Upstream-->>Server: NotFound
Server-->>Client: 404
else 200 / tarball (cache=false)
RemoteRegistry-->>Upstream: 200 body
Upstream->>CircuitBreaker: record_success
alt upstream.caches() == false
Upstream-->>Server: bytes stream (no disk write)
else upstream.caches() == true
Upstream-->>Server: bytes stream (tee to disk cache)
end
Server-->>Client: 200 with body
else Transport/status error
RemoteRegistry-->>Upstream: error
Upstream->>CircuitBreaker: record_failure
Upstream-->>Server: RegistryError
Server-->>Client: 5xx
end
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #12375 +/- ##
==========================================
- Coverage 88.12% 88.12% -0.01%
==========================================
Files 311 311
Lines 41924 42081 +157
==========================================
+ Hits 36947 37083 +136
- Misses 4977 4998 +21 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
cd376df to
7054820
Compare
Integrated-Benchmark Report (Linux)Each scenario reports direct installs and pnpr installs. Bencher consumes pacquet@HEAD and pnpr@HEAD. Scenario: Isolated linker: fresh restore, cold cache + cold store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 4.0610706824400005,
"stddev": 0.14927018917692705,
"median": 4.02247400344,
"user": 3.7198304400000004,
"system": 3.4593697399999996,
"min": 3.90144135494,
"max": 4.345962477940001,
"times": [
3.9285775049400002,
3.9041100239400004,
4.22536951694,
4.13578477994,
4.345962477940001,
3.90144135494,
4.1459511969400005,
3.98983770994,
4.055110296940001,
3.97856196194
]
},
{
"command": "pacquet@main",
"mean": 3.9696701909399996,
"stddev": 0.14760023769783973,
"median": 3.8986793874400005,
"user": 3.72800904,
"system": 3.44347194,
"min": 3.82875336894,
"max": 4.27731359694,
"times": [
3.88611588894,
4.11055817394,
4.27731359694,
4.11821008594,
3.84599692594,
3.9431590829400003,
3.9066996429400005,
3.8892360109400004,
3.82875336894,
3.89065913194
]
},
{
"command": "pnpr@HEAD",
"mean": 2.1229439249400004,
"stddev": 0.14291004059386447,
"median": 2.06323769444,
"user": 2.5943600399999993,
"system": 2.97697504,
"min": 1.9726825829399999,
"max": 2.3718135309400004,
"times": [
2.03313345894,
2.31421147494,
2.05520340294,
2.27010026694,
2.01128504994,
2.3718135309400004,
2.07127198594,
2.12196015194,
2.00777734394,
1.9726825829399999
]
},
{
"command": "pnpr@main",
"mean": 2.1653577430400004,
"stddev": 0.16718165165735605,
"median": 2.1707169859400004,
"user": 2.58839224,
"system": 2.96771234,
"min": 1.9690240909399999,
"max": 2.4001875689400003,
"times": [
2.06167464994,
2.0208877319400003,
2.28195627394,
2.33117704694,
2.0376453419400002,
1.9690240909399999,
2.29991451794,
1.97135088594,
2.2797593219400003,
2.4001875689400003
]
}
]
}Scenario: Isolated linker: fresh restore, hot cache + hot store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 0.62678472968,
"stddev": 0.009944748690332192,
"median": 0.6278446344799999,
"user": 0.37680127999999996,
"system": 1.31533846,
"min": 0.6102446674800001,
"max": 0.64384199548,
"times": [
0.62224189448,
0.63217847948,
0.64384199548,
0.61671390548,
0.62861979148,
0.6102446674800001,
0.62005089948,
0.62706947748,
0.62914505448,
0.63774113148
]
},
{
"command": "pacquet@main",
"mean": 0.64023996628,
"stddev": 0.03980567009053697,
"median": 0.62901161198,
"user": 0.37636578,
"system": 1.3217823599999998,
"min": 0.60420867448,
"max": 0.73386336848,
"times": [
0.64273352448,
0.61994090548,
0.60950496648,
0.63808231848,
0.6147924514800001,
0.63925696248,
0.60420867448,
0.6184655704800001,
0.73386336848,
0.68155092048
]
},
{
"command": "pnpr@HEAD",
"mean": 0.6642405641800001,
"stddev": 0.016482777202454495,
"median": 0.65760180148,
"user": 0.39311057999999993,
"system": 1.3375949599999999,
"min": 0.64592270048,
"max": 0.69732299448,
"times": [
0.67716281448,
0.65196931248,
0.65640827448,
0.64592270048,
0.65057371148,
0.66612366548,
0.65879532848,
0.65546684248,
0.6826599974800001,
0.69732299448
]
},
{
"command": "pnpr@main",
"mean": 0.6748860238800001,
"stddev": 0.02339955913415537,
"median": 0.6679308124800001,
"user": 0.38547677999999996,
"system": 1.34012716,
"min": 0.65194334448,
"max": 0.71839348348,
"times": [
0.71839348348,
0.65412006648,
0.65653034748,
0.6570615454800001,
0.70049252848,
0.67471185848,
0.67304241548,
0.65194334448,
0.69974543948,
0.6628192094800001
]
}
]
}Scenario: Isolated linker: fresh install, cold cache + cold store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 4.178148886,
"stddev": 0.04517309363444247,
"median": 4.1821814881,
"user": 3.6710296,
"system": 3.3225696,
"min": 4.0824067281000005,
"max": 4.241079792100001,
"times": [
4.180078748100001,
4.1707369941,
4.2295494981,
4.0824067281000005,
4.1904587991,
4.241079792100001,
4.1842842281,
4.1713985011000005,
4.1973358661,
4.1341597051
]
},
{
"command": "pacquet@main",
"mean": 4.1841962733999996,
"stddev": 0.06171369972966779,
"median": 4.1693265366,
"user": 3.6993246999999996,
"system": 3.3514220999999997,
"min": 4.0997563291,
"max": 4.3036137051,
"times": [
4.0997563291,
4.1614043911000005,
4.1196847301,
4.175113956100001,
4.1552864031,
4.2479734621,
4.2367698201,
4.1788208201,
4.3036137051,
4.1635391171
]
},
{
"command": "pnpr@HEAD",
"mean": 2.1584465013,
"stddev": 0.14165126611594933,
"median": 2.1510134026,
"user": 2.4472230999999995,
"system": 2.8841744,
"min": 2.0027310501,
"max": 2.3582154050999997,
"times": [
2.0981822901,
2.0224058951,
2.2038445150999997,
2.3038761131,
2.2721417561,
2.0027310501,
2.2909176611,
2.0218555610999998,
2.0102947661,
2.3582154050999997
]
},
{
"command": "pnpr@main",
"mean": 2.2026771443,
"stddev": 0.12025837577450091,
"median": 2.1855375810999997,
"user": 2.4362332999999996,
"system": 2.8722429999999997,
"min": 2.0311656141,
"max": 2.3809472660999997,
"times": [
2.2168289940999997,
2.0311656141,
2.0845914780999997,
2.3336128891,
2.2968969881,
2.3809472660999997,
2.1542461681,
2.3030745160999997,
2.1222062300999998,
2.1032012990999998
]
}
]
}Scenario: Isolated linker: fresh install, hot cache + hot store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 1.3167593063999998,
"stddev": 0.013302422011812217,
"median": 1.3147163573,
"user": 1.3305488600000002,
"system": 1.70386994,
"min": 1.2959707908,
"max": 1.3414410038,
"times": [
1.3414410038,
1.2959707908,
1.3129698498,
1.3211799738,
1.3164628648,
1.3058233368,
1.3294036288,
1.3113265788,
1.3267088787999999,
1.3063061578
]
},
{
"command": "pacquet@main",
"mean": 1.3449849321999998,
"stddev": 0.015402126545891402,
"median": 1.3468907042999998,
"user": 1.33473156,
"system": 1.7322765399999998,
"min": 1.3184252128,
"max": 1.3715250538,
"times": [
1.3427890938,
1.3524721518,
1.3483736447999999,
1.3715250538,
1.3438501828,
1.3522075798,
1.3218405098,
1.3184252128,
1.3529581287999999,
1.3454077638
]
},
{
"command": "pnpr@HEAD",
"mean": 0.6657039173,
"stddev": 0.02812540336395781,
"median": 0.6587900098000001,
"user": 0.34954376,
"system": 1.28956214,
"min": 0.6478848738,
"max": 0.7434221268000001,
"times": [
0.6546380908000001,
0.6487201618,
0.7434221268000001,
0.6647869898000001,
0.6576669418000001,
0.6478848738,
0.6511521818000001,
0.6687184148,
0.6601363138,
0.6599130778000001
]
},
{
"command": "pnpr@main",
"mean": 0.6476592129000001,
"stddev": 0.007437939000548968,
"median": 0.6487087588000001,
"user": 0.34498895999999996,
"system": 1.2744724399999998,
"min": 0.6279229998000001,
"max": 0.6545993188000001,
"times": [
0.6480928348000001,
0.6467978488,
0.6521242608000001,
0.6518080098000001,
0.6474575908000001,
0.6519137568000001,
0.6545993188000001,
0.6465508258000001,
0.6279229998000001,
0.6493246828000001
]
}
]
}Scenario: Isolated linker: fresh install, cold cache + hot store
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 3.0558154421399997,
"stddev": 0.0459558488577677,
"median": 3.04978394464,
"user": 1.8334910599999996,
"system": 1.9569690799999997,
"min": 2.99563141014,
"max": 3.1673494141400003,
"times": [
3.0357952691400003,
2.99563141014,
3.08313995014,
3.05221389214,
3.04735399714,
3.0608630531400003,
3.04513310314,
3.1673494141400003,
3.05451009214,
3.01616424014
]
},
{
"command": "pacquet@main",
"mean": 3.00183054784,
"stddev": 0.0469961699496609,
"median": 2.99198614364,
"user": 1.7963371599999998,
"system": 1.9519771799999996,
"min": 2.94394798314,
"max": 3.1101242081400002,
"times": [
2.99314876614,
2.94394798314,
3.01127858214,
3.00676132614,
2.9877888231400003,
2.9789217441400004,
2.95454330414,
3.1101242081400002,
2.9908235211400003,
3.04096722014
]
},
{
"command": "pnpr@HEAD",
"mean": 0.6876922842400001,
"stddev": 0.007706630034002713,
"median": 0.6872709931400001,
"user": 0.35090996,
"system": 1.32806398,
"min": 0.6795664691400001,
"max": 0.70506737714,
"times": [
0.70506737714,
0.6831016831400001,
0.6816610001400001,
0.69513654414,
0.68779257114,
0.68674941514,
0.6887557681400001,
0.6885404661400001,
0.6795664691400001,
0.6805515481400001
]
},
{
"command": "pnpr@main",
"mean": 0.6541642445400001,
"stddev": 0.013227966136486892,
"median": 0.65007333564,
"user": 0.33587586,
"system": 1.2848566799999999,
"min": 0.64426750414,
"max": 0.6874317681400001,
"times": [
0.65129509314,
0.64575223914,
0.65775898714,
0.64583627314,
0.64885157814,
0.64521211714,
0.64426750414,
0.65142816514,
0.66380872014,
0.6874317681400001
]
}
]
} |
|
| Branch | pr/12375 |
| Testbed | pacquet |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| isolated-linker.fresh-install.cold-cache.cold-store | 📈 view plot 🚷 view threshold | 4,178.15 ms(-0.95%)Baseline: 4,218.39 ms | 5,062.06 ms (82.54%) |
| isolated-linker.fresh-install.cold-cache.hot-store | 📈 view plot 🚷 view threshold | 3,055.82 ms(+1.42%)Baseline: 3,013.03 ms | 3,615.63 ms (84.52%) |
| isolated-linker.fresh-install.hot-cache.hot-store | 📈 view plot 🚷 view threshold | 1,316.76 ms(-1.08%)Baseline: 1,331.12 ms | 1,597.35 ms (82.43%) |
| isolated-linker.fresh-restore.cold-cache.cold-store | 📈 view plot 🚷 view threshold | 4,061.07 ms(-3.14%)Baseline: 4,192.59 ms | 5,031.11 ms (80.72%) |
| isolated-linker.fresh-restore.hot-cache.hot-store | 📈 view plot 🚷 view threshold | 626.78 ms(+1.60%)Baseline: 616.90 ms | 740.29 ms (84.67%) |
|
| Branch | pr/12375 |
| Testbed | pnpr |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Latency | milliseconds (ms) |
|---|---|---|
| isolated-linker.fresh-install.cold-cache.cold-store | 📈 view plot | 2,158.45 ms |
| isolated-linker.fresh-install.cold-cache.hot-store | 📈 view plot | 687.69 ms |
| isolated-linker.fresh-install.hot-cache.hot-store | 📈 view plot | 665.70 ms |
| isolated-linker.fresh-restore.cold-cache.cold-store | 📈 view plot | 2,122.94 ms |
| isolated-linker.fresh-restore.hot-cache.hot-store | 📈 view plot | 664.24 ms |
7054820 to
77d6de2
Compare
PR Summary by Qodopnpr: support verdaccio per-uplink proxy knobs (maxage/timeout/breaker/cache) Description
Diagram
High-Level Assessment
Files changed (7)
|
There was a problem hiding this comment.
Pull request overview
This PR extends pnpr (the registry proxy) to honor Verdaccio-style per-uplink tuning knobs from config.yaml, so each uplink can independently control packument freshness (maxage), request deadlines (timeout), circuit breaking (max_fails/fail_timeout), and tarball mirroring (cache: false).
Changes:
- Add per-uplink fields to runtime config (
UplinkConfig) and parse Verdaccio interval formats with validation. - Implement per-uplink timeouts, a circuit breaker, and cache/no-cache tarball behavior in upstream fetch + server paths.
- Add unit/integration tests covering interval parsing, uplink resolution defaults/errors, circuit breaker behavior,
maxageTTL override, andcache: falsestreaming.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| pnpr/crates/pnpr/tests/server.rs | Adds integration tests for per-uplink maxage overriding global TTL and cache: false tarball streaming without mirroring. |
| pnpr/crates/pnpr/src/upstream/tests.rs | Updates upstream test construction for new UplinkConfig-based API and adds circuit breaker tests. |
| pnpr/crates/pnpr/src/upstream.rs | Implements per-uplink timeout/maxage/cache plumbing and introduces a circuit breaker that can short-circuit requests. |
| pnpr/crates/pnpr/src/server.rs | Wires per-uplink upstream configs into router state; applies uplink maxage to packument TTL and streams tarballs when cache: false. |
| pnpr/crates/pnpr/src/error.rs | Adds UpstreamUnavailable error variant mapped to HTTP 503 for open-circuit behavior. |
| pnpr/crates/pnpr/src/config/tests.rs | Adds tests for interval parsing, uplink knob defaults, explicit knob parsing, and invalid-interval errors. |
| pnpr/crates/pnpr/src/config.rs | Extends UplinkConfig/UplinkFile, adds interval parsing, and resolves per-uplink tuning knobs from config YAML. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Code Review by Qodo
Context used 1. Breaker probe can stick
|
|
Code review by qodo was updated up to the latest commit 77d6de2 |
…, fail_timeout, cache)
Add verdaccio's per-uplink tuning knobs to pnpr's proxy uplinks, parsed
from the same config.yaml shape verdaccio uses:
- maxage: per-uplink packument freshness window; overrides the
global packument_ttl when set, otherwise defers to it.
- timeout: per-request deadline applied to every upstream fetch
(default 30s).
- max_fails / circuit breaker: after max_fails consecutive failures the
fail_timeout uplink short-circuits with a 503 until the fail_timeout
cooldown lapses, then a single probe is allowed through
(defaults 2 / 5m).
- cache: when false, tarballs stream through without being written
to the local mirror (default true).
Interval values accept verdaccio's format ("2m", "30s", "1h30m", or a
bare number of seconds) via a new parse_interval; an unparseable value
is a named InvalidConfig error rather than a silent fallback. Defaults
match verdaccio (timeout 30s, max_fails 2, fail_timeout 5m, cache true).
Covered by unit tests (interval parsing, uplink resolution, circuit
breaker) and integration tests (maxage overriding the global TTL, and a
cache:false uplink streaming a tarball without mirroring it).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
77d6de2 to
da83f42
Compare
|
Code review by qodo was updated up to the latest commit da83f42 |
Address review feedback on the per-uplink settings: - parse_interval: use Duration::try_from_secs_f64 so an out-of-range but finite interval (e.g. "1e30") surfaces as a named InvalidConfig error instead of panicking pnpr at config load. - Circuit breaker: collapse the split atomic counter + mutex timestamp into one Mutex<BreakerState>, so a threshold check and its timestamp can never be observed half-updated (closes the bypass race window). The half-open state now admits exactly one probe (a probe-in-flight permit) rather than letting every waiting caller stampede a recovering upstream. A poisoned lock is recovered via into_inner rather than panicking on the request path. - Only a 5xx counts against the breaker; a non-404 4xx (auth, rate-limit, bad request) is an authoritative answer, not an availability signal, so it no longer opens the circuit and mask the real status behind a 503. - UpstreamUnavailable now carries the configured uplink name instead of the upstream URL, so the client-facing 503 doesn't disclose the internal endpoint. - Drop the unused, self-contradictory UplinkConfig::DEFAULT_MAXAGE constant; an unset maxage defers to the global packument_ttl. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Code review by qodo was updated up to the latest commit 31d5f5c |
Follow-up to the per-uplink review: - Circuit breaker half-open probe could stick: the probe_in_flight flag was only cleared by a record_*() path, so a request cancelled or dropped after try_acquire admitted it (e.g. a client disconnect) left the flag set and short-circuited the uplink with 503 forever. Gate the probe on the cooldown window itself instead — admitting a probe re-arms last_failure, so concurrent callers wait one fail_timeout and an abandoned probe simply lets the window lapse. Drops the sticky flag (and the now-unneeded record_neutral); a non-404 4xx just leaves the breaker untouched. - Bare-number intervals now deserialize: maxage/timeout/fail_timeout were Option<String>, so a verdaccio-shaped `timeout: 30` (a YAML number) failed to parse even though parse_interval advertises bare-number support. A new Interval type accepts both a string and a numeric scalar (read as seconds), keeping the raw value for parse_interval. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Code review by qodo was updated up to the latest commit 55c7b4b |
Summary
Adds verdaccio's per-uplink tuning knobs to pnpr's proxy uplinks, parsed from the same
config.yamlshape verdaccio uses. Previously these fields were accepted and silently dropped; now each uplink can be tuned independently.This is scoped to pnpr (the registry proxy) — no pnpm/pacquet CLI parity work is involved.
Part of the verdaccio-parity tracking issue #11973 — ticks the "Per-uplink timeouts,
max_fails,fail_timeout,cache: false" item under Uplinks & caching.Settings
maxagepackument_ttl--packument-ttl-secs; when omitted, the global value still applies.timeout30smax_fails2fail_timeout5mcachetruefalse, tarballs stream straight through to the client without being written to the local mirror.Behavior notes
max_failsconsecutive failures, requests to the uplink short-circuit with503 Service Unavailable(surfaced asUpstreamUnavailable) instead of hammering a known-down upstream. Oncefail_timeoutlapses, one probe is allowed through: success resets the breaker, failure restarts the cooldown. A 404/304/200 counts as success; transport errors and 5xx count as failures. On the packument path an open breaker degrades gracefully to the stale-cache fallback.max_fails: 0disables the breaker.2m,30s,1h30m,2m 30s), or a bare number read as seconds. An unparseable value is a namedInvalidConfigerror (naming the offending field) rather than a silent fallback to the default.timeout 30s,max_fails 2,fail_timeout 5m,cache true).Testing
max_fails, resets on success, reopens after cooldown, disabled at0).maxageoverriding the global TTL (forces a revalidation that would otherwise be a cache hit), an open circuit hitting the upstream exactly once, and acache: falseuplink streaming a tarball without writing it to the mirror.cargo fmt,clippy --deny warnings, rustdoc (-D warnings),dylint, and all 297 pnpr tests.Squash commit message
Written by an agent (Claude Code, claude-opus-4-8).
Summary by CodeRabbit
Release Notes
New Features
Bug Fixes
Tests