feat(install): aube-util::http module + pre-resolver prefetch + cold-path optimizations by imjustprism · Pull Request #529 · endevco/aube

imjustprism · 2026-05-06T19:05:01Z

Added

`aube-util::http` module

new shared HTTP helpers under crates/aube-util/src/http/. consolidates client-side primitives so leaf crates stop reinventing them. killswitch convention follows aube-util — every optimization defaults ON ships an AUBE_DISABLE_* env var documented on the function reading it.

prewarm — multi-origin HEAD prewarm. fire-and-forget over the current tokio runtime; failures trace at debug
priority — RFC 9218 Priority header builder + Urgency enum (Critical / High / Default / Background)
race — speculative parallel GET. first 2xx wins, siblings abort. for future anycast-IP racing on Cloudflare-fronted origins
resolve — DNS preresolve via tokio::net::lookup_host + host_port parser for registry-style URLs
ticket_cache — cross-invocation TLS session ticket store. on-disk JSON via fs_atomic::atomic_write, MAX_AGE pruning at load, MAX_PER_HOST eviction, SPKI fingerprint binding for cert-rotation invalidation

Pre-resolver direct-dep packument prefetch (`crates/aube/src/commands/install/prefetch.rs`)

reads package.json keys as raw strings and fires fire-and-forget packument GETs for every registry-shaped direct dep before workspace yaml load, settings resolve, lockfile parse, pnpmfile setup, and resolver construction run. by the time the resolver pops its first task, the on-disk packument cache and reqwest connection pool are warm. spec parsing skips workspace/file/link/git/http/catalog protocols and npm: aliases (alias targets prefetch via the resolver's alias rewrite path). no-op when offline or when any lockfile is present (resolver hits the integrity-keyed lockfile fast path; prefetch would be pure bandwidth waste). honors AUBE_DISABLE_PREFETCH=1.

Changed

Multi-registry TLS prewarm with parallel DNS preresolve

RegistryClient::prewarm_connection covers default registry + every scoped registry from .npmrc (@org:registry=...) + every per-uri auth registry that owns its own pool. previously prewarmed only the default registry, forcing the first scoped/auth-uri packument to pay the full TLS+TCP+ALPN cost. parallel DNS preresolve fires for every prewarm target before the HEAD requests so DNS RTT hides behind the handshake instead of stacking sequentially.

`compute_graph_hashes_with_patches` off the tokio worker

the BLAKE3 graph-hash walk now runs via spawn_blocking so the canonical-to-contextualized map build, nested_link_targets walk, and first materialize_rx recv keep making progress in parallel with hashing. previously blocked the executor on a CPU-bound pass before any reflink could land.

RFC 9218 Priority header on packument GETs

abbreviated packument fetch marks itself Critical (u=0) so Cloudflare/Fastly H2 schedulers prioritize resolver-blocking metadata ahead of pending tarball frames on the shared connection.

Fixed

Patches loaded once per cwd

load_patches_for_linker previously walked patches/ from disk 2-3 times per install (lockfile-prewarm site + no-lockfile-prewarm site + link-phase site). cached per cwd via OnceLock<Mutex<HashMap<PathBuf, ...>>>; first call populates, subsequent calls return clones. patches do not change mid-install — pnpm.patchedDependencies is read at install entry and the files are static.

Why it's better

structured HTTP utilities — leaf crates (registry/store/future modules) share one warm-pool/preresolve/priority/race surface with consistent killswitch semantics. previously inline in aube-registry/src/client.rs.
cold path overlap — DNS, TLS, packument GET, and graph-hash walk all overlap with the manifest/workspace/settings critical path that previously serialized them.
multi-registry support — installs against a corp registry plus npmjs no longer pay sequential handshakes.
future-proofing — ticket_cache ships the on-disk format + APIs ready for rustls ClientSessionStore wiring; race is ready for anycast-IP candidate sets; priority is ready for tarball-side High + incremental markers.

Killswitches added

every optimization defaults ON, presence-truthy disables.

AUBE_DISABLE_DNS_PRERESOLVE
AUBE_DISABLE_REQUEST_RACING
AUBE_DISABLE_PREFETCH
AUBE_DISABLE_TLS_TICKET_CACHE

joins existing AUBE_DISABLE_SPECULATIVE_TLS.

…old-path optimizations

greptile-apps · 2026-05-06T19:18:44Z

Greptile Summary

This PR introduces a new aube-util::http module consolidating shared HTTP primitives (prewarm, priority, race, resolve, ticket_cache), adds a pre-resolver packument prefetch that overlaps cold network work with the manifest/settings/workspace critical path, expands multi-registry TLS prewarm to cover scoped and auth-uri registries, offloads the BLAKE3 graph-hash walk to spawn_blocking, and caches load_patches_for_linker results per cwd.

aube-util::http: five new submodules with consistent killswitch semantics; race.rs now correctly accumulates all candidate failures (transport + non-2xx) into AllFailed, addressing the previous review finding.
prefetch.rs: fire-and-forget packument GETs for direct deps before resolver construction; correctly no-ops when offline or any lockfile is present; make_client reads .npmrc so scoped registry auth is available at prefetch time.
mod.rs / patches.rs: correctness fixes \u2014 identity dep_path silent-skip on lockfile path and redundant disk walks for patch files.

Confidence Score: 5/5

Safe to merge; all changes are additive optimizations with well-placed killswitches and two bounded correctness fixes.

All changed paths are either new opt-in helpers or targeted fixes to existing install logic. The new HTTP module is not yet wired into production call sites beyond prewarm and priority headers. The prefetch and hash-offload changes are guarded by offline checks and explicit awaits.

ticket_cache.rs — RwLock held during file I/O in save(); prefetch.rs — peerDependencies included in prefetch sweep.

Important Files Changed

Filename	Overview
crates/aube-util/src/http/ticket_cache.rs	New cross-invocation TLS session ticket cache with on-disk JSON persistence, MAX_AGE pruning, SPKI fingerprint binding, and atomic writes; RwLock held during file I/O in save() blocks concurrent put() operations
crates/aube/src/commands/install/prefetch.rs	New pre-resolver direct-dep packument prefetch; correctly guards on offline mode and existing lockfiles, but includes peerDependencies which are often not installed by the declaring project
crates/aube-util/src/http/race.rs	Speculative parallel GET with first-2xx-wins logic; prior review non-2xx discard bug addressed — all candidate failures now collected into AllFailed
crates/aube-registry/src/client.rs	prewarm_connection expanded to cover scoped registries with dedup normalization; RFC 9218 Priority header added to all packument fetches
crates/aube/src/commands/install/mod.rs	compute_graph_hashes_with_patches moved to spawn_blocking; canonical_to_contextualized map logic fixed to avoid silently skipping identity dep_paths on the lockfile path
crates/aube/src/patches.rs	load_patches_for_linker result cached per cwd via OnceLock+Mutex+HashMap to avoid redundant disk walks

_{Reviews (4): Last reviewed commit: "[autofix.ci] apply automated fixes" | Re-trigger Greptile}

jdx · 2026-05-06T20:33:14Z

Benchmark: PR #529 vs `main`

Scenario	main	PR #529	Δ
Warm cache, GVS on (`gvs-warm`)	0.179s ± 0.009s	0.170s ± 0.004s	−5.0% faster
Cold cache, GVS on (`gvs-cold`)	1.410s ± 0.171s	1.331s ± 0.026s	−5.6% faster (within noise)
Warm cache, GVS off (`ci-warm`)	0.336s ± 0.037s	0.252s ± 0.006s	−25.1% faster
Cold cache, GVS off (`ci-cold`)	1.231s ± 0.046s	1.176s ± 0.040s	−4.5% faster

Setup

main = de55e03, PR feat(install): aube-util::http module + pre-resolver prefetch + cold-path optimizations #529 = f8293fc
mise run bench:pr-equivalent invocation: RUNS=5 WARMUP=1 BENCH_TOOLS=aube BENCH_PHASES=0
Hermetic Verdaccio registry behind a 500mbit / 50ms throttle proxy (BENCH_HERMETIC=1 BENCH_BANDWIDTH=500mbit BENCH_LATENCY=50ms)
Fixture: benchmarks/fixture.package.json → 1229 packages, ~55.9 MB
Machine: AMD Ryzen 9 7950X3D, 32 threads, 30 GiB RAM, Linux 6.8 x86_64
Cold-cache scenarios wipe {store, cache, ~/.cache/aube, ~/.local/share/aube} between runs and start from a populated ~/.cache/aube-bench/registry/, so they isolate fetch+import+link from npmjs latency.
"GVS on" = npm_config_enable_global_virtual_store=true; "GVS off" = false (the CI default).

Read

Warm-cache CI install (-25%) is the headline. That's the path the prefetch + multi-registry prewarm + RFC 9218 priority + cached load_patches_for_linker changes target most directly, and it's the one that moves cleanly outside noise.
Warm-cache GVS install also picks up a real (-5%) win on top of an already-fast path (~170ms).
Both cold-cache scenarios trend faster but have wide enough stddevs (especially gvs-cold on main, ±171ms) that I'd want more runs to call the Δ significant. The throttled hermetic registry keeps cold timings dominated by the simulated 50ms RTT, which is exactly what the prefetch is supposed to hide — the trend is the right direction.

This comment was generated by Claude.

imjustprism marked this pull request as draft May 6, 2026 19:09

feat(install): add aube-util::http module + pre-resolver prefetch + c…

c056d5a

…old-path optimizations

imjustprism force-pushed the perf-materializer-fix branch from 05350ed to c056d5a Compare May 6, 2026 19:10

[autofix.ci] apply automated fixes

aacb582

greptile-apps Bot reviewed May 6, 2026

View reviewed changes

Comment thread crates/aube-util/src/http/race.rs

Comment thread crates/aube-registry/src/client.rs Outdated

imjustprism added 2 commits May 6, 2026 21:24

fixes

1f8a980

fixes

f8293fc

imjustprism marked this pull request as ready for review May 6, 2026 21:19

imjustprism marked this pull request as draft May 6, 2026 22:05

imjustprism and others added 2 commits May 7, 2026 00:27

fixes

e9441dd

[autofix.ci] apply automated fixes

eb848b6

imjustprism marked this pull request as ready for review May 6, 2026 22:39

jdx merged commit 33e6936 into endevco:main May 6, 2026
16 checks passed

chenrui333 mentioned this pull request May 7, 2026

aube 1.9.1 chenrui333/homebrew-tap#6761

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(install): aube-util::http module + pre-resolver prefetch + cold-path optimizations#529

feat(install): aube-util::http module + pre-resolver prefetch + cold-path optimizations#529
jdx merged 6 commits intoendevco:mainfrom
imjustprism:perf-materializer-fix

imjustprism commented May 6, 2026

Uh oh!

greptile-apps Bot commented May 6, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

jdx commented May 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

imjustprism commented May 6, 2026

Added

aube-util::http module

Pre-resolver direct-dep packument prefetch (crates/aube/src/commands/install/prefetch.rs)

Changed

Multi-registry TLS prewarm with parallel DNS preresolve

compute_graph_hashes_with_patches off the tokio worker

RFC 9218 Priority header on packument GETs

Fixed

Patches loaded once per cwd

Why it's better

Killswitches added

Uh oh!

greptile-apps Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

jdx commented May 6, 2026

Benchmark: PR #529 vs main

Setup

Read

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

`aube-util::http` module

Pre-resolver direct-dep packument prefetch (`crates/aube/src/commands/install/prefetch.rs`)

`compute_graph_hashes_with_patches` off the tokio worker

greptile-apps Bot commented May 6, 2026 •

edited

Loading

Benchmark: PR #529 vs `main`