Skip to content

perf: dedupe and cache hot-path work in install and resolver#449

Merged
jdx merged 9 commits intoendevco:mainfrom
imjustprism:perf
May 1, 2026
Merged

perf: dedupe and cache hot-path work in install and resolver#449
jdx merged 9 commits intoendevco:mainfrom
imjustprism:perf

Conversation

@imjustprism
Copy link
Copy Markdown
Contributor

A handful of small structural cleanups in the install pipeline, the resolver, and the registry client. None of the changes alter on-disk output, lockfile contents, fetched bytes, or run-script outcomes. The focus is removing work that runs more often than the surrounding code needs it to.

lockfile: graph-hash uses blake3

graph_hash.rs produced an internal-only digest with Sha256::digest, which is inconsistent with the rest of the codebase. Project policy already calls for blake3 on non-crypto hashes, and the digest is never exposed externally (it only names directories under aube's own virtual-store layout). Swapping to blake3::hash brings the function in line with the rest of the hashing surface and removes the lone sha2 dependency from this code path.

The hex output stays the same length, so callers that take a prefix (append_hex_to_leaf, the 16-char short form used in directory names) keep their existing behavior.

delta: fold patch hash into fingerprint, cache touched set

Two issues addressed in delta.rs:

  1. fingerprint() did not consider the pnpm.patchedDependencies patch content. Two installs with identical lockfile entries but different patch text would produce identical fingerprints, so a re-patched package would not land in DeltaPlan.changed. The patch hash is now folded into the fingerprint via the same length-prefixed encoding used for every other field.
  2. DeltaPlan::should_touch was a linear scan over both the added and changed vectors. The linker walks the graph several times per install and probes the plan repeatedly, so the same scan repeats per probe. The plan now stores a precomputed BTreeSet built once during diff(), and should_touch consults it directly. touched_set returns a borrowed reference instead of constructing a new set each call.

compute_subtree_hashes previously walked the leaf fingerprints internally. The standalone compute_package_hashes call site then walked them again. Both call sites now share compute_leaf_and_subtree_hashes, which returns both maps from a single pass.

install: reuse pre-parsed lockfile, thread patch hashes

The metadata-overlay pass in install::run re-parsed the lockfile to recover fields the resolver could not hold (license, funding_url, bun's configVersion). The pre-parse already performed at the top of the function for resolver seeding now feeds that overlay directly. The fallback path still calls parse_lockfile_dir_remapped for installs that do not run with a seeded resolver.

Patch loading was hoisted ahead of the delta hash computation so the new patch-hash field in fingerprint() sees the right value on every code path. Without that ordering, the first install after a patch change would compute hashes against a stale patches map.

settings: capture env once via LazyLock

capture_env walks std::env::vars() and collects into a Vec<(String, String)>. The function is reached from the settings layer, the registry config loader, the install state hasher, and the script runner, often more than once per command. Each call repeats the env walk, which on Windows is a syscall.

The walk now runs lazily on first read, stored in a process-wide LazyLock<Vec<(String, String)>>. The existing capture_env API still returns an owned Vec for callers that mutate the result, and a new process_env() helper exposes a &'static [(String, String)] for read-only callers.

scripts: reuse thread-local buffer in BuildPolicy::decide

BuildPolicy::decide(name, version) did format!("{name}@{version}") on every call to build the lookup key for the versioned-allow set. Every package in the install graph runs through this function, often across multiple passes for graph hashing and policy resolution. The allocation is unnecessary because the key is a transient lookup input that only lives until the HashSet::contains returns.

The function now writes into a thread-local String buffer that is cleared and reused per call. Set lookups use b.as_str() so the borrowed view never outlives the buffer.

cli: pre-parse candidate versions in aube add

The highest_satisfying closure in add::run_with_executor sorted the packument versions with a comparator that called node_semver::Version::parse on both sides every comparison. For packages with hundreds of versions (lodash being the canonical example), the sort drove the parser through O(M log M) calls to satisfy a single dist-tag resolution.

The closure now parses each candidate once into a Vec<(&String, Version)>, sorts the parsed pairs by their already-computed Version, and reuses the same parsed values when scanning for the highest satisfying entry.

cli: scan package.json shape for workspaces field

find_workspace_root walks from the current directory up to $HOME, calling aube_manifest::PackageJson::from_path on every ancestor that contains a package.json. The full parser allocates the deps map, the dev-deps map, the scripts map, and a serde_json::Value for the flatten-extra catch-all, all to answer a yes-or-no question: is the workspaces field present?

The walk now uses a private ShapeOnly struct that deserializes a single Option<serde::de::IgnoredAny> for workspaces and ignores everything else, so the parser short-circuits as soon as the field is decided. find_workspace_yaml_root still uses the original path for the catalog/settings loaders that need the typed struct.

registry: memoize resolved auth token per URL

registry_auth_token_for(url) walked auth_by_uri for a longest-prefix match on every authed request. The token-helper output was already cached, but the surrounding URL match still ran for every fetch. Tarball fetches in particular call this on every request, so the same prefix walk repeats for every package in the install.

RegistryClient gains an auth_token_by_url map that stores the resolved token (or None) per registry URL. The first request for a URL pays the prefix walk; later requests read the cached value. The cache stays valid for the life of the client because the underlying config does not change once loaded.

@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 1, 2026

Greptile Summary

This PR applies a focused set of hot-path performance improvements across the install pipeline, resolver, and registry client: SHA-256 is replaced with BLAKE3 in graph_hash.rs (consistent with project policy), patch content is folded into delta fingerprints to correctly detect re-patched packages, and redundant work — repeated env walks, per-call format! allocations, linear should_touch scans, O(M log M) semver re-parses, and repeated parse_lockfile and compute_package_hashes traversals — is eliminated via LazyLock, thread-local buffers, pre-computed sets, and shared leaf-hash passes. Both P2 concerns from the previous review round (the double KEY_BUF.with entry and the touched/touch_set naming collision) have been addressed in this revision.

Confidence Score: 5/5

Safe to merge — no correctness issues found across any changed file.

All changes are well-scoped optimizations with no behavioral regressions. Edge cases are handled (cached-as-None auth tokens, key-identity check on leaf-hash reuse, correct IgnoredAny semantics for the workspaces shape check). Previous P2 review findings have been resolved. No P1 or P0 findings identified.

No files require special attention.

Important Files Changed

Filename Overview
crates/aube-lockfile/src/graph_hash.rs SHA-256 → BLAKE3 swap for hash_canonical; doc comment updated to match; sha2 import removed, blake3 added. Output is the same 64-char hex string, so prefix callers are unaffected.
crates/aube-registry/src/client.rs Adds auth_token_by_url: Mutex<BTreeMap<String, Option<String>>> to memoize per-URL auth resolution. Fast path correctly handles the cached-as-None case (BTreeMap returns Some(&None), which is matched and cloned as None).
crates/aube-scripts/src/policy.rs Thread-local KEY_BUF replaces per-call format! allocation; single KEY_BUF.with closure now answers both deny and allow versioned probes, addressing the previous double-with concern. Deny order changed (wildcard now sits between exact-name-deny and versioned-deny) but all three are unconditional early returns so the outcome is identical.
crates/aube-settings/src/values.rs Env snapshot moved to LazyLock<Vec<(String,String)>>; capture_env() clones the cached vec, new process_env() returns a static slice. Callers that mutate the result still work correctly via the cloned owned Vec.
crates/aube/src/commands/add.rs Pre-parses and sorts packument versions once before the highest_satisfying closure, reducing O(M log M) parse calls to O(M + log M). Closure captures parsed_versions by shared ref; Rust's borrow checker ensures packument outlives the closure.
crates/aube/src/commands/install/delta.rs Adds touch_set: BTreeSet<String> (private, built once in diff()), making should_touch O(log N); renames old touched_set to return &BTreeSet<String>. Adds patch_hash param to fingerprint() and folds it via update_optional. Adds compute_leaf_and_subtree_hashes to share the BLAKE3 leaf pass. New naming touch_set explicitly avoids collision with the existing touched() counter method.
crates/aube/src/commands/install/mod.rs Patch loading hoisted before subtree-hash computation; pre-parsed lockfile reused for metadata overlay when resolver seeded in Fix/Prefer mode; leaf+subtree computed together via compute_leaf_and_subtree_hashes. Writeback reuse guard now checks both length AND key identity, addressing the node-count-only concern from the prior thread.
crates/aube/src/dirs.rs Adds package_json_has_workspaces using a ShapeOnly serde struct with Option<IgnoredAny> for the workspaces field, short-circuiting the full PackageJson parse during workspace-root walks. Correctly treats absent and null as None, matching original manifest.workspaces.is_some() semantics.
crates/aube-lockfile/Cargo.toml Adds blake3 dep alongside retained sha2 dep (sha2 is still used elsewhere in the crate).

Reviews (2): Last reviewed commit: "fixes" | Re-trigger Greptile

Comment thread crates/aube/src/commands/install/mod.rs
@jdx jdx merged commit f37054f into endevco:main May 1, 2026
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants