perf: dedupe and cache hot-path work in install and resolver#449
perf: dedupe and cache hot-path work in install and resolver#449jdx merged 9 commits intoendevco:mainfrom
Conversation
Greptile SummaryThis PR applies a focused set of hot-path performance improvements across the install pipeline, resolver, and registry client: SHA-256 is replaced with BLAKE3 in Confidence Score: 5/5Safe to merge — no correctness issues found across any changed file. All changes are well-scoped optimizations with no behavioral regressions. Edge cases are handled (cached-as-None auth tokens, key-identity check on leaf-hash reuse, correct IgnoredAny semantics for the workspaces shape check). Previous P2 review findings have been resolved. No P1 or P0 findings identified. No files require special attention. Important Files Changed
Reviews (2): Last reviewed commit: "fixes" | Re-trigger Greptile |
A handful of small structural cleanups in the install pipeline, the resolver, and the registry client. None of the changes alter on-disk output, lockfile contents, fetched bytes, or run-script outcomes. The focus is removing work that runs more often than the surrounding code needs it to.
lockfile: graph-hash uses blake3
graph_hash.rsproduced an internal-only digest withSha256::digest, which is inconsistent with the rest of the codebase. Project policy already calls for blake3 on non-crypto hashes, and the digest is never exposed externally (it only names directories under aube's own virtual-store layout). Swapping toblake3::hashbrings the function in line with the rest of the hashing surface and removes the lonesha2dependency from this code path.The hex output stays the same length, so callers that take a prefix (
append_hex_to_leaf, the 16-char short form used in directory names) keep their existing behavior.delta: fold patch hash into fingerprint, cache touched set
Two issues addressed in
delta.rs:fingerprint()did not consider thepnpm.patchedDependenciespatch content. Two installs with identical lockfile entries but different patch text would produce identical fingerprints, so a re-patched package would not land inDeltaPlan.changed. The patch hash is now folded into the fingerprint via the same length-prefixed encoding used for every other field.DeltaPlan::should_touchwas a linear scan over both theaddedandchangedvectors. The linker walks the graph several times per install and probes the plan repeatedly, so the same scan repeats per probe. The plan now stores a precomputedBTreeSetbuilt once duringdiff(), andshould_touchconsults it directly.touched_setreturns a borrowed reference instead of constructing a new set each call.compute_subtree_hashespreviously walked the leaf fingerprints internally. The standalonecompute_package_hashescall site then walked them again. Both call sites now sharecompute_leaf_and_subtree_hashes, which returns both maps from a single pass.install: reuse pre-parsed lockfile, thread patch hashes
The metadata-overlay pass in
install::runre-parsed the lockfile to recover fields the resolver could not hold (license,funding_url, bun'sconfigVersion). The pre-parse already performed at the top of the function for resolver seeding now feeds that overlay directly. The fallback path still callsparse_lockfile_dir_remappedfor installs that do not run with a seeded resolver.Patch loading was hoisted ahead of the delta hash computation so the new patch-hash field in
fingerprint()sees the right value on every code path. Without that ordering, the first install after a patch change would compute hashes against a stale patches map.settings: capture env once via LazyLock
capture_envwalksstd::env::vars()and collects into aVec<(String, String)>. The function is reached from the settings layer, the registry config loader, the install state hasher, and the script runner, often more than once per command. Each call repeats the env walk, which on Windows is a syscall.The walk now runs lazily on first read, stored in a process-wide
LazyLock<Vec<(String, String)>>. The existingcapture_envAPI still returns an ownedVecfor callers that mutate the result, and a newprocess_env()helper exposes a&'static [(String, String)]for read-only callers.scripts: reuse thread-local buffer in BuildPolicy::decide
BuildPolicy::decide(name, version)didformat!("{name}@{version}")on every call to build the lookup key for the versioned-allow set. Every package in the install graph runs through this function, often across multiple passes for graph hashing and policy resolution. The allocation is unnecessary because the key is a transient lookup input that only lives until theHashSet::containsreturns.The function now writes into a thread-local
Stringbuffer that is cleared and reused per call. Set lookups useb.as_str()so the borrowed view never outlives the buffer.cli: pre-parse candidate versions in aube add
The
highest_satisfyingclosure inadd::run_with_executorsorted the packument versions with a comparator that callednode_semver::Version::parseon both sides every comparison. For packages with hundreds of versions (lodash being the canonical example), the sort drove the parser throughO(M log M)calls to satisfy a single dist-tag resolution.The closure now parses each candidate once into a
Vec<(&String, Version)>, sorts the parsed pairs by their already-computedVersion, and reuses the same parsed values when scanning for the highest satisfying entry.cli: scan package.json shape for workspaces field
find_workspace_rootwalks from the current directory up to$HOME, callingaube_manifest::PackageJson::from_pathon every ancestor that contains apackage.json. The full parser allocates the deps map, the dev-deps map, the scripts map, and aserde_json::Valuefor the flatten-extra catch-all, all to answer a yes-or-no question: is theworkspacesfield present?The walk now uses a private
ShapeOnlystruct that deserializes a singleOption<serde::de::IgnoredAny>forworkspacesand ignores everything else, so the parser short-circuits as soon as the field is decided.find_workspace_yaml_rootstill uses the original path for the catalog/settings loaders that need the typed struct.registry: memoize resolved auth token per URL
registry_auth_token_for(url)walkedauth_by_urifor a longest-prefix match on every authed request. The token-helper output was already cached, but the surrounding URL match still ran for every fetch. Tarball fetches in particular call this on every request, so the same prefix walk repeats for every package in the install.RegistryClientgains anauth_token_by_urlmap that stores the resolved token (orNone) per registry URL. The first request for a URL pays the prefix walk; later requests read the cached value. The cache stays valid for the life of the client because the underlying config does not change once loaded.