Skip to content

chore: refresh benchmarks for v1.5.2#452

Merged
jdx merged 1 commit intomainfrom
bench-refresh
May 1, 2026
Merged

chore: refresh benchmarks for v1.5.2#452
jdx merged 1 commit intomainfrom
bench-refresh

Conversation

@mise-en-dev
Copy link
Copy Markdown
Collaborator

@mise-en-dev mise-en-dev commented May 1, 2026

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 1.5.2; the workspace is now 1.5.2. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Public ratios: warm installs vs Bun 4x -> 5x; warm installs vs pnpm 7x -> 6x; repeat test vs Bun 4x -> 3x; repeat test vs pnpm 34x -> 28x.

Benchmark aube bun pnpm npm yarn
Fresh install (warm cache) 380ms -> 288ms (-24%) 1600ms -> 1504ms (-6%) 2523ms -> 1849ms (-27%) 6313ms -> 6064ms (-4%) 12527ms -> 12899ms (+3%)
Fresh install (cold cache) 5080ms -> 5057ms (0%) 4264ms -> 4361ms (+2%) 4964ms -> 4907ms (-1%) 8206ms -> 7525ms (-8%) 24501ms -> 26049ms (+6%)
CI install (warm cache, GVS disabled) 1225ms -> 416ms (-66%) 1205ms -> 966ms (-20%) 2168ms -> 2237ms (+3%) 5699ms -> 6294ms (+10%) 11501ms -> 12875ms (+12%)
CI install (cold cache, GVS disabled) 5322ms -> 5053ms (-5%) 4310ms -> 3918ms (-9%) 4745ms -> 5491ms (+16%) 7192ms -> 7388ms (+3%) 25125ms -> 25383ms (+1%)
npm install && npm run test 9ms -> 11ms (+22%) 37ms -> 37ms (0%) 303ms -> 304ms (+0%) 674ms -> 690ms (+2%) 364ms -> 364ms (0%)
Add dependency 888ms -> 539ms (-39%) 2217ms -> 885ms (-60%) 2237ms -> 2634ms (+18%) 6629ms -> 6286ms (-5%) 12477ms -> 13214ms (+6%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.

Generated by the bench-refresh workflow.

@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 1, 2026

Greptile Summary

This PR refreshes benchmarks/results.json for aube v1.5.2 and updates the BENCH_RATIOS prose block in README.md accordingly. The README ratios are arithmetically consistent with the new data (warm-install: pnpm/aube = 1849/288 ≈ 6x ✓, Bun/aube = 1504/288 ≈ 5x ✓; repeat-test: pnpm/aube = 304/11 ≈ 28x ✓, Bun/aube = 37/11 ≈ 3x ✓), and within results.json the integer values, floating-point means, stddevs, and formatted text fields are all internally consistent across every benchmark scenario. The two large deltas called out in the PR description — aube CI warm install (1225ms → 416ms, −66%) and Bun add-dependency (2217ms → 885ms, −60%) — are the most obvious candidates to sanity-check against the raw hyperfine output before merging.

Confidence Score: 4/5

Safe to merge once the large aube CI-warm and Bun add-dependency improvements are spot-checked against raw run data.

No code defects found; all numbers are internally self-consistent. Score is 4 rather than 5 because benchmark PRs carry inherent unverifiability — the two unusually large deltas (−66% CI warm, −60% Bun add) warrant a manual cross-check of the raw hyperfine output as the PR itself recommends.

benchmarks/results.json — specifically the ci-warm and add scenario values for aube and Bun.

Important Files Changed

Filename Overview
README.md Updated BENCH_RATIOS block to reflect new warm-install and repeat-test ratios; ratios match the regenerated benchmark data.
benchmarks/results.json Refreshed all six benchmark scenarios; values, means, stddevs, and text summaries are internally consistent across all managers.

Reviews (1): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Benchmark changes

Public ratios: warm installs vs Bun 4x -> 8x; warm installs vs pnpm 7x -> 11x.

Benchmark aube bun pnpm
Fresh install (warm cache) 380ms -> 232ms (-39%) 1600ms -> 1910ms (+19%) 2523ms -> 2575ms (+2%)
CI install (warm cache, GVS disabled) 1225ms -> 401ms (-67%) 1205ms -> 1975ms (+64%) 2168ms -> 2531ms (+17%)
CI install (cold cache, GVS disabled) 5322ms -> 3778ms (-29%) 4310ms -> 4141ms (-4%) 4745ms -> 4803ms (+1%)

6850c8b vs 51e6a9c | aube/bun/pnpm | 3 scenarios | 3 runs | 500mbit/50ms | generated by Codex.

@jdx jdx merged commit afebb3b into main May 1, 2026
17 checks passed
@jdx jdx deleted the bench-refresh branch May 1, 2026 14:07
@chenrui333 chenrui333 mentioned this pull request May 1, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mise-en-dev @jdx