Skip to content

chore: refresh benchmarks for v1.5.2#459

Merged
jdx merged 1 commit intomainfrom
bench-refresh
May 1, 2026
Merged

chore: refresh benchmarks for v1.5.2#459
jdx merged 1 commit intomainfrom
bench-refresh

Conversation

@mise-en-dev
Copy link
Copy Markdown
Collaborator

@mise-en-dev mise-en-dev commented May 1, 2026 •
edited
Loading

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 1.5.2; the workspace is now 1.5.2. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Public ratios: warm installs vs Bun 5x -> 6x; warm installs vs pnpm 6x -> 10x; repeat test vs Bun 3x -> 4x; repeat test vs pnpm 28x -> 35x.

Benchmark aube bun pnpm npm yarn
Fresh install (warm cache) 288ms -> 230ms (-20%) 1504ms -> 1488ms (-1%) 1849ms -> 2367ms (+28%) 6064ms -> 6416ms (+6%) 12899ms -> 13963ms (+8%)
Fresh install (cold cache) 5057ms -> 6047ms (+20%) 4361ms -> 4333ms (-1%) 4907ms -> 4723ms (-4%) 7525ms -> 8154ms (+8%) 26049ms -> 26771ms (+3%)
CI install (warm cache, GVS disabled) 416ms -> 564ms (+36%) 966ms -> 1295ms (+34%) 2237ms -> 2361ms (+6%) 6294ms -> 6616ms (+5%) 12875ms -> 13860ms (+8%)
CI install (cold cache, GVS disabled) 5053ms -> 5800ms (+15%) 3918ms -> 4278ms (+9%) 5491ms -> 4823ms (-12%) 7388ms -> 7794ms (+5%) 25383ms -> 26621ms (+5%)
npm install && npm run test 11ms -> 9ms (-18%) 37ms -> 40ms (+8%) 304ms -> 318ms (+5%) 690ms -> 686ms (-1%) 364ms -> 375ms (+3%)
Add dependency 539ms -> 476ms (-12%) 885ms -> 935ms (+6%) 2634ms -> 2500ms (-5%) 6286ms -> 6214ms (-1%) 13214ms -> 12967ms (-2%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.

Generated by the bench-refresh workflow.

@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 1, 2026 •
edited
Loading

Greptile Summary

This PR refreshes benchmarks/results.json with measurements from a new hermetic Verdaccio run and updates the BENCH_RATIOS marketing copy in the README. All advertised ratios (10× pnpm warm, 6× Bun warm, 35× pnpm repeat, 4× Bun repeat) are arithmetically consistent with the new JSON values.

Confidence Score: 5/5

Safe to merge; only benchmark data and README copy are changed, with no logic or API surface affected

All README ratios verify correctly against the updated JSON. The only notable finding is a high-variance ci-warm aube sample (P2), which the PR description already advises reviewers to inspect. No structural changes, no code path modifications.

benchmarks/results.json — the ci-warm aube stddev warrants a second look before merging

Important Files Changed

Filename Overview
README.md Updates BENCH_RATIOS marketing copy; new ratios (10x pnpm, 6x Bun warm; 35x pnpm, 4x Bun repeat) are consistent with the values in results.json
benchmarks/results.json Refreshed all benchmark measurements; the ci-warm aube entry has a notably high coefficient of variation (~39%) with a max nearly 2× its mean, worth a second look

Reviews (2): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026 •
edited
Loading

Benchmark changes

Public ratios: warm installs vs Bun 5x -> 5x; warm installs vs pnpm 6x -> 7x.

Benchmark aube bun pnpm
Fresh install (warm cache) 288ms -> 434ms (+51%) 1504ms -> 2027ms (+35%) 1849ms -> 3252ms (+76%)
CI install (warm cache, GVS disabled) 416ms -> 972ms (+134%) 966ms -> 2104ms (+118%) 2237ms -> 2753ms (+23%)
CI install (cold cache, GVS disabled) 5053ms -> 4129ms (-18%) 3918ms -> 4449ms (+14%) 5491ms -> 5461ms (-1%)

f705a32 vs 8a60d6d | aube/bun/pnpm | 3 scenarios | 3 runs | 500mbit/50ms | generated by Codex.

@jdx jdx merged commit 28582d9 into main May 1, 2026
16 checks passed
@jdx jdx deleted the bench-refresh branch May 1, 2026 20:33
@greptile-apps greptile-apps Bot mentioned this pull request May 1, 2026
Merged
@chenrui333 chenrui333 mentioned this pull request May 1, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mise-en-dev @jdx