Skip to content

chore: refresh benchmarks for v1.16.1#808

Merged
jdx merged 1 commit into
mainfrom
bench-refresh
May 29, 2026
Merged

chore: refresh benchmarks for v1.16.1#808
jdx merged 1 commit into
mainfrom
bench-refresh

Conversation

@mise-en-dev

Copy link
Copy Markdown
Collaborator

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 1.16.0; the workspace is now 1.16.1. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Versions:

  • aube: 1.16.0 -> 1.16.1
  • deno: 2.8.0 -> 2.8.1
  • pnpm: 11.3.0 -> 11.4.0

Public ratios: warm installs vs Bun 3x -> 3x; warm installs vs pnpm 7x -> 5x; repeat test vs Bun 4x -> 4x; repeat test vs pnpm 32x -> 30x.

Benchmark aube bun deno pnpm npm yarn
Fresh install (warm cache) 420ms -> 552ms (+31%) 1384ms -> 1408ms (+2%) 1253ms -> 1648ms (+32%) 2851ms -> 2817ms (-1%) 9230ms -> 8090ms (-12%) 10082ms -> 9401ms (-7%)
Fresh install (cold cache) 6848ms -> 7028ms (+3%) 5929ms -> 5295ms (-11%) 8572ms -> 8086ms (-6%) 18438ms -> 16292ms (-12%) 10978ms -> 9718ms (-11%) 15915ms -> 12514ms (-21%)
npm install && npm run test 12ms -> 11ms (-8%) 53ms -> 42ms (-21%) 73ms -> 77ms (+5%) 378ms -> 328ms (-13%) 719ms -> 692ms (-4%) 1564ms -> 1197ms (-23%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.


Generated by the bench-refresh workflow.

@greptile-apps

greptile-apps Bot commented May 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR refreshes benchmark data from aube 1.16.0 to 1.16.1, also picking up deno 2.8.1 and pnpm 11.4.0, and updates the BENCH_RATIOS marketing copy in the README accordingly.

  • benchmarks/results.json: all timestamps, versions, integer values, and full stats objects (mean, stddev, min, max) updated; internal consistency between the ms values and the seconds-based stats.mean checks out across every manager and scenario.
  • README.md: pnpm warm-install ratio correctly adjusted from 7× to 5×, repeat-test ratio from 32× to 30×; the Bun warm-install ratio is retained at "3×" even though the new measurement (1408 / 552 ≈ 2.55×) no longer rounds to 3×.

Confidence Score: 4/5

Safe to merge after deciding whether to round the Bun warm-install ratio to 2.5× or keep 3×; benchmark JSON data is internally consistent.

The only discrepancy is that the README still claims "about 3× faster than Bun" for warm installs when the actual measured ratio has slipped to ~2.55×. Everything else — version strings, integer values, and underlying stats — cross-checks correctly.

README.md — specifically the BENCH_RATIOS Bun warm-install ratio claim.

Important Files Changed

Filename Overview
README.md BENCH_RATIOS block updated; pnpm warm ratio correctly downgraded from 7x to 5x and repeat-test ratio from 32x to 30x, but the Bun warm ratio retained at "3x" despite the new value being ~2.55x
benchmarks/results.json All three benchmark suites refreshed for aube 1.16.1 / deno 2.8.1 / pnpm 11.4.0; integer values fields are consistent with the corresponding stats.mean entries across every tool and scenario

Reviews (1): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

Comment thread README.md

<!-- BENCH_RATIOS:START -->
**[Fast installs](https://aube.en.dev/benchmarks).** Warm installs are about 7x faster than pnpm and about 3x faster than Bun in the current benchmarks. Repeat test commands run up to 32x faster than pnpm and up to 4x faster than Bun.
**[Fast installs](https://aube.en.dev/benchmarks).** Warm installs are about 5x faster than pnpm and about 3x faster than Bun in the current benchmarks. Repeat test commands run up to 30x faster than pnpm and up to 4x faster than Bun.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The new warm-install ratio against Bun is 1408 / 552 ≈ 2.55×, which rounds more naturally to "about 2.5×" or "almost 3×". The previous run (1384 / 420 ≈ 3.3×) genuinely rounded to "3×", but keeping "about 3×" at 2.55× overstates the advantage by roughly 18%.

Suggested change
**[Fast installs](https://aube.en.dev/benchmarks).** Warm installs are about 5x faster than pnpm and about 3x faster than Bun in the current benchmarks. Repeat test commands run up to 30x faster than pnpm and up to 4x faster than Bun.
**[Fast installs](https://aube.en.dev/benchmarks).** Warm installs are about 5x faster than pnpm and about 2.5x faster than Bun in the current benchmarks. Repeat test commands run up to 30x faster than pnpm and up to 4x faster than Bun.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@jdx jdx merged commit df34e62 into main May 29, 2026
17 checks passed
@jdx jdx deleted the bench-refresh branch May 29, 2026 14:29
@greptile-apps greptile-apps Bot mentioned this pull request May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants