This repository was archived by the owner on May 14, 2026. It is now read-only.
bench(integrated-benchmark): replace 26-snapshot fixture with 1352-snapshot one#262
Merged
Conversation
…apshot one The previous `alotta-modules` fixture had 7 direct deps and 26 snapshots, small enough that a frozen-lockfile install completed in ~50 ms. At that scale the wall-time is dominated by per-process overhead (binary startup, lockfile parse), so scaling wins in the install pipeline — shared SQLite connections, import-method changes, symlink-pass parallelism — don't show up. Swap in a realistic 1352-snapshot lockfile of `alotta-modules` (389 KB vs 13 KB) so the benchmark reflects a workload where the install pipeline actually matters. First CI run will need to repopulate the cached verdaccio storage; subsequent runs are served from the existing cache step.
Micro-Benchmark ResultsLinux |
`allowBuilds: {core-js: false, es5-ext: false}` silences pnpm v11's
`ERR_PNPM_IGNORED_BUILDS` warnings for two packages whose postinstalls
would fire in this 1352-snapshot fixture. Pacquet's benchmark-side
`.npmrc` already sets `ignore-scripts=true` so nothing actually runs,
but pnpm still prints to stderr about the ignored builds and that
noise can reach hyperfine.
Stage the new file the same way `package.json` is staged — embedded
via `include_str!` for the default fixture, copied from `--fixture-dir`
when set. The `--fixture-dir` copy is gated on the file's existence
so existing fixture directories that predate this change don't start
erroring out.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #262 +/- ##
==========================================
- Coverage 90.04% 89.63% -0.41%
==========================================
Files 58 58
Lines 4471 4527 +56
==========================================
+ Hits 4026 4058 +32
- Misses 445 469 +24 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Integrated-Benchmark Report (Linux)Scenario: Frozen Lockfile
BENCHMARK_REPORT.json{
"results": [
{
"command": "pacquet@HEAD",
"mean": 1.09545879052,
"stddev": 0.009108090554857748,
"median": 1.09625982232,
"user": 0.40491678000000003,
"system": 2.5872452399999997,
"min": 1.08182751782,
"max": 1.10723339882,
"times": [
1.08182751782,
1.09655198682,
1.10723339882,
1.0836143638200002,
1.10507124582,
1.1021163448200002,
1.10337615782,
1.0874212228200002,
1.0959676578200002,
1.09140800882
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
]
},
{
"command": "pacquet@main",
"mean": 1.0938490233200002,
"stddev": 0.007353889992471131,
"median": 1.09376099332,
"user": 0.41615018,
"system": 2.5719799400000003,
"min": 1.08208248382,
"max": 1.1041609298200001,
"times": [
1.0908739118200002,
1.0887367758200002,
1.09520853482,
1.1023697538200001,
1.0923134518200002,
1.08208248382,
1.1041609298200001,
1.10153707582,
1.0956018578200002,
1.08560545782
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
]
},
{
"command": "pnpm",
"mean": 2.55005058402,
"stddev": 0.025888052063577947,
"median": 2.5636800008200002,
"user": 3.16584818,
"system": 2.2522639399999997,
"min": 2.5119818078200002,
"max": 2.58461474082,
"times": [
2.51874097082,
2.52996399582,
2.56645358782,
2.58461474082,
2.5631142058200003,
2.5642457958200002,
2.56477127382,
2.52443437782,
2.57218508382,
2.5119818078200002
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
]
}
]
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The current `alotta-modules` fixture has 7 direct deps and 26 snapshots — a frozen-lockfile install runs in ~50 ms, so wall-time is dominated by per-process overhead (binary startup, lockfile parse) rather than the install pipeline. Scaling wins — shared SQLite connections (#261), import-method changes, symlink-pass parallelism — don't show up because the work they optimize barely exists at this size.
Swap in a 1352-snapshot lockfile of `alotta-modules` (389 KB vs 13 KB) so the benchmark reflects a realistic install.
Test plan