Skip to content

perf(pacquet/package-manager): treat unsupported settings as no-opinion in optimistic-repeat-install#12005

Merged
zkochan merged 1 commit into
mainfrom
perf/11992
May 28, 2026
Merged

perf(pacquet/package-manager): treat unsupported settings as no-opinion in optimistic-repeat-install#12005
zkochan merged 1 commit into
mainfrom
perf/11992

Conversation

@zkochan

@zkochan zkochan commented May 27, 2026

Copy link
Copy Markdown
Member

Summary

state.settings == current in settings_match compared every field on WorkspaceStateSettings, including settings pacquet doesn't yet implement (peersSuffixMaxLength, dedupeDirectDeps, packageExtensions, …). Pnpm's defaults populate those fields when it writes the state; pacquet's current_settings leaves them None. So any cross-package-manager scenario — pnpm wrote the workspace-state file, pacquet runs next — rejected the optimistic fast path and fell into the slower frozen no-op short-circuit on iter 1. That's the 9.09× babylon × lockfile+node_modules regression in #11992.

This PR switches settings_match to compare only the fields current_settings actively writes, mirroring pnpm's Object.entries(workspaceState.settings) walk semantically. It also special-cases allowBuilds so pnpm's Some({}) and pacquet's None match (both mean "no opinion"), mirroring pnpm's opts.allowBuilds ?? {} coercion on the read side.

What's not compared, and why

Each entry below has a matching line in settings_match's skip-list comment. As each one is ported end-to-end (yaml plumbing → Config field → real install consumer → joined into current_settings), it drops out of the skip-list and starts gating the fast path. See #12009 for the porting tracker.

Tracked at #12009 (genuinely unimplemented in pacquet today):

  • dedupeDirectDeps, dedupeInjectedDeps, dedupePeers
  • excludeLinksFromLockfile
  • injectWorkspacePackages
  • packageExtensions
  • peersSuffixMaxLength
  • preferWorkspacePackages

Hitch a ride on the same skip-list (pacquet does consume them during install, but they aren't surfaced into the workspace-state crate yet — separate short follow-up):

  • minimumReleaseAge, minimumReleaseAgeExclude, minimumReleaseAgeIgnoreMissingTime, minimumReleaseAgeStrict
  • trustPolicy, trustPolicyExclude, trustPolicyIgnoreAfter

Permanently outside the comparison:

  • catalogs — pnpm itself always ignores (ignoredSettings.add('catalogs'))
  • workspacePackagePatterns — already tracked via WorkspaceManifest.packages from pnpm-workspace.yaml

Test plan

End-to-end on vltpkg/benchmarks/fixtures/babylon, after pnpm has written the workspace-state file:

Step Before After
pacquet iter-1 (sees pnpm state) ~531 ms (frozen no-op) ~96 ms (optimistic fast path)
pacquet iter-2+ (sees own state) ~90 ms ~90 ms
pnpm itself, warm ~687 ms ~687 ms

After the fix, pacquet's iter-1 is 7× faster than pnpm's warm install on the same fixture, matching iter-2's behavior.

  • cargo nextest run -p pacquet-package-manager optimistic_repeat_install — 18/18 pass (the conservative-posture test returns_skipped_when_unported_pnpm_settings_present was replaced with returns_up_to_date_when_state_carries_unported_pnpm_settings covering every Port pnpm settings missing from pacquet #12009 setting + catalogs + workspacePackagePatterns; second new test locks in the allowBuilds empty-vs-None coercion).
  • cargo clippy --locked -p pacquet-package-manager --all-targets -- --deny warnings — clean
  • cargo fmt --check — clean
  • End-to-end repro on babylon fixture: pacquet iter-1 after pnpm wrote state hits the optimistic fast path; pnpm-shape state file is preserved (pacquet's fast path doesn't overwrite it, matching pnpm).

Closes #11992. Refs #12009.


Written by an agent (Claude Code, claude-opus-4-7).

Summary by CodeRabbit

  • Bug Fixes

    • Improved settings comparison so cached workspace state is correctly recognized as up-to-date, avoiding unnecessary reinstalls. Empty allow-lists and absent values are treated as equivalent to prevent false mismatches.
  • Tests

    • Expanded test coverage for settings-compatibility edge cases, including unconsumed/external settings and allow-list normalization to validate the fast-path behavior.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 27, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c523fea3-8392-403d-a369-f15caff708b5

📥 Commits

Reviewing files that changed from the base of the PR and between f0d3d5b and 90d3a48.

📒 Files selected for processing (2)
  • pacquet/crates/package-manager/src/optimistic_repeat_install.rs
  • pacquet/crates/package-manager/src/optimistic_repeat_install/tests.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • pacquet/crates/package-manager/src/optimistic_repeat_install.rs
  • pacquet/crates/package-manager/src/optimistic_repeat_install/tests.rs

📝 Walkthrough

Walkthrough

The settings comparison now compares only the fields produced by current_settings instead of whole-struct equality, and normalizes allow_builds (treating None and empty maps as equivalent). Tests were updated to assert unported pnpm-only fields are ignored and that empty-map/None equivalence yields Decision::UpToDate.

Changes

Settings comparison and drift detection

Layer / File(s) Summary
Field-by-field settings comparison logic
pacquet/crates/package-manager/src/optimistic_repeat_install.rs
settings_match replaces whole-struct equality with explicit per-field checks constrained to fields current_settings produces; adds allow_builds_match to treat None and empty maps as equivalent, otherwise requiring exact map equality.
Settings matching test coverage
pacquet/crates/package-manager/src/optimistic_repeat_install/tests.rs
Tests updated: one populates many pnpm-only/unported WorkspaceStateSettings fields and asserts Decision::UpToDate; another verifies stored Some(empty_map) vs computed None for allow_builds are treated as equivalent and return Decision::UpToDate.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers

  • anonrig

Poem

A rabbit checks fields, not the whole,
Quietly matching bit by bit,
Empty maps wink back as None,
Unported keys left unspun,
UpToDate fast-path—hop, commit! 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: treating unsupported pnpm settings as no-opinion in optimistic-repeat-install to avoid performance regressions.
Linked Issues check ✅ Passed The PR successfully implements the fix for issue #11992 by narrowing settings comparison to only consumed fields and handling allow_builds equivalence, enabling optimistic-repeat-install fast path.
Out of Scope Changes check ✅ Passed All changes are focused on the optimistic_repeat_install settings comparison logic and its tests, directly addressing the linked issue with no extraneous modifications.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch perf/11992

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

Review Summary by Qodo

Add diagnostic logging for optimistic repeat install rejections

✨ Enhancement

Grey Divider

Walkthroughs

Description
• Add comprehensive debug logging to check_optimistic_repeat_install function
  - Logs specific rejection reasons: settings drift, missing modules, stale manifests
  - Names the camelCase setting that differs (e.g., allowBuilds, peersSuffixMaxLength)
  - Reports missing directory paths and manifest mtimes for diagnostics
• Refactor decision handling in Install::run to match all decision variants
  - Logs Skipped decisions via tracing::debug! for benchmark analysis
  - Enables TRACE=pacquet::optimistic_repeat_install=debug to surface rejection gates
• Enhance helper functions to return first failing item instead of boolean
  - first_setting_drift returns setting name causing mismatch
  - missing_modules_dir returns path of absent directory
  - first_newer_manifest returns path of stale manifest
• All 17 existing tests pass; behavior and error codes unchanged
Diagram
flowchart LR
  A["check_optimistic_repeat_install"] -->|"settings drift"| B["first_setting_drift"]
  A -->|"missing modules"| C["missing_modules_dir"]
  A -->|"stale manifest"| D["first_newer_manifest"]
  A -->|"state load error"| E["tracing::debug!"]
  B -->|"setting_name"| F["tracing::debug!"]
  C -->|"missing_dir path"| F
  D -->|"manifest path"| F
  F -->|"Decision::Skipped"| G["Install::run logs reason"]

Loading

Grey Divider

File Changes

1. pacquet/crates/package-manager/src/install.rs ✨ Enhancement +30/-11

Log optimistic repeat install skip reasons

• Refactor check_optimistic_repeat_install call to match both UpToDate and Skipped decision
 variants
• Add tracing::debug! logging for Skipped decisions with rejection reason
• Extract decision into variable for pattern matching instead of inline conditional

pacquet/crates/package-manager/src/install.rs


2. pacquet/crates/package-manager/src/optimistic_repeat_install.rs ✨ Enhancement +205/-38

Wire debug logging for all optimistic repeat install gates

• Add tracing::debug! events for workspace state load failures and missing state file
• Replace settings_match boolean with first_setting_drift that returns camelCase setting name
• Replace modules_dirs_present boolean with missing_modules_dir that returns missing path
• Replace manifests_unchanged_since boolean with first_newer_manifest that returns stale
 manifest path
• Add detailed debug logging at each rejection branch with specific diagnostic values
• Enhance project_structure_matches to log count delta, missing entries, and name/version
 mismatches

pacquet/crates/package-manager/src/optimistic_repeat_install.rs


Grey Divider

Qodo Logo

@github-actions

github-actions Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Micro-Benchmark Results

Linux

group                          main                                   pr
-----                          ----                                   --
tarball/download_dependency    1.03      8.3±0.60ms   524.6 KB/sec    1.00      8.1±0.31ms   538.5 KB/sec

@codecov-commenter

codecov-commenter commented May 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.15%. Comparing base (0cefccf) to head (90d3a48).

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #12005   +/-   ##
=======================================
  Coverage   88.14%   88.15%           
=======================================
  Files         228      228           
  Lines       28523    28547   +24     
=======================================
+ Hits        25142    25165   +23     
- Misses       3381     3382    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Integrated-Benchmark Report (Linux)

Scenario: Isolated linker: fresh restore, cold cache + cold store

Command Mean [s] Min [s] Max [s] Relative
pacquet@HEAD 1.948 ± 0.079 1.878 2.132 1.01 ± 0.05
pacquet@main 1.923 ± 0.069 1.870 2.077 1.00
BENCHMARK_REPORT.json
{
  "results": [
    {
      "command": "pacquet@HEAD",
      "mean": 1.94771179524,
      "stddev": 0.07859864824905877,
      "median": 1.92524792364,
      "user": 2.6343886999999997,
      "system": 3.3739079599999995,
      "min": 1.87842708064,
      "max": 2.13156535764,
      "times": [
        1.92239752364,
        1.87842708064,
        1.8883684696399998,
        1.88149119564,
        1.96705957164,
        1.89338435664,
        2.13156535764,
        1.97573479764,
        2.01059127564,
        1.92809832364
      ]
    },
    {
      "command": "pacquet@main",
      "mean": 1.9229670344399998,
      "stddev": 0.06899595435424474,
      "median": 1.89669476014,
      "user": 2.7607906000000004,
      "system": 3.34948596,
      "min": 1.8703543946399999,
      "max": 2.0770909096400003,
      "times": [
        2.0770909096400003,
        1.88570664564,
        1.89336455364,
        1.90247817564,
        2.0232290436400002,
        1.9038754706399998,
        1.89454231564,
        1.8703543946399999,
        1.8801816306399999,
        1.89884720464
      ]
    }
  ]
}

Scenario: Isolated linker: fresh restore, hot cache + hot store

Command Mean [ms] Min [ms] Max [ms] Relative
pacquet@HEAD 627.8 ± 38.4 607.9 735.8 1.00
pacquet@main 639.3 ± 27.8 615.1 708.9 1.02 ± 0.08
BENCHMARK_REPORT.json
{
  "results": [
    {
      "command": "pacquet@HEAD",
      "mean": 0.6277924196400001,
      "stddev": 0.038447326338102446,
      "median": 0.6154318489400001,
      "user": 0.34941481999999996,
      "system": 1.32561286,
      "min": 0.60787272694,
      "max": 0.7358377059400001,
      "times": [
        0.7358377059400001,
        0.6290754979400001,
        0.6157460649400001,
        0.60823736394,
        0.6139996519400001,
        0.61320303594,
        0.6193649289400001,
        0.6194695869400001,
        0.60787272694,
        0.6151176329400001
      ]
    },
    {
      "command": "pacquet@main",
      "mean": 0.6392799328400001,
      "stddev": 0.027832568260846242,
      "median": 0.62888314844,
      "user": 0.37956112000000003,
      "system": 1.3299999599999999,
      "min": 0.61510265894,
      "max": 0.7088502079400001,
      "times": [
        0.7088502079400001,
        0.62802468194,
        0.6319028349400001,
        0.61510265894,
        0.6612354139400001,
        0.6467627829400001,
        0.6222146069400001,
        0.62248006594,
        0.62974161494,
        0.62648445994
      ]
    }
  ]
}

Scenario: Isolated linker: fresh install, cold cache + cold store

Command Mean [s] Min [s] Max [s] Relative
pacquet@HEAD 2.222 ± 0.034 2.165 2.284 1.01 ± 0.02
pacquet@main 2.205 ± 0.043 2.142 2.283 1.00
BENCHMARK_REPORT.json
{
  "results": [
    {
      "command": "pacquet@HEAD",
      "mean": 2.2216002002399997,
      "stddev": 0.03403456122241408,
      "median": 2.22481444884,
      "user": 3.72304192,
      "system": 3.13585354,
      "min": 2.16531780234,
      "max": 2.28438916834,
      "times": [
        2.18669146634,
        2.21091009434,
        2.19531303134,
        2.22354602034,
        2.2496239513400003,
        2.22608287734,
        2.23571670734,
        2.2384108833400003,
        2.16531780234,
        2.28438916834
      ]
    },
    {
      "command": "pacquet@main",
      "mean": 2.20457087144,
      "stddev": 0.04277838614055283,
      "median": 2.20076785084,
      "user": 3.79416442,
      "system": 3.07185014,
      "min": 2.14177991834,
      "max": 2.28343286634,
      "times": [
        2.28343286634,
        2.24553734634,
        2.1893974533400002,
        2.23791242234,
        2.20203786034,
        2.14840764634,
        2.19949784134,
        2.20309509034,
        2.14177991834,
        2.19461026934
      ]
    }
  ]
}

Scenario: Isolated linker: fresh install, hot cache + hot store

Command Mean [s] Min [s] Max [s] Relative
pacquet@HEAD 1.378 ± 0.022 1.345 1.409 1.00
pacquet@main 1.418 ± 0.029 1.349 1.454 1.03 ± 0.03
BENCHMARK_REPORT.json
{
  "results": [
    {
      "command": "pacquet@HEAD",
      "mean": 1.3782522250000002,
      "stddev": 0.02223022254620925,
      "median": 1.3833297729,
      "user": 1.6360304399999996,
      "system": 1.7744545799999998,
      "min": 1.3450235399000001,
      "max": 1.4085520889,
      "times": [
        1.3967103959,
        1.3450235399000001,
        1.3457174179,
        1.3898267709,
        1.3745033199,
        1.3581840999,
        1.3948482189,
        1.3923236229,
        1.3768327749,
        1.4085520889
      ]
    },
    {
      "command": "pacquet@main",
      "mean": 1.4176806976,
      "stddev": 0.02943639176492092,
      "median": 1.4285053999000001,
      "user": 1.6960483399999997,
      "system": 1.78889248,
      "min": 1.3487996439,
      "max": 1.4538308469,
      "times": [
        1.4319838049,
        1.4538308469,
        1.4343444269,
        1.4294731979000002,
        1.4024665329000001,
        1.4387409619,
        1.4014580269,
        1.4081719319000001,
        1.4275376019000001,
        1.3487996439
      ]
    }
  ]
}

@github-actions

github-actions Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branchpr/12005
Testbedpacquet
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
milliseconds (ms)
(Result Δ%)
Upper Boundary
milliseconds (ms)
(Limit %)
isolated-linker.fresh-install.cold-cache.cold-store📈 view plot
🚷 view threshold
2,221.60 ms
(-6.59%)Baseline: 2,378.43 ms
2,854.12 ms
(77.84%)
isolated-linker.fresh-install.hot-cache.hot-store📈 view plot
🚷 view threshold
1,378.25 ms
(-9.31%)Baseline: 1,519.75 ms
1,823.70 ms
(75.57%)
isolated-linker.fresh-restore.cold-cache.cold-store📈 view plot
🚷 view threshold
1,947.71 ms
(-4.13%)Baseline: 2,031.57 ms
2,437.88 ms
(79.89%)
isolated-linker.fresh-restore.hot-cache.hot-store📈 view plot
🚷 view threshold
627.79 ms
(-5.21%)Baseline: 662.32 ms
794.78 ms
(78.99%)
🐰 View full continuous benchmarking report in Bencher

@zkochan zkochan changed the title perf(pacquet/package-manager): log why optimisticRepeatInstall skipped perf(pacquet/package-manager): treat unsupported settings as no-opinion in optimistic-repeat-install May 27, 2026
…on in optimistic-repeat-install

The `state.settings == current` check compared every field on
`WorkspaceStateSettings`, including settings pacquet doesn't yet
implement (`peersSuffixMaxLength`, `dedupeDirectDeps`, `packageExtensions`,
…). Pnpm's defaults populate those fields when it writes the state,
while pacquet's `current_settings` leaves them `None`, so any
cross-package-manager scenario (pnpm wrote the workspace state,
pacquet runs next) rejected the fast path on iter 1 and fell into
the slower frozen no-op short-circuit. That's the 9.09× `babylon ×
lockfile+node_modules` regression in #11992.

Switch `settings_match` to compare only the fields `current_settings`
actively writes. Mirrors pnpm's
[`Object.entries(workspaceState.settings)`](https://github.com/pnpm/pnpm/blob/72d997cc34/deps/status/src/checkDepsStatus.ts) walk: pnpm iterates the
settings present in the state, which by symmetry are the settings
the writer cared about. Pacquet's `current_settings` is the symmetric
"settings pacquet cares about" set, so comparing against it is the
natural way to honour the same contract.

Special-case `allowBuilds`: pnpm writes `Some({})` for an empty
allow-list, pacquet writes `None` — both mean "no opinion," so treat
them as equivalent (mirroring pnpm's `opts.allowBuilds ?? {}` coercion
on the read side).

The settings *not* compared are tracked at #12009 with a
matching skip-list comment in code; each one drops out of the skip
list as it's ported end-to-end (yaml plumbing → `Config` field →
real install consumer → joined into `current_settings`). Two extra
groups (`minimumReleaseAge*`, `trustPolicy*`) hitch a ride: pacquet
*does* consume them during install, but they aren't surfaced through
the workspace-state crate yet. And two stay outside the comparison
permanently — `catalogs` (pnpm itself always ignores) and
`workspacePackagePatterns` (covered via `WorkspaceManifest.packages`
from `pnpm-workspace.yaml`).

End-to-end on `vltpkg/benchmarks/fixtures/babylon`, after pnpm has
written the workspace-state file:

  - before fix: pacquet iter-1 falls through to the frozen no-op
    short-circuit (~531 ms locally, ~9× pnpm on vlt CI).
  - after fix:  pacquet iter-1 fires the optimistic fast path
    (~96 ms locally, faster than pnpm's own warm fast path at
    ~687 ms — same 7-8× lead as iter-2+).

The `returns_skipped_when_unported_pnpm_settings_present` test that
locked in the conservative posture is replaced with
`returns_up_to_date_when_state_carries_unported_pnpm_settings`,
which exercises every #12009 setting plus `catalogs` and
`workspacePackagePatterns` together. A second new test,
`returns_up_to_date_when_state_has_empty_allow_builds_and_current_has_none`,
locks in the `allowBuilds` empty-vs-`None` coercion.

Closes #11992.
Refs: #12009
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(pacquet/install): optimistic_repeat_install fails to fire under vlt's lockfile+node_modules bench

2 participants