Conversation
* fix(ci): micro-benchmark (again) * fix(benchmark): fixture path
* ci: clippy and doc on all platforms * ci: fewer jobs * chore(cargo): remove unnecessary aliases * chore(just): add --locked to lint * fix(clippy): windows * fix(clippy): windows * ci(clippy): add --locked * fix(clippy): windows * fix(ci): windows
…o `pacquet@{revision}` and `pnpm` (#156)
## Summary * Move code that handles package managing to its own crate named `pacquet_package_manager`. * Convert functions with multiple arguments into structs. * Make `pacquet add` reuses the algorithm from `pacquet install`. * Achieved by having `pacquet add` creates an in-memory `package.json` and pass it to the `Install` subroutine. * Convert the error-prone tests in `crates/cli` into proper integration tests that invoke the `pacquet` binary. * Tests become shorter and more readable. * Solve race condition issues with `cargo test`. * All code with `set_current_dir` have been replaced by either integration tests or dependency injection. * Solve race condition issues with `cargo test`. * `thiserror` has been completely removed. `derive_more` is used in its stead. * `thiserror` cannot handle generic trait bound. * `derive_more` is already used to generate custom string types in the lockfile. * one less macro crate leads to slightly faster compile time. * `pacquet_package_json::PackageJson` has been renamed to `pacquet_package_manifest::PackageManifest`. * Other minor changes.
## Limitations * **Statistical outliers:** Caused by other jobs still running in parallel. The current workaround is to re-run the job. To completely fix this however, this benchmark must be set to be after all other jobs have been completed, which requires touching other workflows. * **No windows:** Executing this on Windows CI is just too hard: cannot use powershell because of execution policy, and cannot use bash because it points to WSL which causes error. * **No macOS:** The benchmark values are too chaotic and unreliable. * **No clean install yet:** It's too slow.
* ci: add taplo * style: use taplo to format * style: allow only 1 blank line * ci: fix binary name
* feat(fs): make_file_executable * refactor: re-use make_file_executable
Resolves pnpm/pacquet#165 **Other changes:** * A new crate named `pacquet-store-dir` has been created. * The field `store-dir` in `Npmrc` has been changed from `PathBuf` to `StoreDir`. * The function `write_sync` in `pacquet-cafs` has been replaced by 2 functions (pnpm/pacquet@8f98e73). * The `pacquet-cafs` crate has been merged into `pacquet-store-dir` (pnpm/pacquet@db673ef). * The `pacquet-tarball` crate now also writes index files (3b6e68a48b0711a2666bbe4964b9fd485a9842d3..8f98e730c1c35e2371b1d5ebd7f9e3f86b99104e). * The command `pacquet prune` now panics on `todo!()` for being incomplete (pnpm/pacquet@c8526d0).
* refactor: convert 2 functions into a struct * refactor: symmetry * refactor: rename a constructor * docs: consistent grammar
* ci(benchmark): embed the JSON report * fix: quoting
* refactor: parse integrity eagerly The parsing of integrity has been moved from tarball to serde * docs: clarify * docs: remove the 3rd option * refactor: remove unnecessary `pipe_ref`
* refactor: only use cache when required * refactor: remove unnecessary Arc * refactor: clarify "in-memory" part of the cache
* feat(benchmark): allow overriding lockfile * test: fix snapshot * test: fix snapshot * test: filter out index files temporarily * test: fix snapshot
* test: fix snapshot * test: filter out index files temporarily * chore: install registry-mock * chore(just): add pnpm to test * feat: create registry-mock crate * feat(registry-mock): expose get_or_init instead * refactor: local static * feat(testing-utils): add_mocked_registry * refactor: rename new to init * fix: enable_all * refactor: use * feat: add ending / * fix: must be "localhost" * feat(registry-mock): use portpicker * feat(registry-mock): share a single instance * feat(testing-utils): stop skipping index files * test(cli/install): use registry-mock * fix: forgot to save * fix: forgot increment * fix: used to wrong syntax * feat(registry-mock): use advisory-lock * fix: save anchor before early exit * fix: drop before delete * tmp: stored * refactor: remove unused import * fix: truncate the file * feat: use try_lock * feat(registry-mock): separate lock from anchor * refactor: use RAII pattern * fix(registry-mock): user_count * refactor: rename user_count to ref_count * fix: kill groups * fix(registry-mock): ref_count * chore(git): revert broken fix This reverts commit 8c0ba7bafd7670dd6580a142ccd7dea5809ed782. * fix(registry-mock): use sysinfo to kill * fix(registry-mock): kill algorithm * feat(registry-mock): recurse regardless * feat(registry-mock): use reverse recursion instead * refactor: reduce complexity * refactor: remove unnecessary mod declarations * refactor: marks some struct as must_use * test(cli/add): use registry-mock * test(pnpm-compatibility): use registry-mock * refactor: remove unused items * docs: correction * chore(license): allow MPL-2.0 * chore(license): allow Unlicense * fix(registry-mock): failure on few threads * chore: move pnpm install to just * ci: cache pnpm * fix(test): kill leftovers * fix(test): allow delete to fail * fix(ci): Install just * fix(test): don't kill leftovers This reverts commit ef78d8c54cee691d93fa8e5c169141a0098047f2. * refactor: move some types to their own mods * refactor: move port_to_url * refactor: make listen lazy * refactor: rename `listen` to `url` * refactor: use qualified path * refactor: store port as u16 * feat(registry-mock): PreparedRegistryInfo * refactor: remove unnecessary async * refactor: move init code * refactor: define lib * feat(registry-mock): bin * refactor: call just test * ci: prepare a server * feat(registry-mock): unique prepared * docs(registry-mock): cli * feat(registry-mock/cli): locations of log files * docs(mocked-registry): important items * docs(registry-mock): ignore bin * style: taplo fmt * docs(readme): instruction to run test * refactor: move registry-mock to tasks * ci(codecov): fix * feat(registry-mock): bump max_retries
* refactor: reuse registry-mock in benchmark * refactor: move executor to verify * feat(benchmark): install dependencies * ci(benchmark): fix * ci(benchmark): use just * refactor: reduce lines
Bumps [clap](https://github.com/clap-rs/clap) from 4.4.4 to 4.4.8. - [Release notes](https://github.com/clap-rs/clap/releases) - [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md) - [Commits](clap-rs/clap@clap_complete-v4.4.4...v4.4.8) --- updated-dependencies: - dependency-name: clap dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.37 to 0.1.40. - [Release notes](https://github.com/tokio-rs/tracing/releases) - [Commits](tokio-rs/tracing@tracing-0.1.37...tracing-0.1.40) --- updated-dependencies: - dependency-name: tracing dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.48 to 1.0.50. - [Release notes](https://github.com/dtolnay/thiserror/releases) - [Commits](dtolnay/thiserror@1.0.48...1.0.50) --- updated-dependencies: - dependency-name: thiserror dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.32.0 to 1.34.0. - [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md) - [Commits](mitsuhiko/insta@1.32.0...1.34.0) --- updated-dependencies: - dependency-name: insta dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.8.0 to 3.8.1. - [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md) - [Commits](https://github.com/Stebalien/tempfile/commits) --- updated-dependencies: - dependency-name: tempfile dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
…k.yaml (#434 slice 6) (#495) Pacquet wrote the raw wanted lockfile as the current lockfile. Upstream pnpm writes a filtered version with optional + skipped subtrees pruned and `include` flags applied, so the next install diffs against what was actually materialized rather than the resolver's full ambition. Without the prune, dropped snapshots (slice 1 installability, slice 4 fetch failures, slice 5 `--no-optional`) were claimed present in the current lockfile and the follow-up install would skip work that should have run. Ports <https://github.com/pnpm/pnpm/blob/94240bc046/lockfile/filtering/src/filterLockfileByImportersAndEngine.ts#L46-L94> → <https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L687-L695>. Pacquet-specific simplification: instead of re-running the engine + supportedArchitectures + skipped checks at filter time, `filter_lockfile_for_current` reuses the `SkippedSnapshots` set already produced by the install pipeline. Its three subsets (`installability`, `fetch_failed`, `optional_excluded`) are the exact set upstream's filter would drop — same observable result, no duplicated walk. ## Changes - **`pacquet-lockfile`**: `Clone` derives on `Lockfile`, `LockfileSettings`, `ProjectSnapshot`, `SnapshotEntry`, `PackageMetadata`, `PeerDependencyMeta`, `ResolvedDependencySpec`. Needed to build a filtered lockfile by clone-and-mutate. - **`pacquet-package-manager/src/current_lockfile.rs`** (new): `filter_lockfile_for_current(lockfile, included, skipped) -> Lockfile`. Three-phase filter: 1. BFS the snapshot graph from filtered importer roots, skipping keys in `skipped`. Produces the reachable set. 2. Per-importer: clear dep maps whose `include` flag is false; trim `optional_dependencies` to entries whose target survived. 3. `snapshots:` / `packages:` filtered to the reachable set (packages key off `without_peer()` so peer-variant survivors keep their shared metadata row). - **`install.rs`**: replace the raw `save_current_to_virtual_store_dir` call with `filter_lockfile_for_current(...).save_current_to_virtual_store_dir(...)`. ## Tests 8 unit tests covering each filter behavior: - `skipped_snapshot_pruned_from_snapshots_and_importer_optional` - `include_optional_false_clears_importer_section` - `transitive_under_skipped_snapshot_is_pruned` - `snapshot_reachable_via_kept_path_survives` (mirrors upstream's `:712` shared-dep case at the filter level) - `packages_filtered_to_surviving_metadata_keys` (peer-variant metadata sharing) - `link_optional_entries_survive_post_filter` (workspace link: deps don't get post-filtered) - `empty_skipped_and_full_include_is_identity_for_reachables` (baseline) - `orphan_snapshots_are_pruned` (snapshots unreachable from any importer get dropped) Test-the-test verified by breaking the BFS walker — two tests fail. ## Out of scope - Hoisted-linker current-lockfile variant (`:633`) — pacquet's hoisted node-linker isn't fully wired through the lockfile-write path yet; tracked separately under #438. - `pnpm install --filter` slicing — pacquet has no `--filter` yet. Closes #493.
…ice 4d) (#494) * feat(package-manager): prev_graph diff from current lockfile (#438 slice 4d) `lockfile_to_hoisted_dep_graph` now takes an optional `current_lockfile: Option<&Lockfile>` and populates `result.prev_graph` from a second walk over that lockfile. Ports upstream's wrapper at installing/deps-restorer/src/lockfileToHoistedDepGraph.ts. When the current lockfile is `Some(lf)` with a non-empty `packages:` map, the second walk runs with `force: true` and `skipped: BTreeSet::new()`. Force matters: an orphan that landed under the previous install but would now fail installability (e.g., the host changed platforms) still surfaces in `prev_graph` so Slice 5's linker can find and rimraf the stale directory. Empty skipped matters: the previous install's own filter set is unrelated to "which directories still exist on disk." API change - The public `lockfile_to_hoisted_dep_graph(lockfile, opts)` signature gains a middle `current_lockfile: Option<&Lockfile>` argument. Single existing caller (the tests) updated to pass `None`. Splits the previous body into a private `build_dep_graph` helper that the wrapper calls once or twice depending on whether a current lockfile is present. prev_graph shape - `None` when no current lockfile is supplied, or when the supplied lockfile has no `packages:` map (a brand-new install in progress). Mirrors upstream's `prevGraph = {}` fallback — pacquet uses `None` rather than an empty map, but the linker treats both the same. - `Some(graph)` otherwise, keyed by directory just like the wanted-lockfile graph. Tests - `prev_graph_none_when_current_lockfile_absent` — no current lockfile → `prev_graph` is `None`, wanted graph still produced. - `prev_graph_none_when_current_lockfile_has_no_packages` — current lockfile with `packages: None` → `prev_graph` is `None`. - `prev_graph_contains_orphan_from_current_only_lockfile` — package in current but not wanted appears in `prev_graph`, not in `graph`. - `prev_graph_includes_orphan_even_when_now_incompatible` — darwin-only orphan in the current lockfile, host is linux, wanted lockfile is empty: `prev_graph` still contains it (proves `force: true` overrides the installability check), and the wanted-walk's `skipped` stays empty (proves the prev-walk's skipped set is independent). All 4 new tests pass alongside the 15 walker tests from 4a-4c. * fix(package-manager): collapse empty current packages to None for prev_graph (#494 review) `current.packages.is_some()` matched `Some(empty_map)` too, causing 4d to do an unnecessary second walk and return `Some(empty_graph)` for a case the doc-comment described as `None`. Tighten the guard to require a non-empty `packages` map. Pacquet uses `Option<DependenciesGraph>` for `prev_graph` (upstream uses an always-present `DependenciesGraph` with `{}` for the no-current case). The no-packages and empty-packages cases both produce the same observable behavior — "no orphans to consider" — so they should share one representation rather than be inconsistent. The doc-comment's stated contract was `None`; the code now matches. Added `prev_graph_none_when_current_lockfile_has_empty_packages` to pin the empty-map case. Caught by Coderabbit on #494. * style(package-manager): rustfmt the 4d follow-up
…#437 slice D2) (#496) Construct the per-archive ignore filter at the install dispatcher. For unscoped `node` the filter matches upstream's [`NODE_EXTRAS_IGNORE_PATTERN`](https://github.com/pnpm/pnpm/blob/94240bc046/engine/runtime/node-resolver/src/index.ts), which strips bundled `npm` / `corepack` from the Node.js runtime archive during the CAS write — pnpm (and pacquet) install pnpm itself as the package manager, so the bundled tooling is dead weight and would shadow the user's pnpm via `node_modules/.bin/`. Wiring matches upstream's [`archiveFilters: { node: NODE_EXTRAS_IGNORE_PATTERN }`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/client/src/index.ts): the per-package-name table is keyed by `pkg.name`, so `@foo/node` and other packages keep `None` and the full archive lands unfiltered. Pacquet uses a hand-coded matcher rather than the upstream regex so `pacquet-tarball` doesn't have to pull in a regex engine. The three branches mirror the regex alternation exactly: 1. `^(?:lib/)?node_modules/(?:npm|corepack)(?:/|$)` 2. `^bin/(?:npm|npx|corepack)$` 3. `^(?:npm|npx|corepack)(?:\.(?:cmd|ps1))?$` A `OnceLock` caches the `Arc<IgnoreEntryFilter>` so per-snapshot clones share one trait object. Unit tests pin every branch of the alternation plus the negative cases (e.g. `lib/node_modules/yarn/...` is *not* matched, `bin/npm.cmd` is *not* matched — the regex's `$` and arm-specific extension rules are deliberately asymmetric). Verified by temporarily collapsing the `bin/` branch to a no-op — the test fails as expected. Bin-link cmd-shims for the runtime executables and `@runtime:` substring handling in skip lists / reporter prefixes are Slice D3. End-to-end runtime install fixtures land in Slice F.
Closes #497. ## Summary Adds per-registry TLS overrides keyed by nerf-darted `.npmrc` URI, the natural follow-up to #490's top-level TLS keys. Corporate environments running a private Verdaccio (or any registry with its own self-signed cert) can now pin scoped `:cafile=…` / `:cert=…` / `:key=…` per host without disabling strict-ssl globally. Three commits, layered: - **`feat(network)`** (eff1248e): adds `RegistryTls` + `PerRegistryTls` types in `pacquet-network` plus the lookup machinery — `pick_for_url` ports pnpm's [5-step `pickSettingByUrl`](https://github.com/pnpm/pnpm/blob/94240bc046/network/fetch/src/dispatcher.ts#L338-L375) exactly (exact > nerf-dart > no-port > shorter prefix > recursive no-port retry). `ThrottledClient::for_installs` gains a third `&PerRegistryTls` parameter and pre-builds one reqwest `Client` per non-empty override. New `acquire_for_url(url: &str)` routes per-request; `acquire()` keeps the default-client behavior for callers without a URL. - **`feat(config)`** (4e69868a): `NpmrcAuth` parses the six per-registry TLS suffixes (`:ca`, `:cafile`, `:cert`, `:certfile`, `:key`, `:keyfile`) matching pnpm's `SSL_SUFFIX_RE` and applies onto `Config.tls_by_uri`. `*file` variants read from disk at parse time (silent on error); inline values get `\\n` → `\n` expansion. `:cert` and `:certfile` share the same `tls.cert` slot — last-write-wins inside one `.npmrc`. - **`refactor(tarball,registry)`** (5f9cae93): three production call sites (registry metadata + version-tag fetches, plus two tarball download paths) move from `acquire()` to `acquire_for_url(url)` so the per-registry routing actually fires. ## Parity policy Bug-for-bug with pnpm v11 ([SHA 94240bc](https://github.com/pnpm/pnpm/blob/94240bc046/config/reader/src/getNetworkConfigs.ts)): - **Field-by-field override**, not replace-all. Each scoped `ca` / `cert` / `key` overrides its top-level counterpart independently (mirroring upstream's `{ ...opts, ...sslConfig }` spread at `dispatcher.ts:143,264`). `strict_ssl` and `local_address` stay top-level-only — pnpm's regex doesn't recognize scoped versions. - **`ca` as `Option<String>`, not `Vec<String>`**: per-registry `ca` is a single string (possibly with concatenated `-----END CERTIFICATE-----` delimiters) — `reqwest::Certificate::from_pem` accepts both shapes. - **Inline `\\n` expansion only on per-registry**: pnpm applies `value.replace(/\\n/g, '\n')` to scoped values but not to top-level `ca=`. The divergence is intentional and matches upstream. - **Lax URI prefix check**: `foo:cert=…` (no `//` prefix) is accepted into the map with `uri_prefix = "foo"`. It never matches a real nerf-darted URL so the entry is dropped at lookup time, but storing it keeps byte-for-byte parsing parity with `tryParseSslKey`. ## Reviewer flags - **Per-registry clients duplicate connection pools.** Each unique override gets its own `reqwest::Client` and therefore its own connection pool. With N per-registry overrides the worker holds N+1 pools instead of one. The semaphore still bounds *concurrent in-flight requests* globally, but socket churn between registries with different TLS configs is now per-client. In practice most users have ≤2 overrides; if this becomes an issue we'd need to switch to rustls + custom certificate verifier (tracked under #499). - **`acquire_for_url` takes `&str` rather than `&Url`** so the existing `format!("{registry}{name}")` call sites don't need to round-trip through `Url::parse`. The lookup itself works on the raw string form via `nerf_dart`.
…ph nodes (#481) (#504) * feat(lockfile): PkgIdWithPatchHash newtype, replace `String` in dep-graph nodes (#481) Port upstream's [`PkgIdWithPatchHash`](https://github.com/pnpm/pnpm/blob/94240bc046/core/types/src/misc.ts) branded string (`string & { __brand: 'PkgIdWithPatchHash' }`) as a non-validating newtype in `pacquet-lockfile`, matching `CLAUDE.md`'s rule 3 for non-validating brands: `From<String>` / `From<&str>` via `derive_more`, `#[serde(transparent)]` for wire-format identity with `String`, no validating constructor. Modeled on `pacquet_modules_yaml::DepPath` — the sibling brand from the same upstream `misc.ts` file under the same rules. Replaces the plain `String` field/parameter in the two pacquet consumers CodeRabbit flagged on #478: - `DependenciesGraphNode.pkg_id_with_patch_hash` in `package-manager/src/hoisted_dep_graph.rs`. - `create_full_pkg_id`'s first parameter and the `lockfile_to_dep_graph` local in `package-manager/src/virtual_store_layout.rs`. A workspace audit (`grep "pkg_id_with_patch_hash"`) turns up no other slots typed as plain `String`. Two new tests in `pkg_id_with_patch_hash.rs` pin the contract: a serde round-trip (`#[serde(transparent)]` keeps the on-disk shape a raw string) and the non-validating-construction property (empty string and `From<&str>`/`From<String>` both work). * fix(lockfile): replace intra-doc links to pacquet-modules-yaml with plain text CI \`Doc\` job runs with \`RUSTDOCFLAGS=-D warnings\`, so the two \`[\`pacquet_modules_yaml::DepPath\`]\` intra-doc links in \`pkg_id_with_patch_hash.rs\` failed \`rustdoc::broken-intra-doc-links\`: \`pacquet-lockfile\` doesn't depend on \`pacquet-modules-yaml\` (the dependency would go the wrong way), so the rustdoc resolver can't follow the link. Switched both references to bare \`pacquet_modules_yaml::DepPath\` prose, with a note that the bare reference is intentional. The docstring still tells a reader where to find the sibling brand; it just stops resolving as a clickable link, which costs nothing practical since the type name is searchable in any IDE. Adding \`pacquet-modules-yaml\` as a dev-dep purely for a doc link would invert the natural crate ordering (lockfile is upstream of modules-yaml in the build graph) — rejected as a cosmetic fix that introduces a real architectural smell.
* feat(package-manager): linkHoistedModules linker (#438 slice 5) New module `crates/package-manager/src/link_hoisted_modules.rs` produces the on-disk `node_modules/` tree from Slice 4's `LockfileToDepGraphResult`. Ports installing/deps-restorer/src/linkHoistedModules.ts. ## Three phases 1. **Orphan removal.** Every directory in `prev_graph` but not in `graph` is silently `rimraf`'d before any insert. `try_remove_dir` swallows all errors (NotFound + PermissionDenied + EBUSY), matching upstream's `tryRemoveDir` tolerance. 2. **Per-node import.** Hierarchy walked top-down, rayon-parallel at each level. Every node goes through `import_indexed_dir` with `force: true, keep_modules_dir: true` — the hoisted-linker call shape Slice 1 was designed for. 3. **Per-`node_modules` bin link.** After a level's children are all imported, `link_direct_dep_bins` populates `<parent>/node_modules/.bin` from the just-materialized direct children. Bin link runs only after every child's subtree is done so package.json reads always see the fully-populated package. ## API shape ```rust pub fn link_hoisted_modules<R: Reporter>( opts: &LinkHoistedModulesOpts<'_>, ) -> Result<(), LinkHoistedModulesError>; ``` `LinkHoistedModulesOpts` borrows `graph`, `prev_graph`, `hierarchy`, and a `cas_paths_by_pkg_id: HashMap<String, HashMap<String, PathBuf>>` keyed by `pkg_id_with_patch_hash`. ## Decoupling from the store Upstream's linker is async and calls `storeController.fetchPackage()` inline during the walk — pacquet's is sync and takes pre-fetched CAS paths. Slice 6 (the install pipeline) is responsible for fetching every package through pacquet's existing tarball / store-dir / git-fetcher machinery before invoking the linker. This keeps the linker focused on file-system layout and reuses the proven fetch chain that the isolated path already exercises. ## Optional-dep tolerance A graph node whose `pkg_id_with_patch_hash` is missing from `cas_paths_by_pkg_id`: - If `node.optional`: silently skipped, no directory created. Mirrors upstream's `if (depNode.optional) return` on fetch failure. - Otherwise: surfaces as `LinkHoistedModulesError::MissingCasPaths { pkg_id, dir }`. ## Tests 7 real-tempdir tests in `link_hoisted_modules/tests.rs`: - `import_pass_creates_package_directory` — single-package smoke. - `orphan_directory_is_removed` — `prev_graph` diff produces rimraf of stale directory; planted contents are gone after. - `nested_hierarchy_materializes_inner_node_modules` — version-conflict layout; loser ends up at `<outer>/node_modules/<inner>`. - `missing_cas_for_required_dep_errors` — required + missing CAS → typed `MissingCasPaths` error. - `missing_cas_for_optional_dep_skips_silently` — optional + missing CAS → no error, no directory. - `no_prev_graph_skips_orphan_pass` — fresh install (no prior lockfile) path. - `orphan_already_removed_is_tolerated` — phantom orphan in `prev_graph` not present on disk doesn't error. Each test plants synthetic CAS files in a tempdir and asserts the on-disk tree after the linker runs. * fix(package-manager): fail-fast on hierarchy/graph inconsistency (#505 review) The hierarchy walk silently skipped entries missing from `graph`, which would produce a partial install layout instead of surfacing the bug. Slice 4's walker keeps the two in sync today, but a future bug there shouldn't yield a partial tree. Add `LinkHoistedModulesError::MissingGraphNode { dir }` and return it when a hierarchy directory has no graph entry. Upstream effectively errors here too — its `graph[dir].fetching` read would throw `Cannot read properties of undefined` — pacquet just spells the failure out. Regression test `hierarchy_entry_missing_from_graph_errors` exercises the path with an empty graph and a hierarchy referencing a phantom dir. Caught by Coderabbit.
… + E) (#506) * feat(package-manager): runtime bin-link + --no-runtime (#437 slice D3 + E) Closes the install-pipeline side of #437: **Slice D3 — runtime bin-link integration.** Runtime archives (`node@runtime:`, etc.) don't ship a `package.json`, so the existing bin-link step had nothing to consume off the slot. `fetch_binary_resolution_to_cas` now synthesizes one in the same shape upstream's `appendManifest` does: { "name": "<pkg.name>", "version": "<pkg.version>", "bin": <BinarySpec> } The bytes go through `StoreDir::write_cas_file` (same content- addressing as every other CAS-imported file), the resulting cas-path is inserted into the snapshot's `cas_paths` under `package.json`, and the existing `link_bins_of_packages` flow picks it up. `link_bins::build_has_bin_set` now includes `Binary` / `Variations` resolutions unconditionally — pnpm v11 doesn't emit `hasBin: true` on runtime metadata, but the synthesized manifest always carries bins, so the lookup must not be gated on the metadata flag. **Slice E — `--no-runtime` flag.** New `Config::skip_runtimes` (default false) and `InstallArgs::no_runtime` CLI flag. The CLI computes `config.skip_runtimes || --no-runtime` and threads it through `Install` → `InstallFrozenLockfile`. When set, every snapshot whose metadata resolution is `Binary` or `Variations` is added to the install-time skip set (re-using `add_optional_excluded` since the bucket count and `.modules.yaml.skipped` semantics line up with `--no-optional`). **Unit tests.** - `synthesize_runtime_manifest_emits_name_version_and_bin_single` — pins `BinarySpec::Single` → JSON string. - `synthesize_runtime_manifest_emits_name_version_and_bin_map` — pins `BinarySpec::Map` → JSON object with all bin entries. - `synthesize_runtime_manifest_preserves_scoped_name` — `@scope/name` round-trips through the `name` field. - `build_has_bin_set_includes_runtime_resolutions_even_when_has_bin_is_absent` — pins the runtime arm. Verified by removing the arm — test fails as expected. End-to-end install fixtures (slice F's remaining items — recorded-archive frozen-lockfile install, variant-mismatch error, `--no-runtime` integration, integrity-mismatch path) need a small recorded Node archive in the repo (or a mockable `pnpm-resolution-mirror`). Tracking them as a separate follow-up so this PR stays reviewable. * fix(package-manager): narrow --no-runtime to importer-direct deps Address CodeRabbit review on #506: my initial implementation iterated `packages` (every metadata entry) and added all `Binary` / `Variations` snapshots to the skip set, which widened the behavior beyond pnpm's `skipRuntimes`. Per the cardinal rule (CLAUDE.md: "Port behavior faithfully. ... Do not invent behavior that pnpm does not have."), match upstream exactly. Upstream's filter at [`installing/deps-installer/src/install/index.ts`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/src/install/index.ts#L1374-L1387) iterates `dependenciesByProjectId` (per-importer direct deps) and adds `depPath` to `ctx.skipped` when `depPath.includes('@runtime:')`. Transitive runtime entries — unusual but possible — stay in the install. Pacquet now mirrors that exactly: walk each importer's `dependencies` / `dev_dependencies` / `optional_dependencies`, build the candidate snapshot key from `(alias, version)`, check for the `@runtime:` substring, and only add the metadata-confirmed `Binary` / `Variations` keys to `skipped`. Aliased runtime deps (unusual) fall through the same way upstream does — pnpm's lookup is by depPath, not by alias.
…508) The Slice 5 linker (#505) was written against `String` for `pkg_id_with_patch_hash` and merged onto a main that had already switched the underlying `DependenciesGraphNode` field to the `PkgIdWithPatchHash` newtype (#504). The merge auto-resolved without flagging the type incompatibility — `cargo check` on main now errors with: error[E0277]: the trait bound `String: Borrow<PkgIdWithPatchHash>` is not satisfied error[E0308]: mismatched types expected `String`, found `PkgIdWithPatchHash` Switch the two slice-5-introduced surfaces to match the newtype: - `CasPathsByPkgId = HashMap<PkgIdWithPatchHash, HashMap<String, PathBuf>>` — the per-package CAS index now keys on the brand the graph node carries, matching the post-#504 invariant across the workspace. - `LinkHoistedModulesError::MissingCasPaths.pkg_id_with_patch_hash: PkgIdWithPatchHash` — same type the graph node has, so the error round-trips the value without `.to_string()`. Tests updated to construct `PkgIdWithPatchHash::from(...)` keys/fields. All 8 linker tests pass on the fixed branch. This unblocks main building again.
…setting check (#434 slice 7) (#507) * feat: ignoredOptionalDependencies config + lockfile field + outdated-setting check (#434 slice 7) Last slice of the optional-dependencies umbrella (#434). Adds the user-facing `ignoredOptionalDependencies` setting: a list of dep-name patterns the user wants entirely excluded from resolution + install. Mirrors 94240bc046's three surfaces: - **Hook**: `hooks/read-package-hook/src/createOptionalDependenciesRemover.ts` builds a matcher and drops matching keys from `optionalDependencies` AND `dependencies` (a package may list the same dep under both for "optional only when consumed"). - **Lockfile field**: `lockfile/types/src/index.ts:19` — `ignoredOptionalDependencies?: string[]` at the top level (sibling of `lockfileVersion`/`overrides`, NOT inside `settings`). - **Drift check**: `lockfile/settings-checker/src/getOutdatedLockfileSetting.ts:58-60` sorts both arrays and compares; mismatch triggers `needsFullResolution`. In pacquet's frozen-only flow this surfaces as `OutdatedLockfile`. ## Changes - **`pacquet-config`**: `Config::ignored_optional_dependencies: Option<Vec<String>>` + `WorkspaceSettings` field + `apply_to` wiring. - **`pacquet-lockfile`**: `Lockfile::ignored_optional_dependencies: Option<Vec<String>>` top-level field with serde round-trip; `check_lockfile_settings(lockfile, config_set)` sorts-and- compares; new `StalenessReason::IgnoredOptionalDependenciesChanged { lockfile, config }` variant. - **`pacquet-lockfile::satisfies_package_manifest`** extended with an `is_ignored_optional: &dyn Fn(&str) -> bool` parameter. Skips matching names in `flat_manifest_specs` and the per-field check so a manifest that still lists ignored entries doesn't falsely surface as drift against a lockfile the resolver correctly built without them. - **`pacquet-package-manager::install.rs`**: builds a matcher from `Config::ignored_optional_dependencies` (reuses `pacquet_config::matcher::create_matcher` — same glob engine as `hoistPattern`); calls `check_lockfile_settings` before `satisfies_package_manifest`; threads the matcher closure into the freshness check. - **`pacquet-package-manager::current_lockfile`**: preserves the lockfile's `ignored_optional_dependencies` through the slice 6 filter so the recorded set round-trips to the current lockfile. ## Tests - `pacquet_config`: yaml-parse + `apply_to` round-trip + omission baseline. - `pacquet_lockfile::freshness::check_settings_*`: both-sides- empty, sorted-match-regardless-of-order, drift in both directions. Test-the-test verified by removing the sort+compare guard. - `pacquet_lockfile::freshness::ignored_optional_filtered_*`: manifest-side filter passes when the matcher fires; polarity test confirms the unfiltered case surfaces as `SpecifiersDiffer`. - `pacquet_lockfile::freshness::ignored_optional_dependencies_round_trips_through_yaml`: serde wire-shape round-trip. Closes #503. Closes the #434 umbrella. * fix(lockfile): scope ignoredOptionalDependencies filter to prod+optional + set-based predicate CodeRabbit review on PR #507 (major): the filter wrongly applied to `devDependencies` too. Upstream's `hooks/read-package-hook/src/createOptionalDependenciesRemover.ts` iterates `manifest.optionalDependencies` and deletes matches from `optionalDependencies` AND `dependencies` only — `devDependencies` is untouched. The previous impl applied the filter to all three groups in both `flat_manifest_specs` and the per-field check, so a stale lockfile could incorrectly pass when the manifest added or removed a matching `devDependency`. Two fixes: 1. **Group gate** in `flat_manifest_specs` and the per-field check: apply the filter only when the group is `Prod` or `Optional`. `Dev` walks ignore the closure. 2. **Set-based predicate** at the call site (`Install::run`): build the "to drop" set from `manifest.optionalDependencies ∩ pattern`, not just from the pattern. A name listed only in `dependencies` (not `optionalDependencies`) that happens to match the pattern is NOT removed by upstream's hook (the hook never iterates that name). The set-based predicate captures that nuance. Both fixes together mirror the hook's exact semantics. Two new regression tests pin the dev-dependency behavior: `ignored_optional_does_not_apply_to_dev_dependencies` and `ignored_optional_dev_only_lockfile_entry_kept`. Test-the-test verified by dropping the group gate inside `flat_manifest_specs` — both tests fail.
…eys (#509) Replaces the `native-tls-vendored` reqwest feature with `rustls`, closing the PKCS#1 parity gap flagged in #499. Native-tls's `Identity::from_pkcs8_pem` accepted only `-----BEGIN PRIVATE KEY-----` (PKCS#8); rustls's `Identity::from_pem` accepts PKCS#1 (`-----BEGIN RSA PRIVATE KEY-----`), PKCS#8, and EC keys — the same surface Node's `tls.createSecureContext` exposes to pnpm via undici. Pacquet now matches pnpm bug-for-bug on the set of client-cert key formats accepted from `.npmrc`'s `key=` / `:key=` / `:keyfile=` entries. PKCS#12 (`.pfx`) stays out of scope — pnpm's `.npmrc` allow-list doesn't expose a `pfx=` option so pacquet doesn't either. ## What changed - `Cargo.toml`: drop `native-tls-vendored`, add `rustls`. Reqwest's `rustls` feature uses `aws-lc-rs` for crypto and `rustls-platform-verifier` for OS trust roots — closest behavioral match to native-tls's "consult the platform trust store" default. - `crates/network/src/lib.rs`: `apply_tls` swaps `Identity::from_pkcs8_pem(cert, key)` for `Identity::from_pem` applied to the concatenated `cert\nkey` PEM buffer. Adds a new `looks_like_pem_cert` syntactic check before `Certificate::from_pem` because rustls's `from_pem` stores the bytes verbatim and validates lazily at `Client::build()` time — a garbage CA entry would otherwise slip through silently and the install would proceed against an unknown trust root. - Updated doc comments on `apply_tls`, `TlsConfig::key`, `RegistryTls::key`, and `TlsError::InvalidClientIdentity` to describe the new surface and drop the PKCS#8-only caveat. - `deny.toml`: allow `CDLA-Permissive-2.0` for `webpki-root-certs` (pulled in by reqwest's `rustls` feature through `rustls-platform-verifier`'s fallback chain). - New fixtures at `crates/network/tests/fixtures/test-client-pkcs1.{crt,key}` loaded via `include_str!`. Regenerable with `openssl genrsa -traditional` + `openssl req -new -x509`. ## Tests `for_installs_with_pkcs1_client_key_builds` pins the contract — if a future change reverts the backend or otherwise narrows the accepted key formats, this build will fail with a clear `InvalidClientIdentity`. All 1175 workspace tests pass. ## Notes for review - **Cert store change.** `rustls-platform-verifier` reads the OS trust store on macOS / Windows / Linux. The lookup is a different syscall path from native-tls's; behavioral parity for "trust roots in the OS store" should hold, but corporate CAs that worked under native-tls and *don't* show up in `rustls-platform-verifier`'s enumeration would now silently fail. Users hitting that should add the CA explicitly via `cafile=` in `.npmrc`. - **Performance.** CI's integrated-benchmark will run on this PR; if it regresses materially on the warm-install path we'd consider falling back to the preprocessing approach (option 2 in #499). - **`hickory-dns` compatibility.** Verified by running the workspace test suite — DNS resolution is independent of the TLS backend. - **`cargo deny` posture.** One new license allowance (`CDLA-Permissive-2.0`) for `webpki-root-certs`. No new advisory surface area beyond what reqwest's `rustls` feature already pulls through `aws-lc-rs` / `rustls` / `rustls-platform-verifier`.
Pnpm v11 writes runtime depPaths with a scheme prefix in front of
the semver — e.g. `node@runtime:22.0.0` in `packages:` /
`snapshots:` keys, and `runtime:22.0.0` in importer `dependencies:`
values. Pacquet's `PkgVerPeer` parser previously accepted only
bare semver and rejected the prefix, so it couldn't load any v11
lockfile containing a runtime dependency.
Extend `PkgVerPeer` to recognise a leading `runtime:` (closed enum
`Prefix::{None, Runtime}`) and preserve it through `Display` /
`serde`. The version + peer parts continue to parse the same way
— the prefix-strip happens once up front, after which the
parenthesis-based peer-suffix detection is unchanged.
Touch points:
- Added `Prefix` enum with `as_str()` and `Display`.
- `PkgVerPeer::prefix()` exposes the variant so downstream
consumers can discriminate runtime entries without
substring-searching the depPath.
- `PkgVerPeer::into_tuple()` keeps the `(Version, String)` shape
for backward compatibility — pre-runtime callers don't see the
prefix.
5 new unit tests pin: bare-semver no-prefix, runtime no-peer,
runtime with peer-suffix, runtime-substring-mid-version is NOT a
prefix (anchored at start), and serde round-trip on the runtime
form. All 100 existing `pacquet-lockfile` tests still pass.
Unblocks #437 slice F (end-to-end install fixtures) — pacquet's
lockfile loader now accepts `node@runtime:X.Y.Z`-shaped keys.
Add the `--node-linker [isolated|hoisted|pnp]` CLI flag to `pacquet install`, mirroring pnpm's flag. Overrides `Config.node_linker` for the invocation; absent flag → config's yaml/npmrc value wins. The CLI parses into a `NodeLinkerArg` mirror enum (kept in the CLI crate so `pacquet-config` stays free of `clap`), then `into_config()` converts to the canonical `pacquet_config::NodeLinker`. Threaded into the `Install` runner as an explicit `node_linker: NodeLinker` field rather than reading `config.node_linker` directly. Matches the existing pattern `supported_architectures` uses (`state.config` is `&'static`, so the override-merge happens at the CLI layer and lands here as a fully-resolved value). `build_modules_manifest` consumes it on the `.modules.yaml.nodeLinker` write so the persisted setting reflects the invocation, not just the config. `pacquet add` uses Config's value by default (`add` doesn't expose `--node-linker` per the umbrella scope; Slice 6's pipeline integration will revisit if needed). 5 new CLI tests: - `node_linker_default_is_none` — flag absent → field is None. - `node_linker_hoisted` / `node_linker_isolated` / `node_linker_pnp` — each value parses and round-trips through `into_config()`. - `node_linker_invalid_value_rejected` — clap rejects unknown values with the bad value in the error message. - `node_linker_arg_into_config_matches_every_variant` — every ValueEnum variant has a canonical mapping (compile-fails on future variants that forget the mapping). `NodeLinker` gains `Clone + Copy + Eq` so it can be threaded by value into `Install` and matched in tests.
#513) * feat(config,cli,tarball): support `offline` / `preferOffline` settings Adds the two settings to `Config`, `pnpm-workspace.yaml` parsing, and `pacquet install` as `--offline` / `--prefer-offline` flags. CLI flags merge into Config (boolean OR) before leak, so any source (yaml, CLI) flips `true` and never the other way. Upstream pnpm gates the metadata-fetch path in [`pickPackage`](https://github.com/pnpm/pnpm/blob/94240bc046/resolving/npm-resolver/src/pickPackage.ts): when `offline: true` and no cached metadata exists, it fails with `ERR_PNPM_NO_OFFLINE_META`; `preferOffline: true` biases the resolver toward the cached copy. Pacquet's frozen-install path has **no metadata fetch** (the lockfile pins every resolution), so the upstream flag is semantically a no-op on pacquet's current flow. To make the flag actually useful for frozen installs, pacquet adds a **tarball-side gate** (no upstream equivalent — pnpm doesn't gate tarball downloads on `offline`): when both the warm prefetch and the SQLite `index.db` lookup miss, the fetcher refuses the network and errors with `ERR_PACQUET_NO_OFFLINE_TARBALL`. Same shape as `ERR_PNPM_NO_OFFLINE_META`, scoped to tarballs because that's what pacquet's frozen install needs network for. Documented as a pacquet- specific divergence in the field's rustdoc. `prefer_offline` is a no-op today — the warm prefetch + `load_cached_cas_paths` already prefer the local store. The field exists so `.npmrc` / yaml / CLI all parse cleanly; Stage 2's resolver will honour it on the metadata path. - `Config.offline` / `Config.prefer_offline`: bool, default `false`. - `WorkspaceYaml.offline` / `WorkspaceYaml.prefer_offline`: `Option<bool>`; applied via the existing `apply!` macro. - `InstallArgs.offline` / `InstallArgs.prefer_offline`: bool CLI flags; merged into Config in `cli_args.rs` between `config()` and `State::init` (the brief window when `Config::leak` returns `&'static mut Config`). - `DownloadTarballToStore.offline` / `DownloadZipArchiveToStore.offline`: bool fields. Wired through the three construction sites in `package-manager`. - `TarballError::NoOfflineTarball { package_id, url }`: new variant. `tarball_error_to_request_retry` updated for exhaustiveness; `code: ERR_PACQUET_NO_OFFLINE_TARBALL` in the retry-logger projection so a future surface that runs this error through the retry path renders the right code. - `offline_mode_skips_network_on_cache_miss` — `mockito` server with `.expect(0)`; the offline gate must fire before the network is touched. Asserts variant shape AND the diagnostic code (the code is part of the user-facing surface). - `offline_mode_still_uses_prefetched_cache` — same `.expect(0)` guard but with a prefetched cache hit; pins that the prefetch branch short-circuits *before* the offline check so warm installs under `--offline` succeed. Regression-catch verified: gating the gate behind `false &&` flipped `offline_mode_skips_network_on_cache_miss` red with the wrong-variant panic message. Reverted before commit. All bulk-edit fixups (15 `DownloadTarballToStore` / `DownloadZipArchiveToStore` literals in tests + micro-benchmark, 1 `Config` literal in package- manager tests) get `offline: false` to match the prior implicit default. * docs(cli): pin pickPackage link to commit SHA (#513 review) Per project guideline: upstream pnpm citations link to a specific commit SHA (first 10 hex chars), not a branch name. Same SHA the PR's rustdoc and commit body already use for the same file. Resolves CodeRabbit review comment on `crates/cli/README.md`: <pnpm/pacquet#513 (comment)>.
…438 slice 6) (#518) * feat(package-manager): wire node_linker hoisted into install pipeline (#438 slice 6) Branches `Install::run` / `InstallFrozenLockfile::run` on `config.node_linker == Hoisted`. Threads the slice 4 `lockfile_to_hoisted_dep_graph` walker output and the per-package CAS index produced by `CreateVirtualStore` (slot writes skipped under hoisted) into the slice 5 `link_hoisted_modules` linker. Persists the walker's `hoisted_locations` into `.modules.yaml` for rebuild and follow-up installs to consume. Pipeline changes under hoisted: - `CreateVirtualStore` skips `CreateVirtualDirBySnapshot` for both warm and cold batches, collects each snapshot's CAS file index keyed by `PkgIdWithPatchHash` into a new `cas_paths_by_pkg_id` output field. - `InstallPackageBySnapshot::run` returns the per-package CAS map unconditionally and skips the virtual-store-slot write when its new `node_linker` field is `Hoisted`. - `InstallFrozenLockfile::run` skips `SymlinkDirectDependencies`, `LinkVirtualStoreBins`, the isolated hoist pass, and `BuildModules` under hoisted; runs `lockfile_to_hoisted_dep_graph` + `link_hoisted_modules` in their place. Folds the walker's augmented skip set back into the install-time `SkippedSnapshots` so `.modules.yaml.skipped` reflects the union. - `Install::run`'s `build_modules_manifest` now takes the walker's `hoisted_locations` and writes it through `Modules.hoisted_locations` (only when non-empty so the isolated path doesn't produce a hoisted-only key). The build phase under hoisted (rebuild over `hoistedLocations` with ancestor-`.bin` lookup, `MISSING_HOISTED_LOCATIONS`) is slice 7's scope and is intentionally left as a no-op here. Workspace and `hoistingLimits` / `externalDependencies` knobs are slices 9-10. Mirrors upstream's hoisted branch in `headlessInstall` at https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L369-L425. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(package-manager): docs intra-link disambiguation + skip-set merge fix (#438 slice 6) Docs CI was failing because `[`crate::link_hoisted_modules`]` is ambiguous between the function and the module of the same name. Disambiguate by switching to the function form `[`crate::link_hoisted_modules()`]` everywhere. Slice 6's hoisted-walker skip-set merge previously folded every entry in `walker_result.skipped` into `SkippedSnapshots::insert_installability`, which would promote pre-existing transient skips (`fetch_failed` / `optional_excluded`) into the persisted-on-disk `.modules.yaml.skipped` set. Diff against the input `walker_skipped` so only walker-discovered (genuinely-new) installability skips flow into the persisted subset.
…lice 7) (#520) Re-enables `BuildModules` for `nodeLinker: hoisted` installs. Slice 6 landed the hoisted install pipeline but skipped the build phase entirely because `BuildModules` walked virtual-store slot directories that don't exist under hoisted. Slice 7 routes the build phase through the slice 4 walker's per-node `dir` so postinstall scripts can run against the on-disk hoisted tree. Changes: - `BuildModules` gains two fields: `pkg_root_by_key: Option<&HashMap<PackageKey, PathBuf>>` overrides the per-snapshot pkg_root lookup with the slice 4 walker's `DependenciesGraphNode::dir` values; `gather_ancestor_bin_paths: bool` switches `extra_bin_paths` to the new `bin_dirs_in_all_parent_dirs` helper, a port of upstream's `binDirsInAllParentDirs` at https://github.com/pnpm/pnpm/blob/94240bc046/building/after-install/src/index.ts#L476-L487. - `pkg_root_for_key` helper: routes between the layout-based slot computation (isolated) and the override map (hoisted). Hoisted snapshots absent from the map (walker-dropped) take the same exit as the isolated `!pkg_dir.exists()` skip. - `InstallFrozenLockfile::run` now builds a snapshot-key → first-recorded-dir map from `walker_result.graph.values()` and threads it (plus `gather_ancestor_bin_paths: true`) into `BuildModules` instead of skipping the phase. Multiple graph nodes with the same dep_path collapse to the first entry, matching upstream's `pkgRoots[0]` pick at https://github.com/pnpm/pnpm/blob/94240bc046/building/after-install/src/index.ts#L348. - New private `HoistedLinkerOutput` struct bundles `hoisted_locations` and `pkg_root_by_key` so the hoisted-branch return doesn't trip `clippy::type_complexity`. Side-effects-cache key shape is unchanged — it's keyed by `pkg_id_with_patch_hash` + dep-graph hash, both layout-independent (`crates/graph-hasher/src/dep_state.rs`). `MISSING_HOISTED_LOCATIONS` is intentionally deferred — pacquet has no `rebuild` command, so the install path always re-runs the walker and never reads `.modules.yaml.hoisted_locations`. Tracked as a follow-up for when `pacquet rebuild` lands. Workspace-aware hoisting (slice 9) and `hoistingLimits` / `externalDependencies` (slice 10) remain. Tests: - Three new helper tests pin `bin_dirs_in_all_parent_dirs` against top-level, conflict-nested, and scoped-package shapes. - Three new helper tests pin `pkg_root_for_key` for the isolated pass-through, the hoisted override hit, and the hoisted-missing short-circuit.
…437 slice F) (#516) * test(package-manager): variant-mismatch + --no-runtime install tests (#437 slice F) Two end-to-end install tests for the runtime-dependency pipeline, now that #512 (`PkgVerPeer::Prefix::Runtime`) unblocked fixture-lockfile parsing. - `frozen_lockfile_install_errors_when_no_variant_matches_host`: Lockfile with a `VariationsResolution` whose only variant targets `aix/ppc64` (a platform pacquet CI never runs on). Install fails with `NoMatchingPlatformVariant` from the cold-batch dispatcher. Variant selection runs before any network fetch, so the bogus archive URL is never read — the test stays hermetic. Closes the variant-mismatch checkbox. - `frozen_lockfile_install_skips_runtime_when_skip_runtimes_set`: Same lockfile, but with `skip_runtimes: true`. The `--no-runtime` filter iterates importer-direct deps, builds `node@runtime:22.0.0` from `(alias, version)`, sees the `@runtime:` substring, and adds the snapshot to the skip set — so variant selection never runs and the unmatchable variant doesn't fail the install. Asserts both the runtime slot and the direct-dep symlink are absent. Closes the `--no-runtime` checkbox. Both tests were verified to catch their respective regressions (disable the `skip_runtimes` filter → the second test fails with `NoMatchingPlatformVariant` instead of succeeding). Required a fix in `PkgNameVerPeer::without_peer`: the existing implementation dropped the `Prefix::Runtime` when stripping the peer-dependency suffix, so a runtime snapshot key like `node@runtime:22.0.0(some@peer)` would resolve to the non-existent `node@22.0.0` `packages:` entry. Now preserves the prefix through the strip. Slice F's remaining items (happy-path archive fetch, integrity mismatch, NODE_EXTRAS verified end-to-end, TEST_PORTING.md entries) need a mock HTTP server + recorded archive fixture, so defer to a follow-up — keeping this PR reviewable. * test(package-manager): use symlink_metadata for absent-entry checks Address CodeRabbit review on #516: `Path::exists()` follows symlinks, so a regression that created a *dangling* symlink at `<modules_dir>/node` would still pass the `!direct_dep.exists()` assertion — the install would have created the entry the skip set was supposed to suppress, but the test wouldn't notice. Switch both "must NOT exist" assertions to `std::fs::symlink_metadata(...).is_err()` so the entry-level check catches dangling symlinks too.
…e 9) (#521) Enables `nodeLinker: hoisted` for multi-importer (workspace) lockfiles. Previously the hoister rejected any lockfile with importers beyond `.` with `UnsupportedWorkspace`; now the whole workspace shares one hoist plan so conflicting versions across projects dedupe, and each importer's project-tree node_modules is materialized per-project. real-hoist: - Drop the upfront UnsupportedWorkspace guard in `hoist()`. The wrapper already constructed non-root importers as Workspace-kind children of the virtual `.` root; only the guard needed to go. - Add `HoistOpts::hoist_workspace_packages: bool` (default true). When false, non-root importers stay out of the shared tree (matches upstream's `hoistWorkspacePackages: false` mode for the Bit CLI). Removed the corresponding error variant. hoisted_dep_graph walker: - Replace the `workspace:`-prefix skip with a recurse-into branch: for each Workspace-kind hoister child, walk its post-hoist children under `<lockfile_dir>/<importer_id>/node_modules`. The workspace node itself is NOT added to the graph or to the parent's hierarchy — it has no contents to import. - Add per-importer `hierarchy` and `direct_dependencies_by_importer_id` entries. Per-importer direct deps are computed from the lockfile (not from the workspace node's post-hoist children) because hoisted-up siblings don't show up in the workspace node's tree. First-recorded location wins, matching upstream's `pkgLocationsByDepPath[depPath][0]` pick. - Add `LockfileToHoistedDepGraphOptions::hoist_workspace_packages` (default true) and plumb to HoistOpts. Config: - Add `Config::hoist_workspace_packages: bool` (#[default = true]). Read from `pnpm-workspace.yaml` via `WorkspaceSettings`. SymlinkDirectDependencies: - Add `link_only: bool` flag. When true, skip every Regular dep and only materialize Link entries (workspace siblings). Used by the hoisted branch in InstallFrozenLockfile::run so workspace-sibling symlinks land under each importer's `node_modules/<alias>` even though the regular deps now live as real directories from the hoisted linker. InstallFrozenLockfile::run: - Plumb `config.hoist_workspace_packages` into the walker. - After link_hoisted_modules, run SymlinkDirectDependencies with `link_only: true` so workspace siblings get their per-project symlinks. Mirrors upstream's hoisted branch at https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L411-L440. Tests: - real-hoist: replace the now-removed `UnsupportedWorkspace` test with one that pins multi-importer hoister output (Workspace children with percent-encoded names + `workspace:<id>` references) and one that pins the `hoist_workspace_packages: false` opt-out. - walker: three new tests cover per-importer direct_deps emission, per-importer hierarchy entries, the `hoist_workspace_packages: false` root-only mode, and version-conflict-across-importers nesting.
…nobs (#438 slice 10) (#522) Plumbs the two programmatic-only hoister knobs from `pnpm-workspace.yaml` through to the slice 4 walker and the slice 3 hoister. Both fields already existed on `HoistOpts`; this slice wires them end-to-end. - `Config::hoisting_limits: BTreeMap<String, BTreeSet<String>>` — per-importer block-list, locator-keyed (`'.@'` for the root). Reads `hoistingLimits: { ".@": [foo, bar] }` from yaml. Mirrors upstream's https://github.com/pnpm/pnpm/blob/94240bc046/installing/linking/real-hoist/src/index.ts#L10 programmatic-only knob, exposed as yaml for parity since the ergonomics of the locator-keyed map don't translate to a CLI flag. - `Config::external_dependencies: BTreeSet<String>` — name slots reserved at the root for an external linker (the Bit CLI is the only known consumer upstream). Reads `externalDependencies: [...]` from yaml. - `LockfileToHoistedDepGraphOptions` gains both fields and forwards them to `HoistOpts` in `build_dep_graph`. - `InstallFrozenLockfile::run` clones the two `Config` fields into the walker opts. Both knobs default to empty (no limits, no externals), matching upstream's default. Neither has any effect under `nodeLinker: isolated` — the isolated linker keeps per-importer subtrees by construction and doesn't consult the hoister. Tests: - `parses_hoisting_limits_from_yaml_and_applies` — yaml round-trip + apply_to. - `parses_external_dependencies_from_yaml_and_applies` — same. - `omitting_hoisting_limits_and_external_dependencies_keeps_defaults` — pins the apply_to skip-on-None branch so a yaml without these keys doesn't accidentally overwrite Config defaults. - `walker_forwards_external_dependencies_to_hoister` — end-to-end: the walker observes an empty graph for an externalised alias because the hoister stripped it. Pins the slice 10 plumbing.
…op-level bin re-link (#342) (#523) Two related bin-linking behaviors deferred from #333. Behavior 1 — hoisted-bin precedence: - Add `BinOrigin { Direct, Hoisted }` discriminator to `PackageBinSource`. - New top tier in `pick_winner`: Direct wins outright over Hoisted regardless of ownership / lexical order. Mirrors upstream's `preferDirectCmds` partition at https://github.com/pnpm/pnpm/blob/4750fd370c/bins/linker/src/index.ts#L92. - New `link_top_level_bins(modules_dir, direct, hoisted)` helper in `pacquet-package-manager` mixes both candidate lists into one `link_bins_of_packages` call so the new tier resolves conflicts in a single pass — previously the two passes (SymlinkDirectDependencies for direct + hoist pass for publicly-hoisted) wrote shims independently and a hoisted bin could shadow a direct one when its package name was lexically smaller. Behavior 2 — lifecycle-script-created bins: - Add a post-`BuildModules` per-importer top-level bin link pass. Re-reading each direct dep's `package.json` after lifecycle scripts run picks up bins generated by `postinstall` (the `@pnpm.e2e/generated-bins` upstream fixture). Idempotent for unchanged shims via `is_shim_pointing_at`. - Mirrors upstream's `linkBinsOfImporter` pass that runs after `buildModules` at https://github.com/pnpm/pnpm/blob/4750fd370c/installing/deps-installer/src/install/index.ts#L1539. Supporting changes: - `PackageBinSource::new(location, manifest)` constructor + `with_origin` builder so existing call sites don't have to spell out the new field. - Public `direct_dep_names_for_importer` helper extracted from `SymlinkDirectDependencies` so the post-build pass uses the same filter (skipped / first-wins / link_only) as the symlink phase. - `InstallFrozenLockfileError::TopLevelBinLink` for the new failure surface. Tests: - `direct_origin_wins_over_hoisted_regardless_of_lexical` — pins the new tier overrides lexical ordering. - `hoisted_origin_loses_to_existing_direct` — pins both arms of the new tier (Direct incumbent vs Hoisted candidate).
…sions (#524) pnpm writes importer dependency `version:` fields in three shapes: bare semver-with-peer (`4.0.0`), `link:<path>`, or — when a specifier (typically `catalog:`) resolves to a different package name — the full npm-alias `<name>@<version>`. The third shape is what `refToRelative` recognises with the same leading-`@` / `@` before `(`/`:` test that `SnapshotDepRef` already uses, and which pnpm v11 emits for entries like: js-yaml: specifier: 'catalog:' version: '@zkochan/js-yaml@0.0.11' Pacquet's `ImporterDepVersion` only modelled `Regular` and `Link`, so deserialising a lockfile with an aliased catalog dep failed with "Failed to parse importer dependency version". Add an `Alias(PkgNameVerPeer)` variant and a `resolved_key` helper that returns the correct snapshot-map key for each shape — the importer-map key paired with the version for `Regular`, the alias's own `(name, suffix)` for `Alias`, and `None` for `Link`. Every site that previously built a key from `as_regular().map(|v| PkgNameVerPeer::new(name, v))` now goes through `resolved_key`, so aliased deps reach the snapshot, the skipped-set, the reachability BFS, the build-sequence root walk, the runtime exclusion check, and both hoist passes correctly. For symlink targets, an aliased dep links the importer-key name to `<slot>/node_modules/<alias-real-name>` (the resolved package's true name inside its slot), matching pnpm's `linkDirectDeps`. The `pnpm:root added` event now reports `realName` as the resolved package name for aliases, where before it always echoed the importer-map key. Upstream reference: `refToRelative` in `pnpm/pnpm@8a80235/deps/path/src/index.ts:96-110`.
…525) * fix(lockfile): skip env document in pnpm v11 combined lockfiles Pnpm v11 writes `pnpm-lock.yaml` as a stream of up to two YAML documents: an optional first document carries the package-manager bootstrap (`packageManagerDependencies` and the snapshots that back it) and the second document is the regular project lockfile. Pacquet hands the file straight to `serde_saphyr::from_str`, which rejects a multi-document stream with "multiple YAML documents detected" — so every install against a v11 repo that has a `packageManager` / `devEngines.packageManager` declaration fails before staleness checking even runs. Port upstream's `extractMainDocument` to a new `yaml_documents` module: if the file starts with `---\n`, return the slice after the next `\n---\n` separator; if there is no second separator, the file is env-only and the loader returns `Ok(None)`. `load_from_path` threads the lockfile content through the filter before deserializing, matching pnpm's `_read` call site at https://github.com/pnpm/pnpm/blob/31858c544b/lockfile/fs/src/read.ts#L103-L110. Three unit tests in `yaml_documents/tests` mirror upstream's `extractMainDocument` test cases, and two integration tests in `load_lockfile/tests.rs` cover the combined-document and env-only paths end-to-end. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(package-manifest): reify devEngines.runtime into devDependencies With #511 / #512 pacquet recognises `runtime:` specifiers in the lockfile, so a frozen install against a v11 repo gets past the YAML parse but then trips the staleness check: the lockfile lists `node@runtime:24.6.0` under the root importer's `devDependencies`, while the on-disk `package.json` only declares the runtime through `devEngines.runtime`. The flat-record diff then surfaces a spurious "node@runtime:24.6.0 was removed" mismatch. Port upstream's `convertEnginesRuntimeToDependencies` to a new free function in `crates/package-manifest`. For each of `node`, `deno`, `bun`, if `devEngines.runtime` (or `engines.runtime`) declares the runtime with `onFail: "download"` and an explicit version, and the target dependencies bucket has no explicit entry yet, insert `<name>: "runtime:<version>"`. Array and single-object runtime shapes are both accepted. Skip when `onFail` is anything other than `"download"` or when the version is absent — upstream warns on the missing-version path; pacquet skips silently and the staleness check still surfaces the gap if it matters downstream. WebContainer's "no runtime download" branch is intentionally omitted since pacquet does not run there. `PackageManifest::read_from_file` calls the function for both `(devEngines, devDependencies)` and `(engines, dependencies)`, mirroring upstream's `convertManifestAfterRead` at https://github.com/pnpm/pnpm/blob/9cad8274fd/workspace/project-manifest-reader/src/index.ts#L227-L231. Six unit tests in `tests.rs` cover the happy path, the no-version skip, the non-`download` `onFail` skip, preservation of an explicit user-declared entry, the array-of-runtimes form, and the `engines` → `dependencies` variant. A seventh test exercises the hook through `PackageManifest::from_path` end-to-end. Upstream reference: https://github.com/pnpm/pnpm/blob/9cad8274fd/pkg-manifest/utils/src/convertEnginesRuntimeToDependencies.ts#L10-L45. --- Written by an agent (Claude Code, claude-opus-4-7).
) `create_symlink_layout` only iterated `snapshot.dependencies`, so a package whose CPU/OS-specific siblings live entirely under `optionalDependencies` (e.g. `@typescript/native-preview`, `@reflink/reflink`, every `*-darwin-arm64` / `*-linux-x64` family) ended up with a slot `node_modules/<scope>/` containing only the parent package — no platform binary sibling. Consumers that do `require.resolve('@typescript/native-preview-darwin-arm64')` from inside `getExePath.js` walked parent directories and found nothing, so `tsgo --version` (and every other tool that delegates to a platform variant) crashed with `Unable to resolve … missing the package on disk`. Port upstream's `dependencies ∪ optionalDependencies` merge — the graph builder at https://github.com/pnpm/pnpm/blob/da65e6262/deps/graph-builder/src/lockfileToDepGraph.ts#L150-L156 unifies both maps into one `allDeps` for each node's children, and `linkAllModules` then symlinks every child with two short-circuits: `alias === depNode.name` (a snapshot referencing itself) and `!pkg.installable && pkg.optional` (a non-materialized optional). See https://github.com/pnpm/pnpm/blob/da65e6262/installing/deps-installer/src/install/link.ts#L521-L549. `create_symlink_layout` now takes both maps and a `SkippedSnapshots` reference, merges them, and applies both short-circuits. The skip set is threaded through `InstallPackageBySnapshot` (cold batch) and the warm-batch `CreateVirtualDirBySnapshot` call site in `CreateVirtualStore::run`, so a target dropped by the installability pass, by `--no-optional`, or by a swallowed optional fetch failure gets no dangling symlink. Five new unit tests in `create_symlink_layout/tests.rs` cover the matching-optional happy path, the skipped-optional dangling-link guard, the self-name guard for entries listed in either bucket, the both-buckets-absent no-op, and the alias-resolve path (aliased deps still link the alias filename while resolving the slot via the target's name). End-to-end verification: `pacquet install --frozen-lockfile` followed by `tsgo --version` in the pnpm v11 repo now succeeds; the matching `native-preview-darwin-arm64` sibling shows up in the slot's `node_modules/@typescript/`. --- Written by an agent (Claude Code, claude-opus-4-7).
Imports https://github.com/pnpm/pacquet at the pacquet/ subdirectory, preserving full commit history (rewritten with git filter-repo so blame and log work for files under pacquet/).
|
Important Review skippedToo many files! This PR contains 288 files, which is 138 over the limit of 150. To get a review, narrow the scope: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (12)
📒 Files selected for processing (288)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
zizmor found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Summary
Imports the pnpm/pacquet repository into this monorepo at the
pacquet/subdirectory, preserving full commit history (967 commits dating back to July 2023).How history was preserved
git filter-repo --to-subdirectory-filter pacquetto rewrite every historical commit so its paths live underpacquet/git merge --allow-unrelated-historiesResult:
git log --follow pacquet/<file>andgit blame pacquet/<file>correctly trace back to the original authors/dates from the pacquet repo.Tags were intentionally not imported (
--no-tagson fetch) to avoid collisions with old pnpmv0.*tags.Suggested merge strategy
Push this branch directly to
mainrather than clicking the merge button. GitHub's "Create a merge commit" would add a second merge commit on top of the one already in this PR. The branch fast-forwards cleanly ontomain. The PR will auto-close as "merged" once its commits appear inmain.Do not use "Squash and merge" — it would collapse the 968 commits into one and destroy the imported history.
Follow-ups (not in this PR)
pnpm-workspace.yamllikely needs to excludepacquet/**so JS workspace tooling doesn't try to ingest the Rust cratespacquet/.github/workflows/(inert) — may want to relocate or rewriteTest plan
pacquet/is populated and matches the pacquet repo'smaintipgit log --follow pacquet/Cargo.tomltraces back to the 2023 initial commitgit blame pacquet/<file>shows original authorsWritten by an agent (Claude Code, claude-opus-4-7).