Skip to content

Skip optional dependencies that can't run on the current engine#467

Merged
zkochan merged 7 commits into
masterfrom
issue448
Nov 7, 2016
Merged

Skip optional dependencies that can't run on the current engine#467
zkochan merged 7 commits into
masterfrom
issue448

Conversation

@zkochan

@zkochan zkochan commented Nov 6, 2016

Copy link
Copy Markdown
Member
  • check OS
  • check Node version
  • cover each use case with tests

Ref #448

@zkochan zkochan merged commit 98ca9a0 into master Nov 7, 2016
@zkochan zkochan deleted the issue448 branch November 7, 2016 18:38
@iamstarkov

Copy link
Copy Markdown
Contributor

sorry, was away. couldnt review

@zkochan

zkochan commented Nov 7, 2016

Copy link
Copy Markdown
Member Author

no problem

pull Bot pushed a commit to dwongdev/pnpm that referenced this pull request May 14, 2026
…pm#434 slice 3) (pnpm#467)

## Summary

Slice 3 of the [pnpm#434 umbrella](pnpm/pacquet#434) (`Proper optionalDependencies support`). Slice 1 (pnpm#439) computed a `SkippedSnapshots` set on every frozen-lockfile install but never serialized it; slice 2 (pnpm#456) closed the input side. This PR closes the loop by mirroring three upstream sites pinned at [pnpm/pnpm@94240bc046](https://github.com/pnpm/pnpm/tree/94240bc046):

- **Write** — `Modules.skipped` was declared at [`crates/modules-yaml/src/lib.rs:197`](https://github.com/pnpm/pacquet/blob/97010013/crates/modules-yaml/src/lib.rs#L197) with sort-on-write but always serialized empty. `build_modules_manifest` now takes `&SkippedSnapshots` and produces the depPath string list pnpm writes at [`installing/deps-installer/src/install/index.ts:1625`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/src/install/index.ts#L1625).

- **Seed** — `install_frozen_lockfile::run` reads `.modules.yaml.skipped` before computing the in-memory set and passes the parsed `PackageKey`s as the seed to `compute_skipped_snapshots`. Mirrors upstream's load at [`installing/read-projects-context/src/index.ts:79`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/read-projects-context/src/index.ts#L79) plus the early return at [`deps/graph-builder/src/lockfileToDepGraph.ts:194`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts#L194).

- **Short-circuit** — `compute_skipped_snapshots` returns the seed verbatim on the fast path (no constraints in the lockfile) and, on the slow path, skips the per-snapshot re-check for keys already in the seed without emitting a duplicate `pnpm:skipped-optional-dependency` event. Non-seeded snapshots still re-run `package_is_installable`, covering the \"host arch changed since last install\" case upstream guards at [`:206-215`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts#L206-L215).

`InstallFrozenLockfile::run` now returns a small `InstallFrozenLockfileOutput { hoisted_dependencies, skipped }` struct so `Install::run` can thread both into `.modules.yaml`. Read errors on `.modules.yaml` degrade to an empty seed — the file is a cache artifact, not authoritative.
pull Bot pushed a commit to dwongdev/pnpm that referenced this pull request May 14, 2026
…e payload (pnpm#434 slice 4) (pnpm#474)

Slice 4 of the [pnpm#434 umbrella](pnpm/pacquet#434) (`Proper optionalDependencies support`). Slices 1–3 (pnpm#439, pnpm#456, pnpm#467) close the installability-driven skip path; this slice adds the second skip pathway upstream handles in the frozen install: an `optional: true` snapshot whose **fetch / extract** step fails at install time must not abort the install. Local-materialization errors (`CreateVirtualDir`) and config-shape errors still abort even for optional snapshots — matching upstream's `linkPkg` path which sits outside the catch.

Mirrors the silent catch sites at [`deps/graph-builder/src/lockfileToDepGraph.ts:294-298`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts#L294-L298) and [`installing/deps-restorer/src/index.ts:912-921`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L912-L921):

```ts
try { ... fetch ... }
catch (err) { if (pkgSnapshot.optional) return; throw err }
```

## Changes

- **Reporter**: `SkippedOptionalPackage` becomes a `#[serde(untagged)]` enum with `Installed { id, name, version }` and `ResolutionFailure { name?, version?, bareSpecifier }` variants. Models upstream's discriminated union at [`core-loggers/src/skippedOptionalDependencyLogger.ts:10-31`](https://github.com/pnpm/pnpm/blob/94240bc046/core/core-loggers/src/skippedOptionalDependencyLogger.ts#L10-L31). The new variant is wire-shape-only in slice 4 — pacquet has no resolver yet, but a future resolver port at the upstream emit site ([`resolveDependencies.ts:1376-1383`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-resolver/src/resolveDependencies.ts#L1376-L1383)) lands without re-touching this type.
- **`SkippedSnapshots`**: gains a `fetch_failed` subset disjoint from the existing `installability` subset. `contains` / `iter` return the union (downstream consumers treat both as absent); `iter_installability` returns only the persistent subset that `.modules.yaml.skipped` records, matching upstream's behavior of never updating `opts.skipped` at the fetch-failure catch sites.
- **`CreateVirtualStore`**: cold-batch dispatch swallows per-snapshot errors when `snapshot.optional` is true **and** the error is fetch-side. A new `is_fetch_side_failure` helper restricts the swallow to `InstallPackageBySnapshotError::DownloadTarball` and `::GitFetch` — the same surface upstream wraps in `storeController.fetchPackage`. Local-materialization (`CreateVirtualDir`) and config-shape errors (`MissingTarballIntegrity` / `UnsupportedResolution`) propagate even for optional snapshots, matching upstream's `linkPkg` path which sits outside the catch. Side benefit: the warm-batch path (which only surfaces `CreateVirtualDir` errors) stays consistently abort-on-error without a parallel swallow site. The per-install `fetch_failed: HashSet<PackageKey>` rides out on `CreateVirtualStoreOutput`.
- **`InstallFrozenLockfile::run`**: folds the returned `fetch_failed` into its mutable `SkippedSnapshots` so downstream phases (`SymlinkDirectDependencies`, `LinkVirtualStoreBins`, `BuildModules`, hoist) treat the dropped snapshots as absent through the same gate they already use for installability skips.

## Test plan

- [x] `reporter::tests::skipped_optional_resolution_failure_event_matches_pnpm_wire_shape` — full payload (`bareSpecifier` camelCase, no `id`).
- [x] `reporter::tests::skipped_optional_resolution_failure_omits_absent_name_and_version` — optional-field shape.
- [x] `install::tests::frozen_install_silently_swallows_unreachable_optional_tarball` — ports [`installing/deps-restorer/test/index.ts:340-360`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/test/index.ts#L340-L360): unreachable optional tarball (`http://127.0.0.1:1/...`), install succeeds, slot not created, `.modules.yaml.skipped` left empty. **Test-the-test verified** by inverting the `optional` guard in the cold-batch match arm. Runs with `enable_global_virtual_store = false` so the slot-path assertion targets the legacy layout.
- [x] `install::tests::frozen_install_propagates_non_optional_fetch_failure` — pins the polarity: same fixture, non-optional snapshot, install must error.
- [x] `just ready` (typos + fmt + check + nextest + clippy).
- [x] `taplo format --check`.
- [x] `just dylint` (perfectionist).
- [x] `RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --workspace`.

## Out of scope

- Resolver-side `resolution_failure` emit site — needs a resolver, tracked separately.
- `--no-optional` / `--omit=optional` plumbing — umbrella slice 5.
- Surfacing fetch failures to the reporter on the frozen path — upstream itself is silent here; only the resolver-side emit fires `pnpm:skipped-optional-dependency`.

Closes pnpm#471.
pull Bot pushed a commit to dwongdev/pnpm that referenced this pull request May 14, 2026
…e 5) (pnpm#485)

Slice 5 of the [pnpm#434 umbrella](pnpm/pacquet#434) (`Proper optionalDependencies support`). Slices 1–4 (pnpm#439, pnpm#456, pnpm#467, pnpm#474) handle installability + the fetch-failure swallow. This slice closes the user-facing `--no-optional` flag: the CLI flag already exists and threads through `Install::dependency_groups`, but only `SymlinkDirectDependencies` and the on-disk `included` field actually honored it. The rest of the install pipeline walked `optional_dependencies` unconditionally, so the optional subtree was still extracted, linked, and built.

Mirrors upstream's depNode filter at [`installing/deps-installer/src/install/link.ts:109-111`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/src/install/link.ts#L109-L111):

```ts
if (!opts.include.optionalDependencies) {
  depNodes = depNodes.filter(({ optional }) => !optional)
}
```

## Changes

- **`SkippedSnapshots`**: gains a third disjoint subset `optional_excluded` alongside `installability` (slice 1) and `fetch_failed` (slice 4). The existing `contains` / `iter` already return the union, so every downstream consumer that filters by skip-set (`CreateVirtualStore`, `SymlinkDirectDependencies`, `BuildModules`, hoist, `link_bins`) now respects `--no-optional` through the same gate. `iter_installability` still returns only the persistent subset for `.modules.yaml.skipped` writes — `--no-optional` exclusions are transient (matching upstream's behavior of never updating `opts.skipped` at the filter site).
- **`InstallFrozenLockfile::run`**: iterates the lockfile snapshots once and inserts every `snap.optional == true` entry into the new subset when the dispatch list doesn't contain `DependencyGroup::Optional`. The `SnapshotEntry::optional` flag is set by the resolver only when the snapshot is reachable **exclusively** through optional edges, so a snapshot reachable through any non-optional edge carries `optional: false` and survives the filter — covers [`optionalDependencies.ts:712`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/test/install/optionalDependencies.ts#L712) (`dependency that is both optional and non-optional is installed`).
github-actions Bot pushed a commit to Eyalm321/pnpm that referenced this pull request May 18, 2026
…pm#434 slice 3) (pnpm#467)

## Summary

Slice 3 of the [pnpm#434 umbrella](pnpm/pacquet#434) (`Proper optionalDependencies support`). Slice 1 (pnpm#439) computed a `SkippedSnapshots` set on every frozen-lockfile install but never serialized it; slice 2 (pnpm#456) closed the input side. This PR closes the loop by mirroring three upstream sites pinned at [pnpm/pnpm@94240bc046](https://github.com/pnpm/pnpm/tree/94240bc046):

- **Write** — `Modules.skipped` was declared at [`crates/modules-yaml/src/lib.rs:197`](https://github.com/pnpm/pacquet/blob/97010013/crates/modules-yaml/src/lib.rs#L197) with sort-on-write but always serialized empty. `build_modules_manifest` now takes `&SkippedSnapshots` and produces the depPath string list pnpm writes at [`installing/deps-installer/src/install/index.ts:1625`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/src/install/index.ts#L1625).

- **Seed** — `install_frozen_lockfile::run` reads `.modules.yaml.skipped` before computing the in-memory set and passes the parsed `PackageKey`s as the seed to `compute_skipped_snapshots`. Mirrors upstream's load at [`installing/read-projects-context/src/index.ts:79`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/read-projects-context/src/index.ts#L79) plus the early return at [`deps/graph-builder/src/lockfileToDepGraph.ts:194`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts#L194).

- **Short-circuit** — `compute_skipped_snapshots` returns the seed verbatim on the fast path (no constraints in the lockfile) and, on the slow path, skips the per-snapshot re-check for keys already in the seed without emitting a duplicate `pnpm:skipped-optional-dependency` event. Non-seeded snapshots still re-run `package_is_installable`, covering the \"host arch changed since last install\" case upstream guards at [`:206-215`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts#L206-L215).

`InstallFrozenLockfile::run` now returns a small `InstallFrozenLockfileOutput { hoisted_dependencies, skipped }` struct so `Install::run` can thread both into `.modules.yaml`. Read errors on `.modules.yaml` degrade to an empty seed — the file is a cache artifact, not authoritative.
github-actions Bot pushed a commit to Eyalm321/pnpm that referenced this pull request May 18, 2026
…e payload (pnpm#434 slice 4) (pnpm#474)

Slice 4 of the [pnpm#434 umbrella](pnpm/pacquet#434) (`Proper optionalDependencies support`). Slices 1–3 (pnpm#439, pnpm#456, pnpm#467) close the installability-driven skip path; this slice adds the second skip pathway upstream handles in the frozen install: an `optional: true` snapshot whose **fetch / extract** step fails at install time must not abort the install. Local-materialization errors (`CreateVirtualDir`) and config-shape errors still abort even for optional snapshots — matching upstream's `linkPkg` path which sits outside the catch.

Mirrors the silent catch sites at [`deps/graph-builder/src/lockfileToDepGraph.ts:294-298`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts#L294-L298) and [`installing/deps-restorer/src/index.ts:912-921`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L912-L921):

```ts
try { ... fetch ... }
catch (err) { if (pkgSnapshot.optional) return; throw err }
```

## Changes

- **Reporter**: `SkippedOptionalPackage` becomes a `#[serde(untagged)]` enum with `Installed { id, name, version }` and `ResolutionFailure { name?, version?, bareSpecifier }` variants. Models upstream's discriminated union at [`core-loggers/src/skippedOptionalDependencyLogger.ts:10-31`](https://github.com/pnpm/pnpm/blob/94240bc046/core/core-loggers/src/skippedOptionalDependencyLogger.ts#L10-L31). The new variant is wire-shape-only in slice 4 — pacquet has no resolver yet, but a future resolver port at the upstream emit site ([`resolveDependencies.ts:1376-1383`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-resolver/src/resolveDependencies.ts#L1376-L1383)) lands without re-touching this type.
- **`SkippedSnapshots`**: gains a `fetch_failed` subset disjoint from the existing `installability` subset. `contains` / `iter` return the union (downstream consumers treat both as absent); `iter_installability` returns only the persistent subset that `.modules.yaml.skipped` records, matching upstream's behavior of never updating `opts.skipped` at the fetch-failure catch sites.
- **`CreateVirtualStore`**: cold-batch dispatch swallows per-snapshot errors when `snapshot.optional` is true **and** the error is fetch-side. A new `is_fetch_side_failure` helper restricts the swallow to `InstallPackageBySnapshotError::DownloadTarball` and `::GitFetch` — the same surface upstream wraps in `storeController.fetchPackage`. Local-materialization (`CreateVirtualDir`) and config-shape errors (`MissingTarballIntegrity` / `UnsupportedResolution`) propagate even for optional snapshots, matching upstream's `linkPkg` path which sits outside the catch. Side benefit: the warm-batch path (which only surfaces `CreateVirtualDir` errors) stays consistently abort-on-error without a parallel swallow site. The per-install `fetch_failed: HashSet<PackageKey>` rides out on `CreateVirtualStoreOutput`.
- **`InstallFrozenLockfile::run`**: folds the returned `fetch_failed` into its mutable `SkippedSnapshots` so downstream phases (`SymlinkDirectDependencies`, `LinkVirtualStoreBins`, `BuildModules`, hoist) treat the dropped snapshots as absent through the same gate they already use for installability skips.

## Test plan

- [x] `reporter::tests::skipped_optional_resolution_failure_event_matches_pnpm_wire_shape` — full payload (`bareSpecifier` camelCase, no `id`).
- [x] `reporter::tests::skipped_optional_resolution_failure_omits_absent_name_and_version` — optional-field shape.
- [x] `install::tests::frozen_install_silently_swallows_unreachable_optional_tarball` — ports [`installing/deps-restorer/test/index.ts:340-360`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/test/index.ts#L340-L360): unreachable optional tarball (`http://127.0.0.1:1/...`), install succeeds, slot not created, `.modules.yaml.skipped` left empty. **Test-the-test verified** by inverting the `optional` guard in the cold-batch match arm. Runs with `enable_global_virtual_store = false` so the slot-path assertion targets the legacy layout.
- [x] `install::tests::frozen_install_propagates_non_optional_fetch_failure` — pins the polarity: same fixture, non-optional snapshot, install must error.
- [x] `just ready` (typos + fmt + check + nextest + clippy).
- [x] `taplo format --check`.
- [x] `just dylint` (perfectionist).
- [x] `RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --workspace`.

## Out of scope

- Resolver-side `resolution_failure` emit site — needs a resolver, tracked separately.
- `--no-optional` / `--omit=optional` plumbing — umbrella slice 5.
- Surfacing fetch failures to the reporter on the frozen path — upstream itself is silent here; only the resolver-side emit fires `pnpm:skipped-optional-dependency`.

Closes pnpm#471.
github-actions Bot pushed a commit to Eyalm321/pnpm that referenced this pull request May 18, 2026
…e 5) (pnpm#485)

Slice 5 of the [pnpm#434 umbrella](pnpm/pacquet#434) (`Proper optionalDependencies support`). Slices 1–4 (pnpm#439, pnpm#456, pnpm#467, pnpm#474) handle installability + the fetch-failure swallow. This slice closes the user-facing `--no-optional` flag: the CLI flag already exists and threads through `Install::dependency_groups`, but only `SymlinkDirectDependencies` and the on-disk `included` field actually honored it. The rest of the install pipeline walked `optional_dependencies` unconditionally, so the optional subtree was still extracted, linked, and built.

Mirrors upstream's depNode filter at [`installing/deps-installer/src/install/link.ts:109-111`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/src/install/link.ts#L109-L111):

```ts
if (!opts.include.optionalDependencies) {
  depNodes = depNodes.filter(({ optional }) => !optional)
}
```

## Changes

- **`SkippedSnapshots`**: gains a third disjoint subset `optional_excluded` alongside `installability` (slice 1) and `fetch_failed` (slice 4). The existing `contains` / `iter` already return the union, so every downstream consumer that filters by skip-set (`CreateVirtualStore`, `SymlinkDirectDependencies`, `BuildModules`, hoist, `link_bins`) now respects `--no-optional` through the same gate. `iter_installability` still returns only the persistent subset for `.modules.yaml.skipped` writes — `--no-optional` exclusions are transient (matching upstream's behavior of never updating `opts.skipped` at the filter site).
- **`InstallFrozenLockfile::run`**: iterates the lockfile snapshots once and inserts every `snap.optional == true` entry into the new subset when the dispatch list doesn't contain `DependencyGroup::Optional`. The `SnapshotEntry::optional` flag is set by the resolver only when the snapshot is reachable **exclusively** through optional edges, so a snapshot reachable through any non-optional edge carries `optional: false` and survives the filter — covers [`optionalDependencies.ts:712`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/test/install/optionalDependencies.ts#L712) (`dependency that is both optional and non-optional is installed`).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants