Skip to content

fix(npm-resolver): request full metadata for optional dependencies#10455

Merged
zkochan merged 7 commits into
pnpm:mainfrom
3w36zj6:feature/request-full-metadata-for-optional-dependencies
Jan 26, 2026
Merged

fix(npm-resolver): request full metadata for optional dependencies#10455
zkochan merged 7 commits into
pnpm:mainfrom
3w36zj6:feature/request-full-metadata-for-optional-dependencies

Conversation

@3w36zj6

@3w36zj6 3w36zj6 commented Jan 14, 2026

Copy link
Copy Markdown
Contributor

close #9950

Biome correctly declares the os, cpu, and libc fields in each platform specific package1.

For context, the npm registry provides two types of metadata responses. Full metadata (application/json) includes the libc field, while Abbreviated metadata (application/vnd.npm.install-v1+json) does not.

Since pnpm uses abbreviated metadata by default, the libc field may be undefined. As a result, the check for libc in checkPlatform() can be skipped, making it impossible to distinguish between musl and glibc. To address this, adding a flag to request fullMetadata for each registry request allows pnpm to consistently obtain complete platform information and resolve optionalDependencies correctly. This change will prevent unsupported versions, such as @biomejs/cli-linux-x64-musl, from being erroneously downloaded on glibc systems.

You can verify the behavior by clearing pnpm's cache and running pnpm install in the directory containing the following package.json. The examples below were tested on Ubuntu 24.04.

{
  "name": "sandbox-pnpm",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "packageManager": "pnpm@11.0.0-alpha.2",
  "type": "module",
  "devDependencies": {
    "@biomejs/biome": "2.3.11"
  }
}

Before:

❯ pnpm install
Packages: +3
+++
Downloading @biomejs/cli-linux-x64@2.3.11: 16.70 MB/16.70 MB, completed
Downloading @biomejs/cli-linux-x64-musl@2.3.11: 16.71 MB/16.71 MB, completed
Progress: resolved 9, reused 0, downloaded 3, added 3, completed

devDependencies:
+ @biomejs/biome 2.3.11

Completed in 8.7s using pnpm v11.0.0-alpha.2

After:

❯ /path/to/pnpm/pnpm/bin/pnpm.mjs install
Packages: +2
++
Progress: resolved 9, reused 0, downloaded 2, added 2, completed

devDependencies:
+ @biomejs/biome 2.3.11

Downloading @biomejs/cli-linux-x64@2.3.11: 16.70 MB/16.70 MB, completed
Completed in 5.1s using pnpm v11.0.0-alpha.2

Footnotes

  1. https://github.com/biomejs/biome/issues/7442#issuecomment-3265414662

Copilot AI review requested due to automatic review settings January 14, 2026 01:59
@3w36zj6 3w36zj6 requested a review from zkochan as a code owner January 14, 2026 01:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes an issue where pnpm was incorrectly downloading both glibc and musl versions of optional dependencies on glibc systems due to missing libc field information in abbreviated metadata.

Changes:

  • Added support for requesting full metadata for optional dependencies to obtain the libc field needed for proper platform filtering
  • Modified cache key generation to include metaDir to prevent collisions between abbreviated and full metadata
  • Propagated the optional flag from package requester through to the npm resolver

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file
File Description
resolving/resolver-base/src/index.ts Added optional field to ResolveOptions interface
resolving/npm-resolver/test/resolveFromNpm.test.ts Added comprehensive integration tests for full metadata usage with optional dependencies
resolving/npm-resolver/test/pickPackage.test.ts Added unit tests for cache key separation and fullMetadata flag propagation
resolving/npm-resolver/src/pickPackage.ts Modified cache key to include metaDir and updated fetch call to pass fullMetadata flag
resolving/npm-resolver/src/index.ts Added logic to use FULL_META_DIR for optional dependencies and pass optional flag through context
resolving/npm-resolver/src/fetch.ts Added fullMetadata parameter to fetch function signature
pkg-manager/package-requester/test/index.ts Added test to verify optional flag is passed to resolve function
pkg-manager/package-requester/src/packageRequester.ts Modified to pass optional flag from WantedDependency to resolve options
network/fetching-types/src/index.ts Added fullMetadata field to FetchFromRegistry options
network/fetch/src/fetchFromRegistry.ts Modified to use per-request fullMetadata option with fallback to default

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@3w36zj6 3w36zj6 force-pushed the feature/request-full-metadata-for-optional-dependencies branch from 581c3fb to 0bc4181 Compare January 14, 2026 02:09
@zkochan

zkochan commented Jan 16, 2026

Copy link
Copy Markdown
Member

TBH, I hate this solution but I don't have better ideas at the moment.

Comment on lines +198 to +200
const resolveResult = await ctx.requestsQueue.add<ResolveResult>(async () => ctx.resolve(wantedDependency, {
...options,
optional: wantedDependency.optional,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are you adding optional to options if it is already passed in via wantedDependency in the first argument?

Comment thread resolving/npm-resolver/src/index.ts Outdated
Comment on lines +218 to +219
metaDir: opts.fullMetadata ? (opts.filterMetadata ? FULL_FILTERED_META_DIR : FULL_META_DIR) : ABBREVIATED_META_DIR,
metaDir: defaultMetaDir,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like it will be better to have two fields. metaDir and metaDirForOptionalPackages. The code will be simpler.

@3w36zj6 3w36zj6 force-pushed the feature/request-full-metadata-for-optional-dependencies branch 3 times, most recently from 6574b3c to a7add93 Compare January 23, 2026 03:12
@3w36zj6 3w36zj6 requested a review from Copilot January 23, 2026 04:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread resolving/npm-resolver/src/index.ts Outdated
@3w36zj6 3w36zj6 force-pushed the feature/request-full-metadata-for-optional-dependencies branch from a7add93 to 75028dc Compare January 23, 2026 06:44
@3w36zj6 3w36zj6 requested a review from Copilot January 23, 2026 06:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@3w36zj6 3w36zj6 requested a review from zkochan January 23, 2026 08:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@zkochan zkochan merged commit bb8baa7 into pnpm:main Jan 26, 2026
18 checks passed
@welcome

welcome Bot commented Jan 26, 2026

Copy link
Copy Markdown

Congrats on merging your first pull request! 🎉🎉🎉

zkochan added a commit that referenced this pull request Jan 26, 2026
…10455)

close #9950

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pnpm install fetch all binaries ignoring os / arch

3 participants