stdenv: PURL fetcher introduction by h0nIg · Pull Request #454333 · NixOS/nixpkgs

h0nIg · 2025-10-21T20:30:13Z

#421125 was merged and reverted later, because of regressions.

the background is described here: #421125 (comment)

@wolfgangwalther outlined the conditions and would like to enhance CI - over time. This is a continuous approach, which is in line with packages which have been found to be defunct and which need a fix. There may be more packages which have problems and we would like to prevent further fallout by a feature flag (prevents accessing + inheritance of drv.src / drv.srcs).

packages list: #453322 (comment)
list of real broken packages: #453322 (comment)
broken packages fix (deferrable PR: #457769)

With the old PR + the broken&platform check fix from #453291 + the feature flag, we enable maintainers to gather experience with PURL and set appropriate information (e.g. jq example, where fetchurl is used instead of fetchFromGithub)

nix-repl> xx = (import /my/nixpkgs {config={derivationPURLInheritance = true;};})
nix-repl> xx.python3Packages.boto3.meta.identifiers
{
  cpeParts = { ... };
  possibleCPEs = [ ... ];
  purl = "pkg:github/boto/boto3@1.40.18";
  purlParts = { ... };
  purls = [ ... ];
  v1 = { ... };
}

nix-repl> xx = (import /my/nixpkgs {})
nix-repl> xx.python3Packages.boto3.meta.identifiers
{
  cpeParts = { ... };
  possibleCPEs = [ ... ];
  purlParts = { ... };
  purls = [ ... ];
  v1 = { ... };
}

Things done

Add a 👍 reaction to pull requests you find important.

h0nIg · 2025-10-21T20:37:17Z

we need to squash all commits later, just for transparency let's keep the commits separated in order to understand what has been changed.

raboof · 2026-03-03T21:34:55Z

i demonstrated which data can get extracted with this patch some time ago: #421125 (comment)

Thank you, it's very helpful to have a concrete example to talk about.

As you know (but repeating for new readers), the main current use case for purls is SBOMs, and there are two general techniques for extracting the dependency relationships from nix trees: evaluating the the nix sources 'in nix', and parsing the drvs from the nix store. Both have their downsides: evaluating 'in nix' currently makes some subtrees unreachable due to NixOS/nix#4677 , while parsing drv files misses useful information from the meta fields until something like #420575 or #466932 would happen.

AFAICT "the jury is still out" on which approach we will land on. Until that time, I think we should not yet merge this PR as-is, as its complex/controversial bits (the propagation between meta and src.meta) only seem useful/necessary for the 'in nix' approach. If we'd land on the drv parsing approach, this propagation is not necessary: I made a small PoC showing that at https://codeberg.org/raboof/nix-build-sbom/src/branch/purl-experiment . Of course that PoC is horrible for several reasons, I'm definitely not suggesting anyone should use that directly, but it does show there are approaches where it's possible to derive purls without having the meta propagation - so it might be premature to introduce it.

The introduction of meta.identifiers.purl/meta.identifiers.purls/drv.src.meta.identifiers.v1.purl(s) in this PR seems more directly useful: for example, for python312Packages.pixelmatch we currently infer something like pkg:generic/pixelmatch-py?vcs_url=https://github.com/whtsky/pixelmatch-py.git@v0.3.0 or pkg:github/whtsky/pixelmatch-py@v0.3.0 as purl, while in fact pkgs:pypi/pixelmatch:0.3.0 would be more accurate. I don't think that information can currently be reasonably inferred, and IMHO it would be great if we could encode that knowledge in meta. This would benefit us immediately for nix-eval-based construction of SBOMs, and also for drv-based construction once we land on a way to expose meta to such tools.

(I have not digested the design decisions of that part of this PR, but it seems to me it might help to discuss those changes in isolation in their own PR, which we might be able to merge more quickly?)

h0nIg · 2026-03-08T21:49:31Z

(I have not digested the design decisions of that part of this PR, but it seems to me it might help to discuss those changes in isolation in their own PR, which we might be able to merge more quickly?)

as agreed in our call together with @raboof, i removed the inheritance and its feature flag

pkgs/by-name/po/popt/package.nix

raboof · 2026-03-24T10:29:57Z

Since the transition to Github Actions we no longer have a CI which performs those checks so this needs to be done manually.

You can see the performance report in the PR summary page: https://github.com/NixOS/nixpkgs/actions/runs/18756048808?pr=454333#summary-53508514968

@wolfgangwalther that link is now dead. This PR was significantly simplified since - would it be possible to do another performance check? How would one do that?

wolfgangwalther · 2026-03-24T11:44:21Z

would it be possible to do another performance check? How would one do that?

The performance data is available in every CI run. Just click your way to the details page of the CI jobs listed at the bottom of the PR and scroll your way through.

raboof · 2026-03-24T12:03:09Z

would it be possible to do another performance check? How would one do that?

The performance data is available in every CI run. Just click your way to the details page of the CI jobs listed at the bottom of the PR and scroll your way through.

Gotcha, for the 'Eval Summary' job, so https://github.com/NixOS/nixpkgs/actions/runs/22830597798?pr=454333 - so no noticeable change in speed, and a size increase of 0.05%-0.3%.

raboof

Starting to look attractive! Some comments here and there.

raboof · 2026-03-24T12:06:37Z

doc/stdenv/meta.chapter.md

+
+### Package URL {#sec-meta-identifiers-purl}
+
+[Package-URL](https://github.com/package-url/purl-spec) (PURL) is a specification to reliably identify and locate software packages. Through identification of software packages, additional (non-major) use cases are e.g. software license cross-verification via third party databases or initial vulnerability response management. Package URL's shall default to the mkDerivation.src, as the original consumed software package is the single point of truth.


"Package URL's shall default to the mkDerivation.src" is no longer true 'in nixpkgs', but consuming tools are expected to make this association (at least for now). That seems like a good start to me.

src or srcs is implementation specific, the tool outside should not try to guess which one is the right approach. The interface is drv.meta.identifiers.purl and the implementation (the drv) should define the "where to search". I would like to keep this

I am in agreement that we shouldn't push tools to trying to guess. Especially with tools that try to be efficient and can parallelize things, it makes the concurrency more difficult. Imo, it would be great if drv.meta.identifiers.purl defaults to drv.src.meta.identifiers.purl. I know people would be concerned with eval performance.

raboof · 2026-03-24T12:15:13Z

doc/stdenv/meta.chapter.md

+This attribute contains an attribute set of all parts of the PURL for this package.
+
+* `type` mandatory [type](https://github.com/package-url/purl-spec/blob/18fd3e395dda53c00bc8b11fe481666dc7b3807a/docs/standard/summary.md) which needs to be provided
+* `spec` specify the PURL in accordance with the [purl-spec](https://github.com/package-url/purl-spec/blob/18fd3e395dda53c00bc8b11fe481666dc7b3807a/purl-specification.md)


why is this field named 'spec'? what's the motivation for splitting the 'type' and the rest of the purl like this, over 'just' having a purl string? Are you expecting more fields in the future?

what if all the PURL components are defined as attrs here? except the schema being a constant "pkg".

i splitted them to give it a thin structure overall

spec is the implementation of the definition schema: https://github.com/package-url/purl-spec/tree/18fd3e395dda53c00bc8b11fe481666dc7b3807a/types

doc/release-notes/rl-2605.section.md

pkgs/top-level/config.nix

doc/stdenv/meta.chapter.md

Eveeifyeve

This looks to good to me, however I would like to see the purl parts be automatically defined by fetchers as well in a followup.

h0nIg mentioned this pull request Oct 21, 2025

stdenv: pURL implementation #421125

Merged

13 tasks

github-project-automation bot added this to Stdenv Oct 21, 2025

h0nIg force-pushed the purl-featureflag branch from 18f4eea to 016c6e6 Compare October 21, 2025 20:58

h0nIg marked this pull request as ready for review October 21, 2025 21:04

h0nIg marked this pull request as draft October 21, 2025 21:04

nix-owners bot requested review from 06kellyjac, 7c6f434c, Artturin, Ericson2314, GetPsyched, RossComputerGuy, alyssais, emilazy, fricklerhandwerk, hsjobeki, ncfavier, philiptaron and reckenrode October 21, 2025 21:05

h0nIg force-pushed the purl-featureflag branch from 016c6e6 to c8afdf6 Compare October 21, 2025 21:14

h0nIg marked this pull request as ready for review October 21, 2025 21:14

h0nIg mentioned this pull request Oct 21, 2025

stdenv: pURL implementation - fix checkMeta #453291

Closed

13 tasks

nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 8, 2026

h0nIg force-pushed the purl-featureflag branch 4 times, most recently from 5528902 to 097b483 Compare March 8, 2026 21:28

nixpkgs-ci bot requested review from a team and infinisil March 8, 2026 21:34

h0nIg force-pushed the purl-featureflag branch 2 times, most recently from ca4dc6c to fed1a8d Compare March 8, 2026 21:47

h0nIg changed the title ~~stdenv: PURL fetcher introduction & feature flag~~ stdenv: PURL fetcher introduction Mar 8, 2026

Eveeifyeve reviewed Mar 21, 2026

View reviewed changes

pkgs/by-name/po/popt/package.nix Outdated Show resolved Hide resolved

h0nIg force-pushed the purl-featureflag branch from fed1a8d to cd3ff71 Compare March 24, 2026 12:32

nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 24, 2026

h0nIg force-pushed the purl-featureflag branch from cd3ff71 to 3c32661 Compare March 24, 2026 13:01

raboof reviewed Mar 24, 2026

View reviewed changes

nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 24, 2026

h0nIg force-pushed the purl-featureflag branch from 3c32661 to c458b18 Compare March 25, 2026 16:35

h0nIg added 2 commits March 25, 2026 16:37

stdenv: PURL fetcher introduction & feature flag

9134a05

stdenv: PURL fetcher introduction

c6035ec

h0nIg force-pushed the purl-featureflag branch from 0caddcd to c6035ec Compare March 25, 2026 16:37

h0nIg requested review from Eveeifyeve and raboof March 25, 2026 16:39

Eveeifyeve approved these changes Mar 31, 2026

View reviewed changes

oneingan requested review from oneingan and removed request for oneingan March 31, 2026 21:09


		### Package URL {#sec-meta-identifiers-purl}

		[Package-URL](https://github.com/package-url/purl-spec) (PURL) is a specification to reliably identify and locate software packages. Through identification of software packages, additional (non-major) use cases are e.g. software license cross-verification via third party databases or initial vulnerability response management. Package URL's shall default to the mkDerivation.src, as the original consumed software package is the single point of truth.

Uh oh!

Conversation

h0nIg commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Things done

Uh oh!

h0nIg commented Oct 21, 2025

Uh oh!

raboof commented Mar 3, 2026

Uh oh!

h0nIg commented Mar 8, 2026

Uh oh!

Uh oh!

raboof commented Mar 24, 2026

Uh oh!

wolfgangwalther commented Mar 24, 2026

Uh oh!

raboof commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

raboof left a comment

Choose a reason for hiding this comment

Uh oh!

raboof Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

h0nIg Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

RossComputerGuy Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

raboof Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

yyovil Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

h0nIg Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Eveeifyeve left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

18 participants

h0nIg commented Oct 21, 2025 •

edited

Loading

raboof commented Mar 24, 2026 •

edited

Loading

yyovil Mar 24, 2026 •

edited

Loading

h0nIg Mar 24, 2026 •

edited

Loading

Eveeifyeve left a comment •

edited

Loading