I ran into this while trying to get LLVM to eval under cross-compilation as part of #194634.

Apologies if this has already been addressed elsewhere; I know @Artturin has been working on fixes and cleanup for splicing recently but I didn't see this specific issue brought up in any of the existing issues and PRs.

Problem

As detailed in 888f966, packages that are in package sets within derivations (i.e. the psutil package within python3.pkgs where python3 is a derivation) do not get spliced.

This can be observed via __splicedPackages (I know this is meant to be an implementation detail but it is the attrset that top-level callPackage draws from and it's handy for debugging):

> np = (import <nixpkgs> { }).pkgsCross.aarch64-multiplatform

> np.__splicedPackages.hello ? __spliced
true

> np.__splicedPackages.python3Packages.psutil ? __spliced
true (!!)

> np.__splicedPackages.python3.pkgs.psutil ? __spliced
false

Note that accessing the python package set via python3Packages in the above does give you spliced packages.

This happens because of the way splicing operates on derivations:

Only the output attributes of a derivation are spliced; all other attributes are passed through as is.

Attrsets, on the other hand, are spliced recursively (on all attributes):

Tying this back to the example above, python3Packages (despite being defined as an alias to python3.pkgs) is spliced because the splice function is passed that attrset when recursively processing the top-level package attrset (whereas when the slice function encounters python3 it does not recurse into python3.pkgs because python3 is a derivation).

Potential Solutions

1: Have pkgs be already-spliced in such places

i.e.: instead of relying on the top-level splicing to splice such package sets, have this be the responsibility of the package set's scope.

Tangent: Splicing and Scopes

There's already some precedent for this; today scopes that use makeScopeWithSplicing are automatically given a newScope and a callPackage that contains spliced versions of the packages in the scope:

Here's how the python scope is set up, for example:

• the set of python overlays and the python package sets from the other pkgsBuildBuild, pkgsBuildHost, etc. sets are given to makeScopeWithSplicing; this yields the pythonPackage fixed point:
  

  nixpkgs/pkgs/development/interpreters/python/default.nix

  Lines 43 to 77 in 7f6ecd4

  	# Function that when called
  	# - imports python-packages.nix
  	# - adds spliced package sets to the package set
  	# - applies overrides from `packageOverrides` and `pythonPackagesOverlays`.
  	({ pkgs, stdenv, python, overrides }: let
  	pythonPackagesFun = import ./python-packages-base.nix {
  	inherit stdenv pkgs lib;
  	python = self;
  	};
  	otherSplices = {
  	selfBuildBuild = pythonOnBuildForBuild.pkgs;
  	selfBuildHost = pythonOnBuildForHost.pkgs;
  	selfBuildTarget = pythonOnBuildForTarget.pkgs;
  	selfHostHost = pythonOnHostForHost.pkgs;
  	selfTargetTarget = pythonOnTargetForTarget.pkgs or {}; # There is no Python TargetTarget.
  	};
  	hooks = import ./hooks/default.nix;
  	keep = lib.extends hooks pythonPackagesFun;
  	extra = _: {};
  	optionalExtensions = cond: as: if cond then as else [];
  	pythonExtension = import ../../../top-level/python-packages.nix;
  	python2Extension = import ../../../top-level/python2-packages.nix;
  	extensions = lib.composeManyExtensions ([
  	pythonExtension
  	] ++ (optionalExtensions (!self.isPy3k) [
  	python2Extension
  	]) ++ pythonPackagesExtensions ++ [
  	overrides
  	]);
  	aliases = self: super: lib.optionalAttrs config.allowAliases (import ../../../top-level/python-aliases.nix lib self super);
  	in makeScopeWithSplicing
  	otherSplices
  	keep
  	extra
  	(lib.extends (lib.composeExtensions aliases extensions) keep))

• that's then exported as a passthru on the python3 derivation as pkgs:
  

  nixpkgs/pkgs/development/interpreters/python/default.nix

  Line 97 in 7f6ecd4

  pkgs = pythonPackages;

Crucially, the spliced packages makeScopeWithSplicing produces are not made available in the actual scope (i.e. the package set; python3.pkgs in the above) itself but are made available to packages within the scope via the callPackage machinery.

This actually means that the contents of scopes like python3Packages (i.e. scopes that are exposed directly – not via a derivation – and are spliced at the top-level) are actually spliced "twice": once as part of the makeScopeWithSplicing call (accessed by members of the scope) and then again at the top-level (accessed by members of the outermost scope, the top-level).

It would be nice if we could reuse the splicing.

Using the outer scope's splicing within the inner scope is easy to do:

• we can:
  • pass in the path to the scope being constructed + __splicedPackages to this function
  • and then use __splicedPackages. ... in lieu of spliced0

But this is problematic in cases where the scope is overriden (i.e. overrideScope) outside of an overlay-like context (where the top-level binding of that scope is also updated). In such cases, the scope's callPackage will continue using the original pre-overrideScope version of the packages within the scope because the splicing would still be pulling things from the top-level __splicedPackages. Put another way: overlays = [(f: p: { python3Packages = p.python3Packages.overrideScope (_: _: { ... }); })] would be okay because nixpkgs.python3Packages is updated to point to the overriden scope but just doing (nixpkgs.python3Packages.overrideScope (_: _: { ... })).some-package would not be okay.

Going the other way (using the scope's splicing for the top-level) seems a little trickier but wouldn't have this issue. We'd need to have the makeScopeWithSplicing expose the spliced attrset with something like a __splicedPackages attr and we'd then want spliceReal's handling of attrset to check for such an attr and use it instead of redoing the splicing itself. I don't think this runs into any recursion issues but I have not tested this yet.

Ultimately this (splicing scopes multiple times) is somewhat orthogonal to this issue but the above has some overlap with the potential solutions below and might influence a decision there.

Now that we know how scopes get spliced:

The most straight-forward way to have our scope yield an already-spliced pkgs attr is probably to get makeScopeWithSpliced to give us it's spliced attrset. We can modify makeScopeWithSpliced to expose the attrset as __splicedPackages = spliced (as discussed above) and then swap out this line:

for pkgs = pythonPackages.__splicedPackages.

We would also have to replicate this change for all other users of makeScopeWithSpliced that export their package set as part of a derivation's attrs.

2: Adjust spliceReal's handling of derivations

Recursing on all of the attributes of every derivation seems fraught but maybe it's safe to recurse on drv.passthru or an opt-in list of attributes (i.e. we could have spliceReal look for a passthru attr named __spliceRecurseAttrs on derivations) or maybe even just pkgs (since that seems to be the convention used).

This has the benefit of not requiring any changes from users of makeScopeWithSpliced and handling splicing for packages that are referenced via a derivation's attrs (depending on how general we adjust spliceReal to be on derivation attrs).

3: Discourage using package sets like python3.pkgs "directly"

(and instead push people to use python3Packages, lua5Packages, etc. in nixpkgs)

This seems suboptimal, both because this will be another thing that'd need to be enforced in nixpkgs to have cross work for packages and because the foo.pkgs.bar pattern (where foo is a derivation) seems pretty pervasive in nixpkgs (python, lua, perl, postgresql, etc.).

A version of option 2 (with the __splicedPackages attr for deduplicating the work of splicing scopes as a follow-up PR if it doesn't cause breakage) seems like the least-worst fix to me but I'm not particularly satisfied with any of these solutions; hopefully there's a more elegant solution that I'm missing 🤞.

cc: @Artturin