@thefloweringash Also pinging you since you did the initial work on the 11.0 SDK.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/902

LGTM

Only part I wonder about is why libobjc and Libsystem were gotten from the source releases before, I think that's the biggest change?

It’s conditional based on the platform. On aarch64-darwin, they come from the SDK. What this change does is always get them from the 11.0 SDK regardless (while keeping the defaults for each platform as-is).

# Pick the source of libraries: either Apple's open source releases, or the
# SDK.
useAppleSDKLibs = stdenv.hostPlatform.isAarch64;

selectAttrs = attrs: names:
  lib.listToAttrs (lib.concatMap (n: if attrs ? "${n}" then [(lib.nameValuePair n attrs."${n}")] else []) names);

chooseLibs = (
  # There are differences in which libraries are exported. Avoid evaluation
  # errors when a package is not provided.
  selectAttrs (
    if useAppleSDKLibs
      then apple_sdk
      else appleSourcePackages
  ) ["Libsystem" "LibsystemCross" "libcharset" "libunwind" "objc4" "configd" "IOKit"]
) // {
  inherit (
    if useAppleSDKLibs
      then apple_sdk.frameworks
      else appleSourcePackages
  ) Security;
};

Just thinking about EOL dates, planning for this seems like a very reasonable stopgap between now and the next stable release. Since 10.15 is going to be EOL this fall, macOS 11 will be the oldest supported release at that time. I know that the tentative plan for #101229 is to try to get SDKs bumped by the fall nixos release, and if it's feasibly to get that up to macOS 11 by then, this workaround may have a max shelf life of something like 6 months and will be a chance to get all the stuff that's been lagging on darwin up to speed in advance of that bump (getting Qt 5.15 and 6 stuff settled, anything using go, etc)

We're not bumping that far. And I'm not entirely sure we even should keep up with Apple's support schedule. That leaves a lot of hardware out there without access to Nixpkgs and no tangible advantage for users on more recent systems.

The current idea is to keep the default SDK recent-ish (emphasis on the ish) and provide ways to build with newer SDKs only for packages that really need it.

Makes sense — sorry for the confusion!

One sensible test that someone on aarch64-darwin should run is whether this has any effect on bootstrapping stdenv.

One sensible test that someone on aarch64-darwin should run is whether this has any effect on bootstrapping stdenv.

What would be the best way to do that? When I try to build stdenv on aarch64-darwin after applying the change, nix still gets it from the cache.

I’m running nix store gc right now to clear things out, so I can disable substituters (--option substituters '' and have it rebuild as much as possible, but that’s going to take some time.

After running nix store gc, and trying to rebuild stdenv, I get an error.

$ nix build --rebuild .#stdenv --option substituters ''
error: derivation '/nix/store/f6d7lwrknzcwbsjds36sgfl6cqpp82qy-bootstrap-stage0-clang-wrapper-11.1.0.drv' requires non-existent output 'out' from input derivation '/nix/store/ns4bvhwbwjjwizgvm4h0yf5gz2p7x7mf-bash-5.1-p16.drv'

Running this on master, I also get a (different) error.

$ nix build --rebuild github:NixOS/nixpkgs#stdenv --option substituters ''
error: derivation '/nix/store/4xfgcw1ajb12hjgvqa2rxz2m88w6ab6x-clang-11-12-LLVMgold-path.patch.drv' requires non-existent output 'lib' from input derivation '/nix/store/rd66ahrsdnghs9yjibscm7rrxlw09afq-llvm-11.1.0.drv'

@reckenrode, does the new build command have a similar option to --check? That should normally rebuild rather than substitute even if it's already in the store.

Those errors don't look good though, it seems that stdenv doesn't even build on master?

@reckenrode, does the new build command have a similar option to --check? That should normally rebuild rather than substitute even if it's already in the store.

I think --rebuild does the same thing. It rebuilds the derivation then compares it to the original. That succeeds; but, as far as I know, it’s not doing a bootstrap of stdenv (because none of the inputs changed).

Those errors don't look good though, it seems that stdenv doesn't even build on master?

That’s the way I read it, assuming disabling substituters is a supported scenario. Is there a better way of forcing a bootstrap of stdenv? I tried changing the name of bootstrapTools in its derivation this morning, and stdenv is rebuilding now, but that seems kind of hacky.

Following up: the hacky approach worked (changed name = "bootstrap-tools"; to name = "bootstrap-tools-foo";. The aarch64-darwin stdenv rebuilt without issue.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-macos-monthly/12330/31

Can you use sdk 11 in a nix-shell that only has stdenv.mkDerivation?

https://discourse.nixos.org/t/remove-entries-from-nix-cflags-compile/20728

ipetkov/crane#325 (comment)

problem x86_64-darwin with rust