With #234710 getting close to done and the LLVM bump imminent, #229210 should be able to move forward adding new SDKs. We’ve had several conversations on Matrix regarding how we’d like to handle multiple SDKs. The purpose of this issue is to capture those conversations and work towards a consensus.

I’m pinging @toonn and @emilazy because we have discussed this on Matrix before. I am also pinging @jtojnar because @drupol mentioned having talked with him about the SDK issue. If there is anyone I forget, feel free to ping them.

Assumptions

• There need to be multiple SDKs. Unfortunately, if we want to build software that uses features on new platforms, we can’t just use a newer SDK by default because static feature detection may result in binaries that don’t run on older platforms.
• x86_64-darwin will continue to maintain an older default than aarch64-darwin. That is currently 10.12 but will need to increase to 10.13 because libc++ 17 has dropped support for 10.12.
• It should be easy to use in the simple case while also composing well with language and other complex frameworks.
• It should be possible to mix dependencies using different SDKs, and switching an SDK should not result in a complete rebuild of the stdenv just to use it. All stdenvs should share a common libc++.

Current Situation

There are two SDKs currently in nixpkgs: the 10.12 SDK and the 11.0 SDK. 10.12 is the default on x86_64-darwin, and 11.0 is the default on aarch64-darwin. The 10.12 SDK is built from Apple’s source releases where possible while the 11.0 SDK is derived from the upstream SDK. There is some shared implementation but also a lot of custom implementation between the two.

The default SDK is darwin.apple_sdk. Packages that need to access SDK packages typically use the package and the inherit (darwin) <package> pattern. To use a different SDK, the common pattern is to use apple_sdk_<version>.callPackage, which overrides some parts of Darwin to provide versions from the new SDK. However, explicitly inheriting packages also requires updating those to reference the new SDK.

Friction Points

The SDK typically ends up providing SDK-specific stdenvs and language frameworks (like rustPlatform). The SDK should not have to concern itself with how it is being used downstream, and those frameworks should just worked based on the chosen SDK (picking it up as well as the updated compilers).

xcbuild hardcodes the SDK it uses. It should be possible to override this in a way that does not require rebuilding xcbuild every time a new SDK is required.

propagatedBuildInputs do not work well with SDK overrides. For example, it is not possible to mix libGLU, which propagates a couple of SDK frameworks, in a package that needs a different SDK because it results in headers being used from the wrong SDK.

The top level of darwin attribute set contains some SDK-specific items (like Libsystem). These need to be moved to their appropriate SDKs. The top level should contain only packages that are Darwin-specific and part of or associated with an SDK.

CF is a massive pain in the ass. It should either be dropped or made to build on aarch64-darwin. Having two different build paths through the stdenv bootstrap complicates things. The way CF propagates also makes the stdenv bootstrap very sensitive to the order things are built on x86_64-darwin. There are also risks for cyclical dependencies¹.

Updating SDKs and making them source-based is difficult. This is going to be a problem for x86_64-darwin when it comes time to bump to clang 18 next year because libc++ will have dropped support for the platform, and additional post-10.12 APIs may be used (meaning it may not be feasible just to revert the commit that removed to 10.12 support).

¹: There are two examples I have encountered while working on the stdenv rework.

• CF depended on curl, which depends on CoreFoundation. The old stdenv avoided this issue by not including the rpath, causing nix not to find the dependency on the linked CF from a prior bootstrap stage. This was fixed in the new stdenv by removing the curl usage from CF. Fortunately, it is not needed in our usage because we do not build swift-corelibs Foundation.
• Bash depends on CF, but CF depends on libxml2 and ICU, which provide scripts that include shebang lines that need patched to use bash from nixpkgs. This worked in the old stdenv because only it used darwin.ICU, and libxml2 just happened to work with the system bash on Darwin. The reworked stdenv uses the nixpkgs icu, which caused problems with packages that used the script to get icu’s build flags. This was fixed by making bash depend on CoreFoundation instead of CF.