(Spurred by some conversation in #7212, so I decided to write down my thoughts.)

Currently, NixOS and Nixpkgs have a lot of areas in which the security story can improve. While I think we're well positioned to take care of many of these, it's going to take some work, and others we'll need to reach consensus on. (The Good News is that many other people have gone down this road, so we have some good things to learn from)

As a rough starting point, I have identified roughly 5 major areas to look for future enhancements, with my personal suggestions on what we might want to do.

Binary hardening

As inspired by #7212, currently, NixOS doesn't default to enabling any hardening capabilities in GCC. There are an array of things we can do here to mitigate exploitability, that we can either enforce as a GCC spec file, or as part of NIX_CFLAGS/gcc-wrapper. Here are a few of them:

• Static compile-time enhancements via -D_FORTIFY_SOURCE=2 to improve protections for various built in string/buffer functions.
• Position-independent-executables to help defend against things like ROP-based code exploitation attacks via -pie -fPIC (PIEs are essentially just shared-objects-in-disguise much like transformers are robots-in-disguise).
• -fno-strict-overflow protects against the compiler optimizing away arithmetic overflow tests, which can happen when it gets particularly aggressive about undefined behavior.
• Stack protection via -fstack-protector*.
  • The default -fstack-protector protects functions with calls to things like alloca or ones that allocate more than 8 bytes of stack data. This buffer size can be controlled via --ssp-buffer-size, for example --ssp-buffer-size=1 triggers on any stack allocations..
  • GCC 4.9 also supports -fstack-protector-strong, which implies more coverage than -fstack-protector by default.
  • There is also -fstack-protector-all which does this for all functions. I am pretty sure this implies --ssp-buffer-size=1, but I'm not 100%.
• PLT/GOT protection: we need -z relro to mark dynamic relocations by the dynamic linker as read-only after ld.so resolves them. However, we also need -z now in order to force non-lazy resolution of relocations so the linker resolves them all at startup: otherwise they are performed on demand, and not marked as read-only until resolved (by which point an attacker could have used an arbitrary write to overwrite the relocation).

Suggestion: default all expressions to -fstack-protector-all --ssp-buffer-size=1 -D_FORTIFY_SOURCE=2 -pie -fPIC -z relro -z now -fno-strict-overflow as part of cc/ld. In other words, the whole gamut. For example, mkDerivation could support disableHardeningOptions = true; to disable options in specific expressions.

Implications: actually, fairly large in theory. These all in conjunction can have a significant penalty to things like startup time (due to resolving dynamic symbols up front), or execution speed. In particular, -fstack-protect-all is going to be costly (I wouldn't say beyond utility, but probably a good 20% speed loss at least). Also, PIE is going to really, really hurt performance on i686-linux, because PIE steals a register from the pathetic 32bit register set. Just PIE alone will have a significant impact on 32bit users, and -fstack-protect-all will add insult to injury.

Really, only benchmarking these things will tell us. I'd estimate the hit for these doesn't matter for a significant amount of software (common things like systemd, coreutils, anything that is setuid, etc). But a lot of things that are sort of nebulous middle grounds can be discussed on a maintainer-by-maintainer basis, I suppose (e.g. networking services are both performance and security critical, so we may want something better here).

Binary determinism

This is issue #2281. I've had this on my queue to finish integrating for a while now; with my new machine this can hopefully be a reality soon enough... Unfortunately as we didn't make the cut for GSoC 2015, there won't be any sponsored work on this. There is still the issue of GCC PGO determinism too, which does not have a clear consensus, but the large majority of the work is elsewhere.

Suggestion: I get off my ass and merge this.

Service hardening

Currently, very few of our NixOS services try to take advantage of any security or isolation features that can be offered e.g. by systemd. Basic examples that are probably worthy of spreading around the tree:

• PrivateTmp, since a lot of services are probably fine with their own /tmp mount. (unless they do something weird like use it for cross service IPC, which is now handled by /run).
• DevicePolicy, which can restrict access to /dev (lots of them could get by with e.g. "closed")
• InaccessibleDirectories, ReadOnlyDirectories, and ReadWriteDirectories (for example, some would never need access to /home, while others like tarsnap could always restrict themselves to ReadOnlyDirectories=/ with a specific ReadWriteDirectories for the cache).
• NoNewPrivileges, which automatically sets prctl(PR_SET_NO_NEW_PRIVS) for daemons. Again, useful for services like tarsnap or logging services (but care is needed if e.g. things invoke setuid programs like ping).

Suggestion: We begin enhancing services with these and encouraging maintainers of modules to do the same. Honestly this can probably be done pretty easily and fairly incrementally by maintainers or any interested newcomers.

We should pay attention to upstream systemd units or units from other distributions here too, since they'll have figured out some of this, too.

Kernel enhancements

Kernel security enhancements mostly come from one thing and one thing only (IMO): grsecurity. The good news is that grsecurity support mostly works with my module, and of course it's possible to go out of band with your own custom linuxPackages.

One thing is that we don't currently offer prebuilt grsecurity packages. Hydra actually builds them, but there's no way for the module to automagically select the right one. This should be fixed. Futhermore, the binary builds and module need some clean up (e.g. RANDSTACK is actually completely useless for pre-built binaries since the random offset can be known by any attacker, so RANDSTACK should always imply a local kernel build.)

Also, my grsecurity module could use a bit of work. Unfortunately it's split up in several places due to needing build/module support, but I do think this could be cleaned up/refactored a bit.

There are other things to consider, too. For example, there are kernel patches floating around to do various other things; it would be really nice, for example, if we could have a patch to randomize the MAC address assigned by the kernel - this would be good for my laptop and could be controlled by a boot switch for the kernel.

Suggestion: Well, it mostly works I guess.

MAC/RBAC

NixOS currently doesn't offer any form of policy enforcement in the place of MAC systems. There are a lot to choose from, but it basically comes down to AppArmor vs SELinux. Right now there's nobody really supporting either, so in lieu of this, the support that does exist is geared towards AppArmor. AppArmor isn't as expressive as SELinux, but it's a hell of a lot simpler and the policies are far easier to maintain. I think this is pretty important to get people to write policies.

The good news is the actual infrastructure is there: apparmor is packaged and works. There are even NixOS modules for it, but nothing really uses it. This is pretty easy though: we can begin clipping apparmor policies from upstream packages and places like Ubuntu.

Suggestion: We should just start writing policies, and enforce AppArmor by default on NixOS. This is the best way to ensure people keep using it, IMO.

Sidenote: gradm

The story is better for grsecurity users: you get gradm which is a totally badass RBAC system. The unfortunate news is that last time I tried it, gradm was buggy on my 3.4 kernel, and I didn't take the time to find a way to reconcile things like wanting NixOS modules to declare RBAC policies with the self-learning mode. In practice it may just be best to instead have services that activate/deactivate the learning mode via systemd, or just do nothing at all and expect users to enforce it themselves.

Small-form categorized TODO list

• Binary hardening
  • Enhance gcc to support transparent hardening.
    • Option 1: Add these things to NIX_CFLAGS, might work OK.
    • Option 2: Hack GCC. Hardened Gentoo does not use spec files, but there is an overlay that patches GCC to automatically add the aforementioned options automatically. This does not imply -D_FORTIFY_SOURCE=2 as far as I can tell, but that one can be added to NIX_CFLAGS most likely or gcc-wrapper.
    • Note: -D_FORTIFY_SOURCE=2 is, IIRC, only available at -O2 or above, so that can be annoying if we don't deal with it in gcc-wrapper. Or we could just patch the stupid warnings it emits out of GCC, too.
  • Implement RELRO/NOW binding (-z relro & -z now), as well as -fno-strict-overflow.
  • Enable PIE. NOTE: Large 32bit performance implication.
  • Decide on stack protector settings. NOTE: Performance implications.
  • Default to hardened binaries and extend mkDerivation to allow opt-out.
• Binary determinism
  • Merge Preview: deterministic build #2281 upstream and make sure system_tarball_pc remains deterministic.
  • Q: What to do about GCC PGO?
• Service hardening
  • Enhance services, make them harder, better, faster, stronger (and hopefully more isolated).
  • We should cross reference our services with upstream ones; many times an upstream will have a better one, or a distro will (such as ArchLinux).
• Kernel enhancements (grsec)
  • Disable RANDSTACK on pre-built images (useless).
  • Cleanup layout; split between nixpkgs, build-support, nixos. Can probably be refactored.
  • Make the grsecurity module serve prebuilt images.
• MAC/RBAC
  • Cut over to AppArmor 2.9 by default.
  • Enable by default on NixOS.
  • Enhance policy coverage for services.
  • (gradm) Make sure it actually works.
  • (gradm) Decide on learning mode support (or none if we decide to give up here).

CC: @wizeman @domenkozar @peti @edolstra