treewide: enable security hardening flags by TredwellGit · Pull Request #104091 · NixOS/nixpkgs

TredwellGit · 2020-11-17T20:45:13Z

Motivation for this change

https://blog.fpmurphy.com/2008/06/position-independent-executables.html

PIE is an address space randomization technique that compiles & links executables to be position independent, i.e. machine instruction code that executes properly regardless of where in memory it actually resides. When combined with a kernel that can recognize it is loading a PIE binary, the kernel loads it into a random address instead of the traditional fixed address locations.

https://fedoraproject.org/wiki/Security_Features_Matrix#Address_Space_Layout_Randomization_.28ASLR.29

This makes memory addresses harder to predict when an attacker is attempting a memory-corruption exploit.
To make ASLR effective all segments must be randomized. Leaving the text segment loading address non-randomized reduces the protection provided by the ASLR since the attackers can use ret2text attacks. The loading address of the text segment in a binary can be randomized by building the binary as PIE (Position Independent Executable).

https://fedoraproject.org/wiki/Changes/Harden_All_Packages#Detailed_Harden_Flags_Description

Copy relocations support in GCC 5 and binutils 2.26 makes the performance [decrease] on x86_64 of PIE literally zero for many programs.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)
- NixOS
Tested execution of all binary files (usually in ./result/bin/)
Ensured that relevant documentation is up to date
Fits CONTRIBUTING.md.

TredwellGit · 2020-11-17T21:30:23Z

Work in progress.

Most packages that need to be disabled are because they are required for bootstrapping and the compiler seems to not support PIE. Notably Perl and CPython included. This does not prevent merging, but should be dealt with.
What is the best way to build and test all packages? I have nearly rebuilt my personal system with nixos-rebuild build -I nixpkgs=..
Haskell is the only merge preventing issue that I have encountered. For example, nix-build -A haskellPackages.hscolour fails to build with errors like can not be used when making a PIE object; recompile with -fPIC. GHC supports PIE and PIC but I am unsure how to get it to work or disable it with how it is packaged. Would really appreciate help with this since it blocks this significant security feature for everything else. @MarcWeber @kosmikus @peti @lostnet
I am also investigating enabling stack clash protection. If this pull request is blocked for long by the Haskell issue, then I might just include that as well to prevent rebuilds.

TredwellGit · 2020-11-17T21:36:44Z

Wonder if issues are caused by this: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4287

lostnet · 2020-11-20T01:36:26Z

I've quite possibly missed conversation on this elsewhere, but if not, I think would be good to test with haskell.packages.ghc901.hscolour as the haskell commit can be easily applied as a patch to 9.0, i.e. I think this would be ok. But the first round of compiling hscolour would be in the bootstrap, using an 8.10.2 binary compiled for debian/ubuntu, so it would probably need to be compiling without pie in that phase.

I'm not really sure what it working in 9.0+patch would mean for the 8.X versions that are the current default as haskell.org seems to only have marked it for 9, also, I'm not optimistic for what will happen with aarch64 as it seems quite a bit more sensitive to relocation problems.. But some version(s) on some platform(s) would be a great start.

TredwellGit · 2020-11-20T06:49:38Z

Thanks for responding.

I've quite possibly missed conversation on this elsewhere

You did not.

But the first round of compiling hscolour would be in the bootstrap, using an 8.10.2 binary compiled for debian/ubuntu, so it would probably need to be compiling without pie in that phase.

That is the problem. I am unsure how to disable it because hardeningDisable = [ "pie" ]; is not respected in https://github.com/NixOS/nixpkgs/tree/staging/pkgs/development/compilers/ghc or https://github.com/NixOS/nixpkgs/tree/staging/pkgs/development/haskell-modules.

lostnet · 2020-11-20T11:40:04Z

That is the problem. I am unsure how to disable it because hardeningDisable = [ "pie" ]; is not respected in https://github.com/NixOS/nixpkgs/tree/staging/pkgs/development/compilers/ghc or https://github.com/NixOS/nixpkgs/tree/staging/pkgs/development/haskell-modules.

Booting the stage1 of the real ghc is probably going to need these settings for ghc itself not to fail as a PIE with the binary libraries(?), but before, when the bootstrap ghc is setting up its bootPkgs (and then after the real ghc is built and is building its package set) I think you need to override the haskell-modules side, which should go from make-package-set.nix to generic-builder

TredwellGit · 2020-11-21T21:32:04Z

Upon further investigation, I have determined that GHC's support for PIC and PIE is bad. I have it working with PIE totally disabled for Haskell; will push later with additional security hardening enabled.

utsl42 · 2020-11-25T23:54:27Z

See also #101666 (comment)
There were some past issues referencing ghc using gold, so that may have some relevance to what you're trying to do.

globin · 2021-03-05T00:26:31Z

Beware that this can and used to break a lot at runtime, that's why we decided to not enable it by default

TredwellGit · 2021-03-05T01:07:10Z

Does not seem to be that much of a problem. I am more concerned about how different CPUs will behave. Please read: #104091 (comment)

@vcunat Someone cancelled a significant part of the https://hydra.nixos.org/jobset/nixpkgs/hardening-flags jobset which I need to test some CPU differences. Will you please restart them?

vcunat · 2021-03-05T09:13:07Z

Done but that doesn't mean they will get finished soon. The build farm is often quite loaded.

FRidh · 2021-03-24T09:17:01Z

What is the status of this? Considering NixOS/rfcs#85 (comment), can this reasonably make 21.05? If not, I suggest we halt the Hydra job until after the release to gave space to other tasks.

FRidh · 2021-03-24T13:38:06Z

Aborted builds for now to give space to other jobsets.

vcunat · 2021-03-24T16:05:01Z

BTW, there's a plan for OpenSSL 1.1.1 high-severity release tomorrow.

TredwellGit · 2021-03-24T16:37:33Z

This should not be in 21.05. x86_64 Linux glibc is nearly entirely building but everything else needs testing. I was planning to work on x86_64 Linux musl so we can stop building this for a few days. What might be useful at this point is to switch Hydra to build on i686 Linux.

jonringer · 2021-03-25T19:33:36Z

this should also be targeting staging. Not sure if there's value in targeting master since it does an stdenv rebuild anyway, so you're not likely to get any cache hits.

TredwellGit · 2021-03-25T19:58:37Z

GitHub's UI is very bad because it repeatedly hides information. Here is why it is based on master: #104091 (comment)

vcunat · 2021-03-25T20:09:02Z

Well, the main problem isn't GitHub but the fact that staging sometimes contains regressions which then confuse what you see in the Hydra jobset.

TredwellGit · 2021-03-25T20:12:21Z

No, I mean that the GitHub pull request UI has "hidden items" that have to be repeatedly clicked on before you can search all past comments.

TredwellGit · 2021-04-01T09:12:11Z

Hydra x86_64 Linux seems available so I pushed an update. Would prefer if we could let this build with very low priority rather than cancelling it because otherwise it is difficult to work on.

TredwellGit · 2021-04-21T17:55:52Z

@vcunat, please add "i686-linux" to supportedSystems for https://hydra.nixos.org/jobset/nixpkgs/hardening-flags#tabs-configuration.

vcunat · 2021-04-21T18:49:35Z

I set up the default value, as i686 isn't (fully) supported anymore. Anyway, there's probably some evaluation problem, but on a quick look I can't see what it is.

https://blog.fpmurphy.com/2008/06/position-independent-executables.html >PIE is an address space randomization technique that compiles & links executables to be position independent, i.e. machine instruction code that executes properly regardless of where in memory it actually resides. When combined with a kernel that can recognize it is loading a PIE binary, the kernel loads it into a random address instead of the traditional fixed address locations. https://fedoraproject.org/wiki/Security_Features_Matrix#Address_Space_Layout_Randomization_.28ASLR.29 >This makes memory addresses harder to predict when an attacker is attempting a memory-corruption exploit. >To make ASLR effective all segments must be randomized. Leaving the text segment loading address non-randomized reduces the protection provided by the ASLR since the attackers can use ret2text attacks. The loading address of the text segment in a binary can be randomized by building the binary as PIE (Position Independent Executable). https://fedoraproject.org/wiki/Changes/Harden_All_Packages#Detailed_Harden_Flags_Description >Copy relocations support in GCC 5 and binutils 2.26 makes the performance [decrease] on x86_64 of PIE literally zero for many programs.

TredwellGit · 2021-04-26T01:39:50Z

i686 works without issue.
@vcunat, can we add "aarch64-linux" and/or "x86_64-darwin"?

vcunat · 2021-04-26T09:10:25Z

OK, they were quite free now so I added them, i.e. now it's full as the default nixpkgs/trunk jobset.

At some point it would be nice to separate the common (stdenv) changes from the package changes, I think, for better visibility.

Mic92 · 2021-05-24T06:14:13Z

With this PR also static-pie will work: #123989

r-burns · 2021-10-08T23:58:16Z

If this gets revived at some point (I hope it does!) be sure to take a second look at any packages which needed hardening disabled; many should be fixed by #135619.

r-burns · 2021-10-26T04:11:25Z

Oh, I just realized I could revive this myself since the hydra job is still active - I'd just have to push to this branch. Would that be all right with you? Don't want to step on any toes

TredwellGit · 2021-10-26T08:03:48Z

I am working on rebasing and writing documentation for this, but I have been hesitant to push because I do not want to overload Hydra given the work for 21.11.

vcunat · 2021-10-26T09:53:15Z

Since several days ago there's also the caveat that cancelling whole evaluations doesn't work (and restarting individual jobs doesn't work).

nixos-discourse · 2023-08-08T00:15:39Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/where-did-you-get-stuck-in-the-nix-ecosystem-tell-me-your-story/31415/6

Eveeifyeve

The diff looks good in my eyes.
This needs to be ran by hydra, so could someone hook this up.
This also needs a rebase.

ofborg bot added 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: stdenv Standard environment labels Nov 17, 2020

TredwellGit force-pushed the PIE branch from 4dcaf3e to 02c32bb Compare November 19, 2020 15:40

TredwellGit changed the title ~~treewide: enable position independent executables (PIE)~~ treewide: enable security hardening flags Nov 25, 2020

TredwellGit force-pushed the PIE branch from 02c32bb to 4426da5 Compare November 25, 2020 22:05

ofborg bot added the 6.topic: haskell General-purpose, statically typed, purely functional programming language label Nov 25, 2020

TredwellGit mentioned this pull request Nov 25, 2020

Fix hardening default for pkgsMusl to reenable -pie #101606

Merged

10 tasks

TredwellGit force-pushed the PIE branch from 4426da5 to b96e6d9 Compare November 25, 2020 22:17

TredwellGit force-pushed the PIE branch 3 times, most recently from 9028018 to 7c90efe Compare December 6, 2020 08:22

ofborg bot added 8.has: documentation This PR adds or changes documentation 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package labels Dec 6, 2020

TredwellGit mentioned this pull request Dec 8, 2020

glibc: 2.32-10 -> 2.32-24 #106324

Merged

3 tasks

TredwellGit force-pushed the PIE branch 2 times, most recently from a9ae7f2 to b4942cc Compare December 10, 2020 03:34

TredwellGit mentioned this pull request Mar 23, 2021

musl: 1.2.1 -> 1.2.2 #117375

Merged

3 tasks

FRidh mentioned this pull request Mar 24, 2021

hydra.nixos.org nixpkgs/nixos jobs coordination #117495

Closed

TredwellGit mentioned this pull request May 30, 2021

nix: enable pie #124187

Merged

9 tasks

r-burns mentioned this pull request Aug 28, 2021

pkgsMusl.libiscsi: fix build #135903

Merged

12 tasks

r-burns mentioned this pull request Sep 4, 2021

[staging] {cc,binutils}-wrapper: fixes for PIE hardening #135619

Merged

12 tasks

LunNova mentioned this pull request Dec 7, 2022

tracking issue: by default packages aren't built with -fpie/ASLR support #205031

Closed

6 tasks

Eveeifyeve suggested changes Apr 7, 2026

View reviewed changes

Uh oh!

Conversation

TredwellGit commented Nov 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation for this change

Things done

Uh oh!

TredwellGit commented Nov 17, 2020

Uh oh!

TredwellGit commented Nov 17, 2020

Uh oh!

lostnet commented Nov 20, 2020

Uh oh!

TredwellGit commented Nov 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lostnet commented Nov 20, 2020

Uh oh!

TredwellGit commented Nov 21, 2020

Uh oh!

utsl42 commented Nov 25, 2020

Uh oh!

globin commented Mar 5, 2021

Uh oh!

TredwellGit commented Mar 5, 2021

Uh oh!

vcunat commented Mar 5, 2021

Uh oh!

FRidh commented Mar 24, 2021

Uh oh!

FRidh commented Mar 24, 2021

Uh oh!

vcunat commented Mar 24, 2021

Uh oh!

TredwellGit commented Mar 24, 2021

Uh oh!

jonringer commented Mar 25, 2021

Uh oh!

TredwellGit commented Mar 25, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vcunat commented Mar 25, 2021

Uh oh!

TredwellGit commented Mar 25, 2021

Uh oh!

TredwellGit commented Apr 1, 2021

Uh oh!

TredwellGit commented Apr 21, 2021

Uh oh!

vcunat commented Apr 21, 2021

Uh oh!

TredwellGit commented Apr 26, 2021

Uh oh!

vcunat commented Apr 26, 2021

Uh oh!

Mic92 commented May 24, 2021

Uh oh!

r-burns commented Oct 8, 2021

Uh oh!

r-burns commented Oct 26, 2021

Uh oh!

TredwellGit commented Oct 26, 2021

Uh oh!

vcunat commented Oct 26, 2021

Uh oh!

nixos-discourse commented Aug 8, 2023

Uh oh!

Eveeifyeve left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

14 participants

TredwellGit commented Nov 17, 2020 •

edited

Loading

TredwellGit commented Nov 20, 2020 •

edited

Loading

TredwellGit commented Mar 25, 2021 •

edited

Loading