This should be a gentler way to introduce CET-compiled binaries into general usage: the "permissive" mode will silently disable CET when a non-CET-compiled .so rather than throwing an error.

While I understand the immediate effect, what's the rationale behind that? That this currently produces e.g. libs without CET if built on a builder that's too old (AFAIU --enable-cet=auto is the default for gcc)?

It's not just libs that were built on older systems, it's that we're probably going to have to disable CET on a bunch of packages that get broken by the funny stack shenanigans that CET performs (JITs maybe?). The default "hard fail" mode is probably stronger than most people are going to tolerate on a normal system.

Granted we're in a slightly better position than most distributions because at the time we're building an executable, we (mostly) know exactly what libs it's going to be linked to, so in theory we could put together some kind of mechanism that could check if any of the closure's packages have had CET disabled - but I see little point in putting such a build-time mechanism together when a runtime one is provided.

(FWIW other distributions have already started building packages with CET enabled, but I think that was probably a mistake on their part, as they've presumably not been able to test them properly until very recently and won't have been able to discover the ones that CET breaks.. suggesting there will be a lot of packages floating around in their ecosystem marked as "CET enabled" which may well break when CET mode is actually enabled)

cc @ajs124 who originally implemented that.

It's not just libs that were built on older systems, it's that we're probably going to have to disable CET on a bunch of packages that get broken by the funny stack shenanigans that CET performs (JITs maybe?). The default "hard fail" mode is probably stronger than most people are going to tolerate on a normal system.

I'm not sure I fully understand yet why this is a "new" problem: is it an existing problem or something new (given that "they've presumably not been able to test them properly until very recently")? in that case, glibc 2.39 + linux kernel >=6.6 is the cause? Asking because that might be relevant on whether we should schedule the upcoming glibc update together with this for instance.

glibc 2.39 + linux kernel >=6.6 is the cause?

That and the fact that we haven't been building CET-enabled packages so far - I haven't added the hardening flag for that yet. So there's no specific deadline for this, but it's another thing I'd like in place before I go adding hardening flags.

cc @ajs124 who originally implemented that.

Don't have any additional insight, but trust you to do the right thing.

Does this also affect dependencies of a program where this needs to be disabled?
The glibc INSTALL section only mentions the opposite case when using --enable-cet=permissive, i.e. that a CET-enabled program must not dlopen a non-CET program, so I’m not sure

That is a detail I had missed - I think you're right and that it's only for dlopen. I guess the idea is that libs you're linked against can probably be trusted, but the "immediate failure" is to protect against a program being tricked into opening a .so without CET support and disabling the protection.

Still, I'd argue that our preferred mode should be permissive, at least to start with.

A little bit more detail on support in various components here: https://lpc.events/event/7/contributions/729/attachments/496/903/CET-LPC-2020.pdf

That and the fact that we haven't been building CET-enabled packages so far

OK, that was part of what confused me, sorry!
I misread in https://gcc.gnu.org/install/configure.html that --enable-cet=auto being the default implies CET being enabled on our end. After rebuilding pkgs.hello with -fcf-protection and comparing the results with diffoscope I realized that this is clearly not the case.

My confusion came from the wrong assumption that part of this was already in place and thus at least people with 6.6 should've noticed issues already if any. Given that this is not the case, I agree with your assessment 👍

Might I be able to get an approval for this?

@risicle just to confirm -- with the mode set to permissive, CET-enabled programs are still able to dlopen non CET-enabled programs, correct? As an example, setting the mode to permissive won't break any programs loading proprietary libraries that are not CET-enabled (like CUDA).

That is my understanding, yes. This mode is less strict than its current value, so it shouldn't "break" anything.

I'll probably file another trivial glibc change later (#303193). Would consider merging them in one round since both are rebuilds of the same (large) package.