CodSpeed Performance Report

Merging #12357 will not alter performance

_{Comparing altendky-patch-3 (2b88f27) with trunk (4711c34)}

Summary

✅ 25 untouched benchmarks

@adiroiban, looks like you win probably. https://github.com/twisted/twisted/actions/runs/11713225981/job/32625578908?pr=12357 macos-14 is ok and macos-15 is failing consistently.

@altendky It looks like all the tests are UDP-on-localhost, failing with routing errors?

I think that this may have to do with the (new in Sequoia?) "local network" permission?

Okay, neat, if I do this:

I can reproduce this error immediately, and it looks just the same.

Not sure how we can manipulate the TCC database in github actions.

I'd heard a thing about some changes but I don't use macOS personally and hadn't heard any details about it. So I agree with your guess, for whatever little that's worth. :]

This PR gets testing going on macOS ARM at least. Bumping to 15 could be good when the necessary setup adjustments are figured out. Or maybe 'we' are supposed to start requesting permissions?

I am not sure how we could request these permissions from the command line. Apple intentionally makes this difficult, for obvious reasons, and I cannot find anyone writing about how to do this with Github Actions specifically: https://apple.stackexchange.com/questions/472592/grant-permissions-from-the-command-line

So, the problem here is that we are joining an arbitrary local-network multicast group, which actually blasts stuff out to the ethernet port, or at least could do that to the address it's connected to, which is why we need "local network" permission, which "localhost" normally wouldn't require. I started wondering if I could port this to IPv6 to use its loopback address which doesn't exist in IPv4, but then discovered that we super do not support IPv6 multicast.

Not sure what the right way forward is. If someone wants to ask some github actions support folks that might be a good next step, but I don't know how to reach them.

Testing on macOS 14 is definitely a step up though, so for sure not a blocker here.

Just leaving this here for future reference; I tried to adjust twisted/internet/udp.py to do https://stackoverflow.com/questions/3183194/send-and-receive-ipv6-link-local-multicast-udp-datagrams-in-python but messed it up somehow, so I should try again starting from there as an example.

What we can do for macos15 is to skip those tests ... some tests are just hard to setup on public testing infrastructure

caveat that that is a temporary 'solution' that's good until macos15 is the oldest. looks like that will be ballpark two years.

What we can do for macos15 is to skip those tests ... some tests are just hard to setup on public testing infrastructure

caveat that that is a temporary 'solution' that's good until macos15 is the oldest. looks like that will be ballpark two years.

I think that skipping in the specific case of macOS 15 is a good starting point here. When we try to add 16, then we will see the issue again which will prompt us to fix that, and we will have prompts at both 16 and 17 before 15 disappears :). If we implement #6597 then we may be able to sidestep the need for the permission, and get more or less equivalent coverage, only skipping IPv4.

I filed a ticket upstream for this. actions/runner-images#10924

@altendky I sneakily put in a capability-detection check in #12360 which should make it possible to enable macOS 15 without any further work. We won't be testing multicast on macOS unless github grants that permission, but I think that's fine; the API doesn't actually differ between platforms, so there's no additional code to cover.

#12363