guix-daemon socket activation does not work on the hurd

Hi,

today i reconfigured my system and after a reboot I am unable to use the guix-daemon on a childhurd.

guix build hello -n

guix build: error: failed to connect to `/var/guix/daemon-socket/socket': Protocol error

Offloading:

guix offload: error: failed to connect over SSH to daemon at 'localhost', socket /var/guix/daemon-socket/socket

Daemon Logs:

socket-activated with 1 socket

unexpected build daemon error: reading from file: Resource temporarily unavailable

Starting the daemon as the root user normally continues to work as before so i suspect the socket activation change is to blame.

Guix commit: 6af680670bf9055b90e6f8b63c4c2ab7b08e7c56

After mentioning this on IRC Ludovic pushed 8d31cafbdcb818160852a5d1e6fc24c1a9c53e41 to the shepherd repo.

I wanted to try this out and reconfigured using the shepherd from this commit as pid1 in the vm (a bit tricky because of help2man).

The first connection still fails in the same way.unexpected build daemon error: reading from file: Resource temporarily unavailable

A client mentions:

guix build: error: corrupt input while restoring archive from #<closed: file 2396ea8>

However subsequent connections work.

yelninei--- via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

I looked a bit into this, and I think shepherd is doing the right

working as expected, making the socket blocking before executing

guix-daemon (it’s clear when stracing it on Linux).

So there must be something specific at play on the Hurd.

I tried this snippet (server on one side, client on the other side) and

it works as expected: ‘accept’ blocks and subsequent read does not get

EAGAIN.

So I’m at loss here. Does ‘tests/systemd.sh’ succeed when ran natively?

(In particular the check added in

8d31cafbdcb818160852a5d1e6fc24c1a9c53e41.)

Thanks,

Ludo’.

Hello,

Apr 15, 2025, 16:08 by ludo@gnu.org:

Yes, it is passing both on 1.0.3 and 1.0.4. The only thing failing now is the system-log test.

As before when using #:lazy-start #f it works as expected which makes the only difference the timing of the first connection. What would the most minimal guix-daemon client need to look like to trigger the EAGAIN

 

I tried to verify that the port is definitly blocking before being passed to guix-daemon and it is. I am very confused.

Do you know of other processes (with not a lot of dependencies) that can be socket activated to try to replicate this with something less complicated than guix-daemon?

Hi,

yelninei@tutamail.com writes:

Intriguing.

Well there’s ‘guix publish’, and otherwise the examples from

‘tests/systemd.sh’ (following ‘define %command’).

Otherwise we could mimic it by writing a C program that that opens a

SOCK_NONBLOCK socket, binds + listens + select(2) until something

happens, then calls fcntl(2) to clear the O_NONBLOCK flag, and then

forks + execs and call accept(2) in the child process.

Ludo’.

Hello,

Apr 16, 2025, 20:19 by ludo@gnu.org:

I tested guix-publish and that had no issues.

Some checks I did yesterday with guix-dameon:

- Shepherd is passing a blocking socket

- The "fdSocket" in "acceptConnection" is always blocking.

- the "remote" socket in "acceptConnection" is O_NONBLOCK on the first connection only.

- Then also the "from.fd" socket in  "processConnection" is O_NONBLOCK on the first connectionThis then causes EAGAIN on trying to read the clientVersion.

On linux none of this is an issue.

Adding the same check as for the fd 3 socket  for O_NONBLOCK to the "connection" socket after accept  to tests/systemd.sh passes on Linux but causes a failure on the Hurd.

Is glibc accept doing something weird?

I am struggling to understand how the first connection would be any different than subsequent ones (and only in the #:lazy-start? #t case)

I am unsure what to do about this because shepherd seems to do everything correctly. I saw that ci.g.g.o has started to build i586-gnu substitutes (in particular gcc-final) but if you are restarting the builders more aggressively now then each first build will fail because of this and idk if cuirass can reschedule builds on such failures.

Maybe the easiest is to to expose the #:lazy-start? option for now and disable it for guix-daemon in %base-services/hurd ?

Hi,

yelninei@tutamail.com writes:

You mean the first ‘wget -O …’ passes?

Looking at ‘accept4.c’ in libc, the only way ‘remote’ can be O_NONBLOCK

is if:

1. ‘accept4’ is passed SOCK_NONBLOCK, but that’s not the case here

(see ‘accept.c’);

2. ‘__socket_accept’ returns a O_NONBLOCK socket, which would be a bug

in the server, pflocal.

At first sight ‘S_io_set_all_openmodes’ in pflocal does the job and

‘S_socket_accept’ honors those flags.

So we have a reproducer.

Could you pass it on to bug-hurd? :-) It may be easier if the whole

thing is in C.

Yeah, it’s not great. Those will have to be restarted manually I’m

afraid, but most of the time anybody can click on the “Restart” button

in Cuirass.

Hmm maybe. Let’s first figure out if this is Hurd bug.

Thanks for investigating!

Ludo’.

For the record, this issue is now fixed upstream:

https://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?id=029ab7d7b38c76ba14c24fcbf526ccef29af9e88

https://lists.gnu.org/archive/html/bug-hurd/2025-05/msg00016.html

Closing!

Ludo’.

Hi Ludo,

Thank you again for finding the cause.Could we add your patch to our hurd either for master or core-packages-team as it will be a while until it is available in a tagged snapshot.It would fix the hurd ci builders randomly failing, the childhurd system test and the minor annoyance that the manual offload is failing.

From what I can see only adding it to hurd (and not the headers) should not cause a rebootstrap. 

May 14, 2025, 17:03 by ludo@gnu.org:

Hi yelninei,

yelninei@tutamail.com writes:

Yes, sounds like a good idea. Do you want to give it a try?

Thanks,

Ludo’.

Hello Ludo,

Something like this? I called the patch hurd-socket-activation.patch to indicate what it is addressing. Do you have a better suggestion?

I added it to master but this will create a minor merge conflict with the hurd update on core-packages-team.

May 14, 2025, 21:51 by ludo@gnu.org:

Hello,

yelninei@tutamail.com writes:

Perfect; applied, thank you.

Hopefully we can easily address it.

Ludo’.