Skip to content

Add Eio_unix.Net module#516

Merged
talex5 merged 1 commit intoocaml-multicore:mainfrom
talex5:unix-net
May 23, 2023
Merged

Add Eio_unix.Net module#516
talex5 merged 1 commit intoocaml-multicore:mainfrom
talex5:unix-net

Conversation

@talex5
Copy link
Copy Markdown
Collaborator

@talex5 talex5 commented May 12, 2023

  • Add Eio_unix.Net module and move network-related items from Eio_unix to there.
  • Replace Eio_unix.socketpair with Eio_unix.Net.socketpair_{stream,datagram}. Previously, it took a type argument (stream or datagram), but always returned a stream_socket.
  • Move sockaddr conversions to eio.unix so they can be shared.
  • Implement getnameinfo in eio.unix to share it by default.
  • Eio_linux.Low_level.send_msg no longer throws away the number of bytes sent.
  • Add close to all socket types.
  • The destination in send is now optional, for unconnected datagram sockets.
  • send now takes a iovec, not just a single buffer. The stream API allows this and we tested it for datagrams using that API due to the previous type confusion.
  • Add Unix_datagram socket address type for completeness.
  • Add send_msg and recv_msg stubs to Eio_posix. This avoids converting to bytes, and will be useful to add FD passing later.

I was planning to include FD passing in this FD, but it got too long so I've split that out for later.

Fixes #342.

@talex5 talex5 added the enhancement New feature or request label May 12, 2023
@talex5 talex5 force-pushed the unix-net branch 10 times, most recently from 3d9504d to a126e2e Compare May 15, 2023 14:50
@talex5 talex5 marked this pull request as ready for review May 15, 2023 14:51
@talex5 talex5 marked this pull request as draft May 16, 2023 08:24
@talex5
Copy link
Copy Markdown
Collaborator Author

talex5 commented May 16, 2023

Having tried this out for a bit, this API is a little annoying because we also need unix equivalents to Net.run_server, etc. So I think it might be better to make the main Eio types polymorphic instead.

@talex5 talex5 force-pushed the unix-net branch 7 times, most recently from 5f51026 to 485eee8 Compare May 16, 2023 13:35
@talex5 talex5 marked this pull request as ready for review May 16, 2023 13:50
@talex5
Copy link
Copy Markdown
Collaborator Author

talex5 commented May 16, 2023

I've removed the FD passing bits for now and put them at https://github.com/talex5/eio/tree/fd-passing, which I'll submit separately once this is merged. So this PR is just adding a skeleton Eio_unix.Net and sorting out the stream/datagram confusion now, which should make it easier to review.

@talex5 talex5 requested a review from haesbaert May 16, 2023 14:03
@haesbaert haesbaert self-assigned this May 17, 2023
lib_eio/net.ml Outdated

type datagram = [
| `Udp of Ipaddr.v4v6 * int
| `Unix_datagram of string
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still digesting this diff, but not sure why Unix_datagram is named like this in the portable core. We have unix domain sockets, and potentially Windows named pipes, so should this be a little more abstract?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be. The unix(7) man-page says "The AF_UNIX (also known as AF_LOCAL) socket family ...", so perhaps Local_datagram would work?

I guess the question is whether it's useful to use Unix-domain sockets interchangeably with Windows named pipes. If so, they should share a name. If not, we can add another constructor for those.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a somewhat orthogonal set of properties that each expose. Windows named pipes are unidirectional streams I think. Is the common property here that some things have a local name, and some are datagrams or streams.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to https://learn.microsoft.com/en-us/windows/win32/ipc/pipe-names, a pipe name is in the format \\ServerName\pipe\PipeName. If a Linux process wanted to connect to a Windows named pipe on another machine, that would be quite different to connecting to a Unix socket with the same name, so I think they should be separate. Probably something like Windows_pipe (server, local_name).

We could combine Unix and Unix_datagram I think. The only thing that distinguishes between them is getnameinfo to set the NI_DGRAM flag, but that isn't useful for Unix-domain sockets anyway.

I wonder if we should separate out Linux abstract socket addresses too. Unix "/tmp/sock" refers to a file, but Unix "\000/tmp/sock" doesn't. Or perhaps we shouldn't have a Unix network address at all, and instead add Path.bind and Path.connect. They are part of the mount namespace rather than the network namespace.

@talex5
Copy link
Copy Markdown
Collaborator Author

talex5 commented May 22, 2023

I've pushed a couple of minor changes:

  • I reused the Unix tag for datagrams, since it's the same file either way, and therefore the same address. I think Windows named pipes should have their own tag if we want them, but that should be a separate issue.
  • I added Eio_unix.Net.import_socket_datagram to match import_socket_stream.

If there are no objections I'll merge this tomorrow, so I can then submit the FD-passing PR.

- Add `Eio_unix.Net` module and move network-related items from
  `Eio_unix` to there.
- Replace `Eio_unix.socketpair` with
  `Eio_unix.Net.socketpair_{stream,datagram}`. Previously, it took a
  type argument (stream or datagram), but always returned a `stream_socket`.
- Add `Eio_unix.Net.import_socket_datagram`.
- Move sockaddr conversions to `eio.unix` so they can be shared.
- Implement `getnameinfo` in `eio.unix` to share it by default.
- `Eio_linux.Low_level.send_msg` no longer throws away the number of
  bytes sent.
- Add `close` to all socket types.
- The destination in `send` is now optional.
- `send` now takes a iovec, not just a single buffer.
  The stream API allows this and we tested it for datagrams using
  that API due to the previous type confusion.
- Allow `Unix` socket paths as datagram addresses too.
- Add `send_msg` and `recv_msg` stubs to Eio_posix.
  This avoids converting to bytes, and will be useful to add FD passing
  later.
@talex5 talex5 merged commit a407535 into ocaml-multicore:main May 23, 2023
@talex5 talex5 deleted the unix-net branch May 23, 2023 09:44
@avsm
Copy link
Copy Markdown
Contributor

avsm commented May 29, 2023

One implication of this change is that Eio.Net.accept_fork and other connection handlers now must implement close (whereas previously it was just a Flow.two_way).

@avsm
Copy link
Copy Markdown
Contributor

avsm commented May 29, 2023

(For reference, this is the diff I applied to bring other libraries uptodate with this change avsm/eeww@dfdd8f2)

@talex5
Copy link
Copy Markdown
Collaborator Author

talex5 commented May 29, 2023

Good point. At the moment, accept_fork always closes the flow when the handler finishes, so if you close it manually then that will fail, which is why we didn't have close here before. Perhaps we should revert the addition of close here.

@avsm
Copy link
Copy Markdown
Contributor

avsm commented May 29, 2023

It would be cleaner to just pass a Flow.two_way I think, and leave closing to the connection handler.

talex5 added a commit to talex5/eio that referenced this pull request May 31, 2023
Sockets created by `accept_fork` can't be closed manually currently, so
revert the part of ocaml-multicore#516 that said they could.

We might want to revisit this later, for now put things back how they
were.

Reported by Anil Madhavapeddy.
talex5 added a commit to talex5/eio that referenced this pull request May 31, 2023
Sockets created by `accept_fork` can't be closed manually currently, so
revert the part of ocaml-multicore#516 that said they could.

We might want to revisit this later, for now put things back how they
were.

Reported by Anil Madhavapeddy.
talex5 added a commit to talex5/eio that referenced this pull request May 31, 2023
Sockets created by `accept_fork` can't be closed manually currently, so
revert the part of ocaml-multicore#516 that said they could.

We might want to revisit this later, but for now put things back how
they were.

Reported by Anil Madhavapeddy.
talex5 added a commit to talex5/eio that referenced this pull request May 31, 2023
Sockets created by `accept_fork` can't be closed manually currently, so
revert the part of ocaml-multicore#516 that said they could.

We might want to revisit this later, but for now put things back how
they were.

Reported by Anil Madhavapeddy.
talex5 added a commit to talex5/eio that referenced this pull request May 31, 2023
Sockets created by `accept_fork` can't be closed manually currently, so
revert the part of ocaml-multicore#516 that said they could.

We might want to revisit this later, but for now put things back how
they were.

Reported by Anil Madhavapeddy.
@talex5 talex5 added this to the 0.10 milestone May 31, 2023
talex5 added a commit to talex5/eio that referenced this pull request May 31, 2023
Sockets created by `accept_fork` can't be closed manually currently, so
revert the part of ocaml-multicore#516 that said they could.

We might want to revisit this later, but for now put things back how
they were.

Reported by Anil Madhavapeddy.
talex5 added a commit to talex5/eio that referenced this pull request May 31, 2023
Sockets created by `accept_fork` can't be closed manually currently, so
revert the part of ocaml-multicore#516 that said they could.

We might want to revisit this later, but for now put things back how
they were.

Reported by Anil Madhavapeddy.
talex5 added a commit to talex5/eio that referenced this pull request Jun 1, 2023
Sockets created by `accept_fork` can't be closed manually currently, so
revert the part of ocaml-multicore#516 that said they could.

We might want to revisit this later, but for now put things back how
they were.

Reported by Anil Madhavapeddy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Need Unix.socketpair_datagram

3 participants