Both Uwt and Luv are bindings to libuv, the scalable IO library used by node.js.

Advantages of Uwt:

• it is very stable and we've tested it thoroughly in production
• it exposes an Lwt-based API for promises so works easily with the rest of our code

Disadvantages of Uwt:

• It hasn't been worked on in 3 years see OCaml 4.12.0 has Unix.SO_REUSEPORT fdopen/uwt#5
• it uses an old build tool (omake) which prevents us from switching everything to dune, removing Cygwin to fix our Windows build and (optionally) vendoring our dependencies with opam monorepo.

Advantages of Luv:

• it's likely to become the IO foundation for Lwt in future (Luv and Lwt have the same maintainers)
• it builds with dune so will help us remove Cygwin, fix our Windows build and vendor our dependencies

Disadvantages of Luv:

• it exposes the raw callback API of libuv so we need to carefully interface with the Lwt event loop
• we haven't tested it as thoroughly

vpnkit had already separated out IO into a separate module, which was needed to migrate from Lwt_unix to Uwt in the past. However the Luv interface is a thinner layer on top of libuv which means we need to carefully interface the libuv callback-based world with the Lwt promise world. The proposal here is to run Luv on one thread and Lwt on another, and to use Mutex-protected Queues to send work between the two event loops.

A typical function which can be called from Lwt looks like this:

let do_some_io () : unit Lwt.t =
    (* Called from the Lwt universe, which is where the TCP/IP stack runs *)
    Luv_lwt.in_luv (fun return ->
        (* Now we're in the Luv event loop *)
        Luv.Do.something _ (function
            (* libuv calls callbacks when interesting things happen.
               We want to tell the Lwt caller on the other thread.
               The helper function `return` pushes a result on the return queue. *)
            | Error err -> return (Error (`Msg (Luv.Error.strerror err)))
            | Ok () -> return (Ok ())
        )
    )

where Luv_lwt.in_luv pushes the function to a queue and signals the Luv event loop, and then when a return result is finally ready, the function return pushes the value to a queue and signals the Lwt event loop. The queues and signalling is all in a small module Luv_lwt.

The callback code is bit nested in places in this style:

match foo () with
| Error _ ->
| Ok () ->
  match bar () with
  | Error _ ->

see for example binding a UDP socket (you have to expand the collapsed diff of host.ml otherwise it's not visible).

but I thought I'd write everything out before trying to be too clever with combinators.

This was a good opportunity to start writing some inline unit tests to complement the existing end-to-end tests.

It was also a good opportunity to reformat the new code with ocamlformat.

Signed-off-by: David Scott dave@recoil.org