Skip to content

defunctorise MCLOCK, PCLOCK, TIME, RANDOM#1599

Merged
hannesm merged 3 commits intomirage:mainfrom
hannesm:variants
Mar 4, 2025
Merged

defunctorise MCLOCK, PCLOCK, TIME, RANDOM#1599
hannesm merged 3 commits intomirage:mainfrom
hannesm:variants

Conversation

@hannesm
Copy link
Copy Markdown
Member

@hannesm hannesm commented Jan 21, 2025

to be used with mirage/mirage-skeleton#407

This is the final attempt. Last call.

To play around with it, we have an entire mirage-skeleton working \o/ -- mirage/mirage-skeleton#407 is the diff

Since there's a bunch of stuff that needs changes, I prepared https://github.com/hannesm/mirage-time-defunctorised with all the overlays.

Since there are various overlays, I invite everyone to review -- maybe first the mirage-skeleton changes. Take a deep breath.

And then you can take look around in here. Am happy to see reviews, am also happy to hear "please go ahead", in which case I'll open further PRs, and cut some releases so we can jump over the breaking change while it is hot (and the code hasn't suffered from bitrot yet).

FWIW, tcpip and mirage-logs (and ca-certs-nss) have test cases where the monotonic time or posix time needs to be set to specific values -- take a look into the mock subdirectories of mirage-mtime/mirage-ptime repositories. So, indeed it is possible.

FWIW, the mirage-logs implementation no longer needs lwt. :D

The only minor thing I'll adjust is the mirage-kv-mem / crunch and their last_modified handling -- where the semantics is different now -- but I don't think this is an issue.

@samoht
Copy link
Copy Markdown
Member

samoht commented Jan 22, 2025

I haven't looked at the details at all but the resulting mirage-skeleton PR looks very very nice! Many thanks for pushing this through!

@hannesm
Copy link
Copy Markdown
Member Author

hannesm commented Feb 4, 2025

I added a commit which automatically adds a unit argument to delay execution of start in case there are no dependencies and no functor.

I tried to use Logs there, but I don't understand the compilation model, it turns out there's no log reporter installed when the DSL.main function is evaluated. Thus I resort to print_endline.

Feedback & reviews are welcome. I'm eager to get this merged rather sooner than later and fix thereafter potential breakage.

Looking at the mirage-skeleton changes, I'm not afraid of these changes, but I'm looking forward that all our unikernels will be more readable.

@samoht
Copy link
Copy Markdown
Member

samoht commented Feb 5, 2025

LGTM - happy to push any fix for breakage later on

@hannesm
Copy link
Copy Markdown
Member Author

hannesm commented Feb 5, 2025

Before this gets merged, we'll need to release some other things:

@hannesm
Copy link
Copy Markdown
Member Author

hannesm commented Feb 17, 2025

I guess irmin and cohttp are both packages I'm not familiar with, so if anyone has time and motivation to update these for the new APIs, that'd be great. Maybe @samoht?

Apart from that, I'll update the mirage utility and take a look which mirage-skeleton examples are working / not working.

The packages sendmail, dkim, caqti, miragevpn can be deferred to a bit later since mirage-skeleton doesn't use them (neither irmin as far as I can tell).

@palainp
Copy link
Copy Markdown
Member

palainp commented Feb 24, 2025

Dear @hannesm , I've just started to play with this PR, not sure if something could/need to be done, but now I have the delayed execution message even when I try to clean the repository :)

$ mirage clean
adding unit argument to 'start ()' (to delay execution)

@palainp
Copy link
Copy Markdown
Member

palainp commented Feb 24, 2025

Thanks again for your work on this defunctorisation :)
I'm going to investigate my issue, EDIT: should be fixed but currently xen&qubes targets fail using the linking trick and try to link to unix, the following was run in the hello mirage-skeleton unikernel:

$ opam pin
mirage.4.9.0            git  git+https://github.com/hannesm/mirage.git#variants
mirage-runtime.4.9.0    git  git+https://github.com/hannesm/mirage.git#variants
$ ocamlc --version
4.14.2
$ mirage clean && rm -rf duniverse/&& mirage configure -t xen && make depend && dune build
...
Successfully pulled 32/32 repositories
The sources have been pulled to the duniverse folder. Run 'make build' to build the unikernel.
File "duniverse/mirage-sleep/unix/dune", line 5, characters 12-20:
5 |  (libraries lwt.unix duration))
                ^^^^^^^^
Error: Library "lwt.unix" in _build/solo5/duniverse/lwt/src/unix is hidden
(optional with unavailable dependencies).
-> required by library "mirage-sleep.unix" in
   _build/solo5/duniverse/mirage-sleep/unix
-> required by executable main in dune.build:17
-> required by _build/solo5/main.exe
-> required by alias all (context solo5)
-> required by alias default (context solo5)

Copy link
Copy Markdown
Member

@palainp palainp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you again for that PR. So far it's very good to me! \o/

@hannesm
Copy link
Copy Markdown
Member Author

hannesm commented Feb 26, 2025

Dear @hannesm , I've just started to play with this PR, not sure if something could/need to be done, but now I have the delayed execution message even when I try to clean the repository :)

$ mirage clean
adding unit argument to 'start ()' (to delay execution)

I've not much knowledge what the best way would be to avoid this output. We could have some API to discover which mirage subcommand is being executed. But at the same time, I struggle with the fact that clean evaluates the config.ml -- which IMHO it should not.

Since I barely understand how functoria/mirage works - especially how clean is implemented (in my imagination, it should simply remove the generated dune* files and mirage subdirectory -- but it seems it is doing a lot more and fails if it is unable to parse config.ml), I'm not sure how the path forward is.

@palainp
Copy link
Copy Markdown
Member

palainp commented Feb 26, 2025

But at the same time, I struggle with the fact that clean evaluates the config.ml -- which IMHO it should not.

Since I barely understand how functoria/mirage works - especially how clean is implemented (in my imagination, it should simply remove the generated dune* files and mirage subdirectory -- but it seems it is doing a lot more and fails if it is unable to parse config.ml), I'm not sure how the path forward is.

I agree and to me that message is not a blocker for the merge&release :)

Copy link
Copy Markdown
Member

@palainp palainp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I forgot them when I increased the upper limit of ocaml-solo5.

@palainp
Copy link
Copy Markdown
Member

palainp commented Mar 4, 2025

\o/ CI is happy with Ocaml 4.14 and Ocaml 5.2 with unix target, but unhappy with 5.2 with solo5 targets as solo5 restricts to 5.2.1 due to compiler patches (in the meantime I've successfully compiled mirage-skeleton with the current Ocaml 5.3 and the ocaml-solo5 PR).

@hannesm
Copy link
Copy Markdown
Member Author

hannesm commented Mar 4, 2025

Thanks @palainp. I asked at ocurrent/mirage-ci#50 how we can achieve such a runner, and indeed soon [tm] we should be able to get 5.3 up and running.

@hannesm
Copy link
Copy Markdown
Member Author

hannesm commented Mar 4, 2025

I condensed this into 3 commits:

  • the "dep noop" thing
  • the defunctorisation
  • updating the expect tests

@hannesm
Copy link
Copy Markdown
Member Author

hannesm commented Mar 4, 2025

before merge:

  • adjust bounds in mirage.ml (4.9.0 .. 4.10.0)
  • write changes entry
  • do the release

later:

  • adapt mirage-skeleton PR to include the (* mirage >= 4.9 & < 4.10 *) comments.
  • merge that PR as well
  • adapt all other unikernels you can find :)

@hannesm
Copy link
Copy Markdown
Member Author

hannesm commented Mar 4, 2025

Enough CI, let's merge and release :)

@hannesm hannesm merged commit 5a8db30 into mirage:main Mar 4, 2025
1 of 5 checks passed
@hannesm hannesm deleted the variants branch March 4, 2025 17:03
hannesm added a commit to hannesm/opam-repository that referenced this pull request Mar 4, 2025
CHANGES:

- Remove time, pclock, mclock, random from functor arguments
  Instead, use the linking trick in mirage-mtime, mirage-ptime, mirage-sleep
  libraries (mirage/mirage#1599 @hannesm, discussion in mirage/mirage#1513 and on the mailing list)
  This removes the default_time, default_monotonic_clock, default_posix_clock
  bindings, and allows "Mirage.register" to get ?sleep ?ptime ?mtime ?random
  passed optionally, following how ?argv is handled.

  This is a breaking change, and your unikernel likely needs to be updated. Have
  a look at mirage/mirage-skeleton#407 for how our
  examples changed.

  As example, the hello-key unikernel diff:
```
  --- a/tutorial/hello-key/config.ml
  +++ b/tutorial/hello-key/config.ml
  @@ -2,5 +2,5 @@
   open Mirage

   let packages = [ package "duration" ]
  -let main = main ~packages "Unikernel.Hello" (time @-> job)
  -let () = register "hello-key" [ main $ default_time ]
  +let main = main ~packages "Unikernel" job
  +let () = register "hello-key" [ main ]

  --- a/tutorial/hello-key/unikernel.ml
  +++ b/tutorial/hello-key/unikernel.ml
  @@ -5,13 +5,12 @@ let hello =
     let doc = Arg.info ~doc:"How to say hello." [ "hello" ] in
     Mirage_runtime.register_arg Arg.(value & opt string "Hello World!" doc)

  -module Hello (Time : Mirage_time.S) = struct
  -  let start _time =
  +let start () =
     let rec loop = function
       | 0 -> Lwt.return_unit
       | n ->
         Logs.info (fun f -> f "%s" (hello ()));
  -          Time.sleep_ns (Duration.of_sec 1) >>= fun () -> loop (n - 1)
  +      Mirage_sleep.ns (Duration.of_sec 1) >>= fun () ->
  +      loop (n - 1)
     in
     loop 4
  -end
```
- Inject a unit argument to the start function if otherwise it would have
  no binding (discussed various times, including mirage/mirage#1088 mirage/mirage#873)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants