Skip to content

Enthalpy#394

Merged
samoht merged 10 commits intomirage:masterfrom
pqwy:enthalpy
May 4, 2015
Merged

Enthalpy#394
samoht merged 10 commits intomirage:masterfrom
pqwy:enthalpy

Conversation

@pqwy
Copy link
Copy Markdown
Contributor

@pqwy pqwy commented Apr 26, 2015

  • Removes mentions of the ENTROPY interface.
  • Removes TLS_over_conduit that depended on it.
  • If nocrypto is found in the set of transitive dependencies, they are extended with its appropriate seeding module and the seeding process is triggered just before the application-side of the unikernel runs.

I apologise for simply deleting tls+conduit, wasn't sure what to do with it. In general, anything that depends on tls should just work now, so it should be a breeze to re-add.

The detection code is horribly wonky. Since there is no clearly defined point when the variables ps and ls are consumed, and to avoid running ocamlfind several times, it uses a side-effect and a fragile assumption about the ordering of calls to packages and libraries. I could not figure out a non-intrusive way to extend the tool and make this clean at the same time.

Another problem is in the detection logic: it scans for nocrypto only in the set of (opam-level) packages depended on, not in the set of (ocamlfind-level) libraries. The detection will fail if someone has, say, tls installed and wants to use it, but forgets to add_to_opam_packages it. I wasn't sure what exactly to do here -- feedback appreciated.

(Needed for this.)

@pqwy
Copy link
Copy Markdown
Contributor Author

pqwy commented Apr 26, 2015

Detection maybe makes a little more sense now:

The seeding package is pulled in iff any package needs (or is) nocrypto.

The seeding library and the starting trigger are pulled in iff any library needs (or is) nocrypto.

@samoht
Copy link
Copy Markdown
Member

samoht commented Apr 26, 2015

I apologise for simply deleting tls+conduit, wasn't sure what to do with it. In general, anything that depends on tls should just work now, so it should be a breeze to re-add.

this makes sense, these will be replaced by a proper HTTPs combinator later.

Regarding the patch: the only point which is not totally clear by reading the patch is when the call to ocamlfind is done. To be valid it needs to be done after the opam packages are installed, as ocamlfind query -r <xxx> will return stuff about installed packages. [An other option is to ask opam instead but that will be slower (as you need to call the solver) and less precise so the current approach is better]

I'll try to play with the patches tomorrow.

@pqwy
Copy link
Copy Markdown
Contributor Author

pqwy commented Apr 27, 2015

First the presence of a package is detected through configure_opam, which calls packages and this is remembered. Then configure_makefile sees the same answer for packages, and calls libraries.

So I think you are right:

On a clean installation, if a user declares conduit via, say, a combinator, it in turn declares tls. As this is not yet installed, nocrypto is not seen by ocamlfind, and mirage-entropy-xen, its dep for entropy on xen, is not added to the package set. But nocrypto is still pulled in by opam to satisfy tls, and since mirage-xen is pulled in too, opam configures nocrypto to also require mirage-entropy-xen. Depending on whether opam can express that the presence of package A causes package B to require package C, this either installs everything nicely, or aborts because nocrypto now requires mirage-entropy-xen.

If the opam succeeded, the call to libraries correctly detects nocrypto (the lib) as being transitively present, adds the correct entropy lib to the linkage set, and emits the init code.

If there is a way to make opam install mirage-entropy-xen before installing nocrypto if mirage-xen is installed, i.e. if there is a way to conditionally require (as opposed to just conditionally using), we can delete the code around package and rely on that. Is this possible?

Otherwise, yes, this is incorrect and the package detection might have to go via opam. Can we just ask it for immediate deps?

@samoht
Copy link
Copy Markdown
Member

samoht commented Apr 27, 2015

Continuing the discussion of mirleft/ocaml-nocrypto#56 (comment) here to centralise stuff.

while this is not nice and clean, I thought the mirage tool would dependency-inject mirage-entropy-xen if nocrypto is in use (nocrypto needs an optional dependency to mirage-entropy-xen, then it'll be recompiled after installation of mirage-entropy-xen)

I don't really like the double compilation scheme, and this won't work very well with the current mirage tool implementation (could be changed though, but I'll try to minimise build time as much as possible).

I think the simpler and not too hacki-ish solution is to always install mirage-entropy-xen when using the xen mode for now on. And then use the detection scheme introduce in that PR to know if we need to link with the ocamlfind library or not.

@samoht
Copy link
Copy Markdown
Member

samoht commented Apr 27, 2015

The only problem with that scheme is that if the user:

  • starts from scratch
  • configures with the --no-opam option
  • installs the opam package manually

but I think it's fine if it is properly documented and if the mirage tool warns the user that using --no-opam might be a terrible idea and if we hard-code some well-known packages that should bring the linking options (ie. "tls").

@hannesm
Copy link
Copy Markdown
Member

hannesm commented Apr 27, 2015

I'm fine with that. mirage-entropy-xen is also rather lightweight and thus shouldn't harm too much to have compiled whenever someone chooses mirage on xen...

@avsm
Copy link
Copy Markdown
Member

avsm commented Apr 27, 2015

I think the simpler and not too hacki-ish solution is to always install mirage-entropy-xen when using the xen mode for now on.

Agreed. Some comments in the tool referencing why it's included by default (and possibly referencing this PR) would help us remember this whole discussion.

@samoht
Copy link
Copy Markdown
Member

samoht commented Apr 27, 2015

An other approach is to add a virtual mirage-noxen package which will be in conflict with mirage-xen. And then use it to encode implication in the dependency constraints:

depends: [
...
  ("mirage-noxen" | ("mirage-xen" & "mirage-xen-entropy"))
]

Would be better if it is tracked directly by opam though, pinging @AltGr to comment about the possibility of having ocaml/opam#2074 (comment) in opam 1.2.2 ...

@pqwy
Copy link
Copy Markdown
Contributor Author

pqwy commented Apr 27, 2015

I was also toying with the idea of just unconditionally installing mirage-entropy-xen as a quick fix for now.

This is what the patch now does.

Glad to squash the commits if your prefer that.

@samoht
Copy link
Copy Markdown
Member

samoht commented Apr 27, 2015

Looks good to me. If we want to be extra careful we can also verify that the ocamlfind libraries that we are requiring are correctly installed on the system and if not print a warning about potentially missing initialisation function. Not sure it will happen in practice, but as we are providing the --no-opam option people can start making false assumptions ...

@pqwy
Copy link
Copy Markdown
Contributor Author

pqwy commented Apr 27, 2015

How would you add that? I'm a little reluctant to add early exits to functions that look like they are computing extra deps..

lib/mirage.ml Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be let to_list = elements.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it doesn't seem to be used by the PR (also my suggestion would be in fact the reverse of the current to_list).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot about it, thanks!

@samoht
Copy link
Copy Markdown
Member

samoht commented Apr 27, 2015

I'd do it at the end of the configuration phase, before emitting the preamble code.

@pqwy
Copy link
Copy Markdown
Contributor Author

pqwy commented Apr 27, 2015

Updated. Can still squash the commits.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be explicit about the version? (such as >=0.3.0)!? previous mirage-entropy-xen will not work...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was to handle that via opam in nocrypto?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

putting the constraint in nocrypto should be fine.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file should not be here

@samoht
Copy link
Copy Markdown
Member

samoht commented Apr 29, 2015

Looks good to me. Ready to merge and release when the rest of libraries are ready.

@hannesm
Copy link
Copy Markdown
Member

hannesm commented Apr 29, 2015

since we will have mirage-no-xen anyways (for gmp-xen dependency in zarith), I'd put a conditional dependency in nocrypto: ("mirage-no-xen" | ("mirage-xen" & "mirage-entropy-xen")).
Than, we can remove the hardcoded dependency on mirage-entropy-xen within the mirage tool (in this PR).

@pqwy
Copy link
Copy Markdown
Contributor Author

pqwy commented May 3, 2015

@samoht The rest has been merged and tagged, with a PR waiting in opam-dev. This is the last blocker now.

samoht added a commit that referenced this pull request May 4, 2015
@samoht samoht merged commit cbfbd70 into mirage:master May 4, 2015
@pqwy pqwy deleted the enthalpy branch May 5, 2015 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants