Network/HTTP Provider

I'm currently using the WASI provider, but AFAIK WASI doesn't support network yet. I'd like being able to just have an HTTP server on top of a WASI module so that I can send REST requests and retrieve computation results.

The wascc-provider could be the solution, but you also have to "accept" its architecture with actors, signing modules, capability providers etc. One may not desire to stick to such architecture. Also, wascc uses wasmi under the hood instead of wasmtime, AFAIK.

So the only remaining alternative would be writing my own provider, right? In particular, I can see at the least 3 big steps are required:

1. Writing the host code which allows some sort of HTTP capability inside the WASM module, just like a wascc HTTP provider does
2. Implement the k8s network virtualization in Krustlet, aka supporting CNI 😄 I see there was some work already done months ago https://github.com/deislabs/krustlet/issues/293, but it seems that development stopped. Was there any motivation in particular?
3. Allow to expose Krustlet "containers" as services

Good questions here. So as for #293, we stopped work on that so we could focus on the CSI stuff this release (0.6). The focus of the next release it getting networking up and running in a way that can work cross-provider. So we'll be picking that back up soon. CSI (and I assume CNI will run into similar issues) took much longer than expected due to conflicting documentation and supporting it cross platform.

As for writing your own provider, I am guessing we won't have support in WASI for networking for a little while yet. However, @technosophos currently has some plans to make a WAGI provider soon that should give you the functionality you are looking for. That is something you could try out once we have it (or maybe even help with once we get it to a state where people can contribute)

Just to add something on top of what @thomastaylor312 already mentioned: implementing CNI won't mean that the WASI (wasmtime) provider will support WebAssembly modules that can open a socket connection - WASI does not have sock_send/sock_connect/sock_recv support at this time, so the runtime will be unable to fulfill that request.

However, it does open up the opportunity to write new Providers that can utilize CNI support. @kflansburg's krustlet-cri project is one example, and @radu-matei has been working on a proof-of-concept of a WASI runtime with Berkeley Sockets support. Implementing CNI means that we could write a new Provider around this proof-of-concept to demonstrate WebAssembly modules with networking support.

(POC of the sockets work mentioned in the previous comment).

Besides WASI implementing the syscalls @bacongobbler mentioned, there is also a discussion around network clients and servers - given that WebAssembly does not support multi-threading just yet (here is the proposal), even if sock_recv was implemented, it would only allow for a single-threaded server, which is far from ideal. Networking clients, which would just open connections to other servers not running in Wasm runtimes should work, as described in the article above, but are still unsupported, both in standard libraries, and in runtimes.

WAGI is our current workaround for building simple, language-agnostic HTTP handlers compiled to WebAssembly, and a Krustlet provider that used WAGI would be a really nice way to temporarily handle this use case.

I figured out that actually I can workaround by using the exec handler for the time being. It may sound extremely dirty but I think it works and I feel it's better, at least for me, than exploring something like WAGI since I'm not familiar with CGI.

That would also allow to reach the WASM module by using the pod name instead of the node IP (althought I will probably use the latter since I have issues with the TLS from the apiserver). Otherwise, as far as I understand it, the WAGI solution still doesn't address how other k8s containers can reach the WASM module.

Thanks for the article btw, I'll have a read tomorrow. You already helped me a lot with the article about Tensorflow and WASM. Actually, I'm trying to make an inference model compiled in WASI reachable from a standard container in the cluster.

An interesting question emerged while I was trying to implement the exec in wasi-provider. Currently it's expected for the WASM module to have a _start function which will run endlessly or to completion. That is reasonable because it's what one would expect from a K8s pod.

However, it's not possible currently to execute any exported function from the WASM module other than _start. I was wondering what /exec could mean for a WASM module and if it makes sense to have a module which doesn't have _start or a noop _start.

I think it does. An example that comes to my mind is dnsutils, such as https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#create-a-simple-pod-to-use-as-a-test-environment where it sleeps upon start. Its purpose it then to allow executing bash commands to debug DNS.

For a WASM module, it may make sense to have a noop loop _start and act in a similar way. In my case, I would have a ML inference model which is called only when an inference is needed. Usually one would connect to a socket and listen for incoming requests, but since networking is still missing in WASI, this could be a good compromise simple enough for me to implement.

On the implementation side, Instance doesn't implement Send, which makes me sad. mumble mumble

How you are thinking of using exec is definitely what we thought about doing, but it is kinda weird how we'd make it work without multithreading. Right now you can only call one at a time within the same running module. Otherwise it is kinda more like a FaaS type call

Just a quick update that we have released an experimental library that allows experimental outbound HTTP for guest modules running in Wasmtime, meaning we could explore adding this to Krustlet.

https://github.com/deislabs/wasi-experimental-http