Summary

This is a proposal to add wasi-sockets support to wasi-libc as a first step towards full WASI Preview 2 support. This includes adding a new wasm32-wasi-preview2 build target to differentiate it from the existing wasm32-wasi{-threads} targets based on WASI Preview 1.

Background

WASI Preview 2 has been under development for a few years and should be finalized by the end of 2023 or early 2024. Unlike Preview 1, which was defined using WITX and core WebAssembly specification, Preview 2 is based on WIT and the Component Model proposal. In addition, Preview 2 uses unforgeable resource handles instead of file descriptors to track files, sockets, and other host resources.

In order to ease the transition from Preview 1 to Preview 2, the Wasmtime team has created an adapter which may be used with wit-component to convert modules targeting Preview 1 into components targeting Preview 2. Therefore, any toolchain targeting Preview 1 can be used in combination with the adapter and wit-component to generate Preview 2 components. Moreover, developers can use tools like wit-bindgen to access Preview 2 features beyond the scope of Preview 1, including full TCP and UDP socket support via wasi-sockets and high-level HTTP support via wasi-http, with additional interfaces in the works (e.g. wasi-cloud-core).

However, using wasi-sockets via direct host calls is akin to doing networking on native platforms via syscalls, bypassing libc and/or the standard library of the language being used. That makes reusing third-party, network-aware libraries such as database drivers difficult or impossible since they are normally designed to use the relevant standard library. So while it is technically possible to use wasi-sockets in applications today, it will not be ergonomic or practical until the standard libraries of various programming languages have been ported to use it, starting with wasi-libc.

On the other hand, standard libraries with existing Preview 1 support for features such as filesystem access, env variables, clocks, etc. will continue to work in a Preview 2 environment via the adapter, so there's less urgency to update those parts of wasi-libc to use Preview 2 directly.

Proposal

Given the situation described above, we're proposing to create a new wasm32-wasi-preview2 build target for wasi-libc and wasi-sdk which will initially use the Preview 1 host APIs (as implemented by the Preview 1->2 adapter) for everything except sockets, which will bypass the adapter and use the Preview 2 host APIs directly. From there, we'll incrementally replace the Preview 1 parts with their Preview 2 equivalents until the adapter is no longer needed at all.

During the transition period, wasi-libc and the adapter will share responsibility for mapping Preview 1 file descriptors to Preview 2 resource handles, with the former handling sockets and the latter handling files and stdio. In order to avoid confusion (e.g. both wasi-libc and the adapter using the same descriptor to mean different things), we'll add a new adapter_open_badfd function to the adapter, which wasi-libc will use to reserve descriptors for its use, indicating that the adapter should return EBADF if it receives any Preview 1 calls for such descriptors besides fd_close.

Testing

Currently, wasi-libc relies on a subset of a fork of musl's libc-test suite for testing. We plan to expand that subset to include all relevant socket tests. In addition, we'll be updating the Rust and Python standard libraries to match progress made in wasi-libc, enabling the socket tests in their test suites as well.

Prototype

I've created experimental forks of the wasi-libc, the adapter, and Rust, along with a test harness that demonstrates the use of wasi-sockets via the Rust standard library.