Currently wasmtime handles stack overflow in wasm code through the use of the segfault signal handler, relying on wasm code to hit the guard page to trigger a segfault which we the longjmp to recover from. Unfortunately, though, this has a consequence where it doesn't work well for "almost stack overflowed" wasm code which calls native imports.

For example let's say that a wasm module has an extremely deep call stack, but then it calls an imported function where then the imported function hits the guard page. This will trigger a segfault, and we still want this to be a somewhat recoverable situation! In this scenario though it's not actually save to longjmp over a bunch of Rust frames since it can skip some critical destructors.

I think in general we'll want to update wasmtime's handling of stack overflow in WebAssembly code to not rely on segfaults. In talking with some Spidermonkey folks, I think we'll want to switch to a scheme that looks something like:

• We "reserve" a fixed amount of stack space (in bytes) for wasm. Not literally on the side, but we say that the wasm stack can't ever be larger than "N" bytes.
• Each wasm function has a prelude check against "where is the current stack", and manually aborts if it's about to overflow. This may require a form of a dedicated register for storing the latest stack limit.
• Then we'll limit segfault handling only to out of bounds loads/stores in wasm code, and not to stack overflow.

This should help retain all our current speed (with sufficient tweaks/optimization) but also allow us to guarantee that native code imported by wasm always has a reasonable amount of stack space. This means that stack overflow in native code would never be recovered from (it'd abort the process as typical Rust programs do).