Hybrid mode on the dev branch (currently e48ca0d) is very slow.

Hybrid mode attempts to be faster than ptrace mode by avoiding a syscall (and therefore a ptrace-stop) in communication between Shadow and the shim. This only happens when the plugin's request to shadow is responded to before the plugin reaches it's spin limit and falls back to a blocking sem_wait. If the plugin ends up calling sem_wait, then it'll end up making a futex syscall, which will result in a ptrace-stop. When that happens, the syscall ends up having strictly more overhead than in ptrace mode, since now it has to deal with the ptrace-stop hybrid mode was meant to avoid in the first place and handle the IPC request from the shim.

In hybrid mode's current form, this problem will be exacerbated in the configuration we now believe to be most performant: using as many workers as (non-hyperthreaded) CPUs, with plugin processes pinned to the same CPU as its worker. In this case the Shadow worker won't get a chance to run to serve an IPC request until either the plugin process is pre-empted by the kernel (which will never happen if using sched_fifo), or it falls back to the blocking sem_wait. i.e. in this configuration hybrid mode will never be faster than ptrace mode.