We currently have two paths to access managed process memory. The default (--use-memory-manager=true) mmap's most of managed process memory into shared memory (/dev/shm) so that shadow can access it directly. The fallback mechanism (--use-memory-manager=false, or for accessing regions we haven't mmapd) is to use process_vm_readv and process_vm_writev to read and write the memory.

In early iterations of shadow 2.0, the mapping path was a bigger win, because the fallback mechanisms were slower (e.g. transferring control back to the plugin and asking it to copy chunks of memory into a fixed-size shared memory buffer for us in "preload" mode; PTRACE_PEEKDATA or file operations on /proc/.../mem in "ptrace" mode).

Last time we looked at it, mapping does still give us a moderate win in I/O intensive microbenchmarks, but I'm not sure it ends up being a significant win for more realistic simulations.

Downsides of using the mapping path:

• It requires potentially a lot of space in /dev/shm, depending on the size of the simulation. This can require some configuration of the host OS and/or container.
• When we run out of space in /dev/shm, things can fail in difficult-to-debug ways. We try to detect when we've run out of space and fail loudly (e.g. ea80705), but in Tor we recently ran into some difficult to debug failures on both v2.2.0 and v2.3.0. Maybe we can do more to detect this situation and fail loudly, but it's always going to be a bit tricky.
• In general, maintaining Rust soundness for this path is a bit tricky. We haven't run into issues here yet, but I wouldn't be surprised if there were some lurking dragons.

It may be worth doing some benchmarks on more realistic simulation (e.g. tor) to check how much performance benefit the mapping path gives us there. If it's not much, we might want to change the default, and maybe even consider removing that code.