This is part of a series of periodic updates of development in Shadow. This work is sponsored by the NSF. Previous update: 2021-12.

We've merged 82 non-dependabot pull requests and closed 18 issues since our previous update.

Release status

We are continuing to work on Shadow 2.1. The biggest user-facing goal for this release is to support running golang programs in Shadow, especially tor simulations using the snowflake pluggable transport.

We've also begun planning the Shadow 2.2 release, which will largely be a push to refactor and migrate more of the core Shadow code to Rust.

Notable change since last update

Emulation accuracy

• Optionally move time forward in non-blocking syscalls. Historically Shadow doesn't move time forward except when explicitly waiting for an event, such as for a deadline to pass or for data to arrive over the network. Conceptually, this emulates a system with an infinite number of infinitely fast CPUs. Normally this is sufficient for modeling networks where CPUs aren't expected to be a bottleneck. Unfortunately as we've explored running more software under Shadow, we've found a growing number of examples of code with "busy loops", which deadlock in this model. This new feature optionally models every syscall taking some small amount of time (e.g. a microsecond), which allows the simulation to escape most such loops. We are still testing and improving this feature, and expect some version of it to be enabled by default in the next release.
• Fixed a corner case in getaddrinfo.
• Fixed several bugs in handling file descriptors:
  • Fixed pipe writable state when reader is closed.
  • Have pipe writers return EPIPE if there are no readers.
  • Fixed errno for socket with SOCK_SEQPACKET type.
  • Don't fail on TCP_NODELAY.
• Reliably intercept time via vdso. Previously we relied on intercepting calls to VDSO functions (such as gettimeofday) to be intercepted at the libc level via LD_PRELOAD. However, this doesn't work when the VDSO is used more directly, such as in golang, which would cause the program to get the real-world time instead of the simulated time. We now patch the VDSO itself at program start to reliably intercept these functions in such cases.
• Implemented basic signal emulation. This allows managed code to install signal handlers and send and receive signals within the simulation. (Sending a signal from outside the simulation to a managed process is still not supported). Notably this support is required to handle golang programs, and allows simulated processes to be shut down cleanly by scheduling kill processes to send appropriate signals.
• Implemented the select system call.

Usability

• Added an experimental option for strace-style logging of syscalls made by managed code.
• Stabilized the --progress option, which periodically updates stderr with the simulation progress.

Performance

• Nightly shadow benchmarks are now being run to help detect performance regressions. We also have the ability to run the benchmark on our own development branches to investigate performance changes before merging these branches into Shadow. Benchmark results for a 5% Tor network are published publicly, but are only intended to be useful for Shadow developers.
• Added a library for overriding crypto functions. When enabled, this option overrides some openssl APIs with "no-op" implementations. This is a reimplementation of a feature previously available in Shadow's "tor plugin". It is intended primarily to improve the performance of tor simulations on hardware without accelerated AES operations, and is only supported on Debian 11.

Happy simulating!
The Shadow team