Shadow logs heartbeat messages containing various simulation data (for example bytes sent/received, bytes allocated/deallocated, CPU usage, retransmit header sizes, etc).

https://github.com/shadow/shadow/blob/main/docs/2-Getting-Started-Tutorial.md#gathering-statistics

All of this information is periodically written to the log file as a line of comma-delimited numbers. These log lines are difficult to understand, difficult to parse by external tools, and make the log file harder to read. Since some heartbeat messages are logged once per host, there can be many of these lines in larger simulations.

We should instead consider writing this simulation information to a dedicated file in shadow.data/, or writing it to host-specific files in shadow.data/hosts/hostname/. We could write this information in a dedicated data format such as streaming JSON to make it easier to parse, and optionally compress it on the fly.

We may still wish to keep the simulation heartbeat in the log file so that you can tell if a simulation has stalled.