Skip to content

Commit a48272f

Browse files
robgjansenrobgjansen
authored
Merge pull request #1433 from robgjansen/doc-update
Update docs for 2.x Includes: - more details about what shadow is and why it's important on the top-level readme - re-ordered the ToC in docs/readme to better organize contents into sections - added line breaks to long lines across all docs - more consistent use of terms like vertices->nodes, topology->graph - separate tgen example from shadow log parsing and more...
2 parents 14f0c87 + 1912c74 commit a48272f

30 files changed

Lines changed: 1363 additions & 761 deletions

‎README.md‎

Lines changed: 73 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -3,33 +3,87 @@
33
[![Shadow Tests](https://github.com/shadow/shadow/actions/workflows/run_tests.yml/badge.svg?branch=dev&event=push)](https://github.com/shadow/shadow/actions/workflows/run_tests.yml?query=branch:dev+event:push)
44
[![Tor Tests](https://github.com/shadow/shadow/actions/workflows/run_tor.yml/badge.svg?branch=dev&event=push)](https://github.com/shadow/shadow/actions/workflows/run_tor.yml?query=branch:dev+event:push)
55

6-
Shadow is a unique discrete-event network simulator that runs real
7-
applications like Tor and Bitcoin, and distributed systems of thousands of
8-
nodes on a single machine. Shadow combines the accuracy of emulation with the
9-
efficiency and control of simulation, achieving the best of both approaches.
6+
## What is Shadow?
107

11-
Quick Setup (installs Shadow in `~/.local`):
8+
Shadow is a discrete-event network simulator that enables you to simulate
9+
distributed systems of network-connected processes in a **realistic** and
10+
**scalable** private network experiment using your laptop or desktop running
11+
Linux.
12+
13+
Shadow experiments can be scientifically **controlled** and deterministically
14+
**replicated**, making it easier for you to reproduce bugs and eliminate
15+
confounding factors in your experiments.
16+
17+
## How Does Shadow Work?
18+
19+
Shadow directly executes **real applications**:
20+
21+
- Shadow directly executes unmodified, real application code using native OS
22+
(Linux) processes.
23+
- Shadow co-opts the native processes into a discrete-event simulation by
24+
interposing itself at the system call API.
25+
- The necessary systems calls are emulated such that the applications need not
26+
be aware that they are running in a Shadow simulation.
27+
28+
Shadow connects the applications in a **simulated network**:
29+
30+
- Shadow constructs a private, virtual network through which the managed
31+
processes can communicate.
32+
- Shadow internally implements simulated versions of common network protocols
33+
(e.g., TCP and UDP).
34+
- Shadow internally models network routing characteristics (e.g., path latency
35+
and packet loss) using a configurable network graph.
36+
37+
## Why is Shadow Needed?
38+
39+
Network emulators (e.g., [mininet](http://mininet.org)) run real application
40+
code on top of real OS kernels in real time, but are non-determinsitic and have
41+
limited scalability: time distortion can occur if emulated processes exceed an
42+
unknown computational threshold, leading to undefined behavior.
43+
44+
Network simulators (e.g., [ns-3](https://www.nsnam.org) ) offer more
45+
experimental control and scalability, but have limited realism because they run
46+
application abstractions in place of real application code.
47+
48+
Shadow offers a novel, hybrid emulation/simulation architecture: it directly
49+
executes real applications as native OS processes in order to faithfully
50+
reproduce application behavior while also co-opting the processes into a
51+
high-performance network simulation that can scale to large distributed systems
52+
with hundreds of thousands of processes.
53+
54+
## Caveats
55+
56+
Shadow implements **over 150 functions from the system call API**, but does not
57+
yet fully support all API features. Although applications that make _basic_ use
58+
of the supported system calls should work out of the box, those that use more
59+
_complex_ features or functions (e.g., `fork()`) may not yet function correctly
60+
when running it in Shadow. Extending support for the API is a work-in-progress.
61+
62+
That being said, we are particularly motivated to run large-scale [Tor
63+
Network](https://www.torproject.org) simulations and are eager to add necessary
64+
functionality in support of this use-case.
65+
66+
## Quickstart
67+
68+
Build, test, and install Shadow into `~/.local`:
1269
```
13-
$ ./setup build --clean
70+
$ ./setup build --clean --test
1471
$ ./setup test
1572
$ ./setup install
1673
```
1774

18-
Detailed Documentation
19-
+ [docs/README.md](docs/README.md)
75+
## More Information
76+
77+
Homepage:
78+
+ https://shadow.github.io
2079

21-
Questions:
80+
Detailed Documentation:
81+
+ [Local user documentation in docs/README.md](docs/README.md)
82+
+ [Online user documentation](https://shadow.github.io/docs/guide)
83+
+ [Online developer documentation](https://shadow.github.io/docs/rust)
84+
85+
Community Support:
2286
+ https://github.com/shadow/shadow/discussions
2387

2488
Bug Reports:
2589
+ https://github.com/shadow/shadow/issues
26-
27-
Shadow Project Development:
28-
+ https://github.com/shadow
29-
30-
Homepage:
31-
+ https://shadow.github.io
32-
33-
## Contributing
34-
35-
See [docs/contributing.md](docs/contributing.md)

‎docs/README.md‎

Lines changed: 22 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,35 @@
11
# The Shadow Simulator Documentation
22

3-
The docs contain important information about installing and using the Shadow discrete event network simulator. Please [open an issue](https://github.com/shadow/shadow/issues/new/choose) if you notice something out of date, or fix it yourself if you can.
3+
The docs contain important information about installing and using the Shadow
4+
discrete-event network simulator. Please [open an
5+
issue](https://github.com/shadow/shadow/issues) if you notice something out of
6+
date, or [open a pull request](https://github.com/shadow/shadow/pulls) if you
7+
can fix it.
48

59
* The Shadow simulator
6-
* [Design Overview](design_overview.md)
10+
* [Design Overview](design_2x.md)
711
* Installation Guide
812
* [Supported Platforms](supported_platforms.md)
913
* [Dependencies](install_dependencies.md)
1014
* [Shadow](install_shadow.md)
1115
* [System Configuration](system_configuration.md)
1216
* [(Experimental) Shadow with Docker](install_shadow_with_docker.md)
13-
* [Getting Started](getting_started.md)
14-
* [Parsing Statistics](parsing_statistics.md)
15-
* [Log Format](log_format.md)
16-
* Simulation Customization
17-
* [Shadow Configuration](shadow_config.md)
18-
* [Shadow Configuration Options](shadow_config_options.md)
19-
* [Network Configuration](network_config.md)
20-
* [Network Graph Attributes](network_graph_attributes.md)
21-
* [Migrating Simulations from Shadow 1.x](migrating_from_1x.md)
22-
* [Notes and FAQs](notes_and_faq.md)
17+
* Usage Guide
18+
* [Overview](run_shadow_overview.md)
19+
* Running Your First Simulations
20+
* [Basic File Transfer](getting_started_basic.md)
21+
* [Traffic Generation](getting_started_tgen.md)
22+
* [Simple Tor Network](getting_started_tor.md)
23+
* Understanding Shadow Output
24+
* [Format of the Log Messages](log_format.md)
25+
* [Parsing Statistics from the Logs](parsing_shadow_logs.md)
26+
* Configuring Your Own Simulation
27+
* [Shadow Config Overview](shadow_config_overview.md)
28+
* [Shadow Config Specification](shadow_config_spec.md)
29+
* Configuring Your Own Network
30+
* [Network Graph Overview](network_graph_overview.md)
31+
* [Network Graph Specification](network_graph_spec.md)
32+
* [Migrating Simulations from Shadow 1.x](migrating_from_1x.md)
2333
* Developer Guides
2434
* [Debugging and profiling](developer_guide.md)
2535
* [Continous integration tests](ci.md)

‎docs/design_1x.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
## Mission: Run Tor in a Box
2+
3+
| ❗ Notice |
4+
|--------------------------------------------------------------------------------------------------------------|
5+
| This overview describes Shadow's original<br>architecture (version 1.x) and not the current<br>architecture. |
6+
7+
<br>
8+
9+
<!--[[https://raw.githubusercontent.com/wiki/shadow/shadow/assets/torinabox.png|align=right|width=175px]]-->
10+
<!--[Run Tor in a box with Shadow!][image-torinabox]-->
11+
<!--[image-torinabox]: https://raw.githubusercontent.com/wiki/shadow/shadow/assets/torinabox.png-->
12+
13+
<a href="https://raw.githubusercontent.com/wiki/shadow/shadow/assets/torinabox.png"><img align="right" width="225" alt="Run Tor in a box with Shadow!" src="https://raw.githubusercontent.com/wiki/shadow/shadow/assets/torinabox.png"></a>
14+
15+
Shadow was developed because there was a recognized need for an accurate,
16+
efficient, and scalable tool for Tor experimentation: using the PlanetLab
17+
platform is undesirable due to management overhead and lack of control; existing
18+
emulators are far too inefficient when scaling to thousands of virtual hosts;
19+
roll-your-own simulators are often too inaccurate or generic to be useful for
20+
multiple projects; and experiments on the live Tor network are often infeasible
21+
due to privacy risks.
22+
23+
Our goal was to provide a tool that can be used by anyone with a Linux machine
24+
or access to EC2 to hasten the development of research prototypes and reduce the
25+
time to deployment. Although originally written with Tor experimentation in
26+
mind, Shadow can also run Bitcoin and is useful for researching or prototyping
27+
other distributed or peer-to-peer systems including multi-party computation
28+
protocols.
29+
30+
## Feature Overview
31+
32+
Shadow does the following:
33+
34+
+ creates an isolated simulation environment where virtual hosts may
35+
communicate with each other but not with the Internet
36+
+ natively executes **real applications** like Tor and Bitcoin
37+
+ provides efficient, accurate, and **controlled** experiments
38+
+ models network topology, latency, and bandwidth
39+
+ runs without root on a single Linux machine
40+
+ simulates multiple virtual hosts in virtual time
41+
+ simulates the network (TCP stack) and CPU processing delays
42+
+ can run private Tor networks with user/traffic models based on [Tor
43+
metrics][tormetrics]
44+
+ much, much more!
45+
46+
## Real Applications as Shadow Plug-ins
47+
48+
Shadow is a discrete-event simulator that runs **real applications** like
49+
[Tor][tor] . Shadow links to real application software and **natively executes
50+
the application code** during simulation, providing faithful experiments and
51+
accurate results. Shadow models and runs distributed networks using these
52+
applications on a single Linux machine, easing experiment management while
53+
keeping the focus on the results.
54+
55+
#### What are Plug-ins?
56+
57+
Plug-ins are shared library shims that are linked to real applications. Shadow
58+
dynamically loads these libraries to natively execute the application code.
59+
Shadow intercepts a selective set of system calls to enable seamless integration
60+
of an application to the simulated environment. In this way, the application may
61+
be unaware that it is running in the simulator and will function as if it was
62+
running in a standard UNIX environment.
63+
64+
Visit [this page on the wiki][wiki-custom-plugin] for more information about how
65+
to write your own custom Shadow plug-in.
66+
67+
#### What is shadow-plugin-tor?
68+
69+
shadow-plugin-tor is a Shadow plug-in for simulating the [Tor][tor] anonymity
70+
network. shadow-plugin-tor integrates Tor into Shadow by wrapping the [Tor
71+
source code][torsource] with the necessary hooks that allow it to communicate
72+
with the Shadow simulator, thereby leveraging Shadow's unique functionality to
73+
allow rapid prototyping and experimentation of Tor. shadow-plugin-tor also
74+
contains scripts that assist in analyzing results, generating Tor topologies,
75+
and running experiments using the generated topologies.
76+
77+
Visit [the shadow-plugin-tor wiki page][wiki-scallion] for more information on
78+
the Tor plug-in and its memory requirements.
79+
80+
## Simulation Blueprint
81+
82+
The first step to using Shadow is to create a blueprint of an experiment. The
83+
format of the blueprint is standard XML. The XML file tells Shadow when it
84+
should create each virtual host, what software each virtual host should run. It
85+
also specifies the structure of the network topology, and network properties
86+
such as link latency, jitter, and packet loss rates. Shadow contains example XML
87+
files to help get started!
88+
89+
<a href="https://raw.githubusercontent.com/wiki/shadow/shadow/assets/design1.png"><img title="design1" src="https://raw.githubusercontent.com/wiki/shadow/shadow/assets/design1.png" alt="" width="520" /></a>
90+
91+
_Shadow takes a simulation blueprint as input. This XML file specifies the
92+
structure of the topology and the general flow of the experiment. Shadow's event
93+
engine initializes hosts using the blueprint, and runs events on their behalf
94+
until the simulation is complete._
95+
96+
## Discrete Time Events
97+
98+
Shadow creates several bootstrapping events after extracting the information
99+
from the supplied XML blueprint file. Each of these events are executed at a
100+
discrete time instant during the experiment. Each of these bootstrapping events
101+
will cause the virtual hosts to start executing the specified software, which in
102+
turn will spawn additional events for Shadow to process. Shadow tracks the time
103+
each virtual host spends processing inside the application, and delays events
104+
according to the host's configured virtual CPU speed. Events are continuously
105+
executed until the simulation end time.
106+
107+
As applications send data to each other, Shadow packages that data into an
108+
internal type and transfers the pointer between various queues. This process
109+
involves the use of the main Shadow event queue to transfer the packet events
110+
between virtual hosts, and rate-limiting to ensure each host has the desired
111+
bandwidth capacity. The following image, courtesy of Steven Murdoch, may help
112+
visualize this process:
113+
114+
<a href="https://raw.githubusercontent.com/wiki/shadow/shadow/assets/shadow_packet_flow.pdf"><img title="shadow_packet_flow" src="https://raw.githubusercontent.com/wiki/shadow/shadow/assets/shadow_packet_flow.png" alt="" width="520" /></a>
115+
116+
_End-to-end application data flows through Shadow socket and interface buffers,
117+
while the discrete event queue facilitates the transfer of data between virtual
118+
hosts._
119+
120+
## Virtual Host Management
121+
122+
Each virtual host in Shadow runs real software, which are linked as Shadow
123+
plug-ins. For each instance of the plug-in that a host is configured to run,
124+
Shadow loads the plug-in into a private namespace container using a custom
125+
loader and `dlmopen()`. The loader takes care to minimize duplicated memory
126+
usage on multiple loads of the same plug-in.
127+
128+
With the virtual host's plug-in loaded, execution is then passed to the
129+
application by calling the `main()` function using a version of [GNU portable
130+
threads (pth)][gnu-pth]. Pth is an application-space non-preemptive
131+
priority-based threading library that can jump call stacks to support plug-ins
132+
that run multiple threads and block during execution. Shadow runs the plug-in
133+
instance until it would block, and then moves on to execute other plug-ins and
134+
hosts.
135+
136+
## Function Interposition
137+
138+
Shadow runs real applications that run on regular UNIX-type systems. These
139+
applications expect a wide range of kernel libraries to be available for use.
140+
For example, sending and receiving data over the network, system time, and
141+
device polling are all generally handled by the kernel at some level. These
142+
system functions (and others) are intercepted and redirected through
143+
Shadow-specific versions. In this way, Shadow provides the ability for hosts to
144+
seamlessly communicate with each other over the virtual network without
145+
requiring any changes to the application code.
146+
147+
## More information
148+
149+
See [the original Shadow webcast][youtube-shadow-design] for more information
150+
about Shadow's original design, and for an explanation of some experiments that
151+
utilize this unique architecture. An explanation of recent architecture updates
152+
to support blocking system calls (read/write/send/recv/sleep) and applications
153+
that spawn threads [can be found here][cset-rpth-slides]. Checkout [Google
154+
Scholar](https://scholar.google.com/scholar?oi=bibs&hl=en&cites=12341442653770148265)
155+
for research publications that cite Shadow.
156+
157+
<!--<iframe width="420" height="315" src="http://www.youtube-nocookie.com/embed/Tb7m8OdpD8A" frameborder="0" allowfullscreen></iframe>-->
158+
159+
[tor]: https://www.torproject.org/
160+
[tormetrics]: https://metrics.torproject.org/
161+
[torsource]: https://gitweb.torproject.org/tor.git
162+
[wiki-scallion]: https://github.com/shadow/shadow-plugin-tor/wiki
163+
[wiki-custom-plugin]: https://github.com/shadow/shadow/wiki/2-Simulation-Execution-and-Analysis#shadow-plug-ins
164+
[youtube-shadow-design]: http://youtu.be/Tb7m8OdpD8A
165+
[cset-rpth-slides]: http://www.robgjansen.com/talks/shadowbitcoin-cset-20150810.pdf
166+
[gnu-pth]: https://www.gnu.org/software/pth/

‎docs/design_2x.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Shadow 2.x Design
2+
3+
_TODO: This document should be expanded._
4+
5+
## Overview
6+
7+
Shadow directly executes real applications using native OS processes and co-opts
8+
them into a high-performance discrete-event network simulation. Shadow enables
9+
realistic and scalable private network experiments that can be scientifically
10+
controlled and deterministically replicated.
11+
12+
## Rationale
13+
14+
What about ns-3? [ns-3](https://www.nsnam.org) is a network simulator that is
15+
designed to replicate network-layer protocol behavior with very high fidelity.
16+
It contains accurate reimplementations of many network layer protocols and
17+
communication substrates, and is thus targeted primarily for use by researchers
18+
designing new network-layer protocols or protocol features. It does not (really)
19+
support running unmodified, real applications, leaving users to implement
20+
synthetic application abstraction models in place of real application code.
21+
22+
What about mininet? [mininet](http://mininet.org) is a network emulator that is
23+
designed to run real kernel, switch, and application code in real time. The real
24+
time requirement severely limits the number of processes that can be run in a
25+
mininet experiment: time distortion can occur if the processes exceed a
26+
computational threshold, which can result in undefined behavior and artifacts
27+
that lead to untrustworthy results.
28+
29+
Shadow aims to fill the gap between these tools. Like mininet, Shadow directly
30+
executes real applications in order to faithfully reproduce application
31+
behavior. But like ns-3, Shadow runs a discrete-event simulation in order to
32+
scientifically control and deterministically replicate network experiments.
33+
Shadow uniquely targets experiments with large-scale distributed systems and
34+
thus our simulator design prioritizes high-performance computing.

0 commit comments

Comments
 (0)