More human-readable Shadow config #773
Description
Support for a more human-readable format such as YAML would be nice as it would be easier to read/write/modify config files and would help us eventually move away from XML configuration files. This work could be split into multiple steps:
- (Merged in Tool that converts XML shadow config file to YAML. #803) - Create an external tool that converts between
shadow.config.xmlXML files andshadow.config.yamlYAML files (i.e., converts XML to YAML, and YAML to XML). - (Merged in Allow to read config from stdin #895) - Modify Shadow so that it will accept its XML configuration on stdin rather than requiring that we read it from the
shadow.config.xmlfile. (It should be able to read from stdin or from a file.) - (Merged in Convert existing tests from xml to yaml, and use the yaml2xml converter in the test cases #972) - Convert existing tests from xml to yaml, and use the yaml2xml converter in the test cases.
- Long term: add support to Shadow so it can natively read and understand YAML files.
- Long term: new "options" section in the config file. Here we could specify any global options, and make it so that command line options could also instead be added to the yaml file (and Shadow internally would process the options using the same code for consistency whether the options are specified on the command line or in the config file).
As an example, the small example in the shadow repo currently looks like this in XML:
<shadow stoptime="3600">
<topology><![CDATA[<?xml version="1.0" encoding="utf-8"?><graphml xmlns="http://graphml.graphdrawing.org/xmlns" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
<key attr.name="packetloss" attr.type="double" for="edge" id="d6" />
<key attr.name="latency" attr.type="double" for="edge" id="d5" />
<key attr.name="packetloss" attr.type="double" for="node" id="d4" />
<key attr.name="countrycode" attr.type="string" for="node" id="d3" />
<key attr.name="bandwidthdown" attr.type="int" for="node" id="d2" />
<key attr.name="bandwidthup" attr.type="int" for="node" id="d1" />
<key attr.name="ip" attr.type="string" for="node" id="d0" />
<graph edgedefault="undirected">
<node id="isp">
<data key="d0">0.0.0.0</data>
<data key="d1">2251</data>
<data key="d2">17038</data>
<data key="d3">US</data>
<data key="d4">0.0</data>
</node>
<edge source="isp" target="isp">
<data key="d5">50.0</data>
<data key="d6">0.01</data>
</edge>
</graph>
</graphml>
]]></topology>
<plugin id="tgen" path="~/.shadow/bin/tgen"/>
<host id="server">
<process plugin="tgen" starttime="1" arguments="tgen.server.graphml.xml"/>
</host>
<host id="client">
<process plugin="tgen" starttime="2" arguments="tgen.client.graphml.xml"/>
</host>
</shadow>And we could convert between XML and something like this in YAML:
options:
- stoptime: 3600
topology:
- graphml: "<topology><![CDATA[<?xml version="1.0" encoding="utf-8"?><graphml xmlns="http://graphml.graphdrawing.org/xmlns" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
<key attr.name="packetloss" attr.type="double" for="edge" id="d6" />
<key attr.name="latency" attr.type="double" for="edge" id="d5" />
<key attr.name="packetloss" attr.type="double" for="node" id="d4" />
<key attr.name="countrycode" attr.type="string" for="node" id="d3" />
<key attr.name="bandwidthdown" attr.type="int" for="node" id="d2" />
<key attr.name="bandwidthup" attr.type="int" for="node" id="d1" />
<key attr.name="ip" attr.type="string" for="node" id="d0" />
<graph edgedefault="undirected">
<node id="isp">
<data key="d0">0.0.0.0</data>
<data key="d1">2251</data>
<data key="d2">17038</data>
<data key="d3">US</data>
<data key="d4">0.0</data>
</node>
<edge source="isp" target="isp">
<data key="d5">50.0</data>
<data key="d6">0.01</data>
</edge>
</graph>
</graphml>
]]></topology>"
plugins:
- id: tgen
path: ~/.shadow/bin/tgen
hosts:
- id: server
bandwidthdown: 102400
bandwidthup: 102400
processes:
- plugin: tgen
starttime: 1
arguments: tgen.server.graphml.xml
- id: client
bandwidthdown: 102400
bandwidthup: 102400
processes:
- plugin: tgen
starttime: 2
arguments: tgen.client.graphml.xml NOTE: We should continue to support graphml for the network topology graphs because we use igraph to read and to store the graph internally and to do graph operations on it (path lookups etc.). For anything but trivial examples, the network graph I use is quite large, and I usually don't want to reproduce that in the config file. Also, I almost always use the same graph, so having a path to it works well. For trivial graphs, like the ones usually included in the examples, we will have to either read from the YAML format and then programmatically build up an igraph graph, or specify the graph in a format that igraph understands. We chose graphml since it is common, and I don't see a reason for moving away from it now.