{"id":3373,"date":"2020-04-01T10:30:58","date_gmt":"2020-04-01T10:30:58","guid":{"rendered":"https:\/\/karneliuk.com\/?p=3373"},"modified":"2021-03-21T21:13:28","modified_gmt":"2021-03-21T21:13:28","slug":"dc-part-17-emulating-a-hyper-scale-datacentre-with-microsoft-azure-sonic-as-docker-containers-sonic-p4","status":"publish","type":"post","link":"https:\/\/karneliuk.com\/2020\/04\/dc-part-17-emulating-a-hyper-scale-datacentre-with-microsoft-azure-sonic-as-docker-containers-sonic-p4\/","title":{"rendered":"HS. Part 1. Emulating a hyper-scale datacentre with Microsoft Azure SONiC as Docker containers (SONiC-P4)."},"content":{"rendered":"\n<p>Hello my friend,<\/p>\n\n\n\n<p>In <a href=\"http:\/\/bit.ly\/2SktMT1\">the previous article<\/a> from <a href=\"https:\/\/karneliuk.com\/category\/network\/\">the networking series<\/a> we have started the discussion about SONiC (Software for Open Networking in Clouds), which is a network infrastructure behind Microsoft Azure cloud. Today we continue this discussion from slightly different angle: we will emulate the whole data centre infrastructure end-to-end with leafs, spines and servers.<\/p>\n\n\n\n<!--more-->\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/><\/div><\/td><td><div class=\"text codecolorer\">No part of this blogpost could be reproduced, stored in a <br \/>\nretrieval system, or transmitted in any form or by any <br \/>\nmeans, electronic, mechanical or photocopying, recording, <br \/>\nor otherwise, for commercial purposes without the <br \/>\nprior permission of the author.<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Network automation training \u2013 boost your career<\/h3>\n\n\n\n<p>Don\u2019t wait to be kicked out of IT business. Join our network automation training to secure your job in future. <a href=\"http:\/\/bit.ly\/2mP3SJy\">Come to NetDevOps side<\/a>.<\/p>\n\n\n<div class='code-block code-block-1' style='margin: 8px auto; text-align: center; display: block; clear: both;'>\n<p><a href=\"http:\/\/bit.ly\/2mP3SJy\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" class=\"aligncenter wp-image-4479 size-large\" src=\"https:\/\/karneliuk.com\/wp-content\/uploads\/2022\/08\/4_trainings.jpg\" alt=\"\"\/><\/a><\/p><\/div>\n\n\n\n\n<p>How does the <a href=\"http:\/\/bit.ly\/2mP3SJy\">training differ<\/a> from <a href=\"http:\/\/bit.ly\/2wmAIWw\">this blog post series<\/a>? Here you get the basics and learn some programming concepts in general, whereas in the training you get comprehensive set of knowledge with the detailed examples how to use Python for the network and IT automation. <a href=\"http:\/\/bit.ly\/2mP3SJy\">You need both<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Thanks<\/h3>\n\n\n\n<p>Big thanks to a colleague of mine, <a href=\"https:\/\/www.linkedin.com\/in\/michael-salo\/\">Michael Salo<\/a>, who shared with me useful insights on namespaces networking in Linux.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Brief description<\/h3>\n\n\n\n<p>Previously<a href=\"http:\/\/bit.ly\/2SktMT1\"> we\u2019ve run the SONiC on the Mellanox SN2010 switch<\/a>, which was provided to me <a href=\"https:\/\/karneliuk.com\/?s=mellanox&amp;post_type=post\">by the Mellanox team to do some reviews<\/a>. I\u2019ve returned it back, so I started looking for the opportunities to run the Microsoft Azure SONiC in a virtual environment. As I dealt previously with virtual network functions, such as <a href=\"http:\/\/bit.ly\/2sdhzBc\">Cisco IOS XRv, Nokia VSR, Cumulus Linux, Arista vEOS and many more<\/a>, I tried to find an Azure SONiC as a VM. I have failed to do that, but I have found something even more interesting and intriguing: SONiC as a Docker container. Copying information from the official webpage:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>SONiC-P4 is a software switch that runs on the P4-emulated SAI behavioural model software switch ASIC. It uses the sai_bm.p4 to program the P4-emulated switch ASIC to emulate the data plane behaviour. On top of that, it runs the real SONiC network stack.<\/p><cite>(c) https:\/\/github.com\/Azure\/SONiC\/wiki\/SONiC-P4-Software-Switch<\/cite><\/blockquote>\n\n\n\n<p>As far as I can see, <a href=\"https:\/\/github.com\/Mellanox\/SAI-P4-BM\">this P4 software switch is also created by Mellanox<\/a>, though I\u2019m not 100% sure about that. This software switch is build using the most widely used framework in the network disaggregation and white boxes industry, which is SAI API. SAI stands for Switch Abstraction Interface and defines the set of rules, how the network operation system interacts with the underlying hardware such as ASIC or NPU.<\/p>\n\n\n\n<p>From the testing point of view, the Microsoft Azure SONiC running inside the container with emulated P4-switch is managed almost in the same way as a physical switch running such as Mellanox SN2010 running SONiC.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What are we going to test?<\/h3>\n\n\n\n<p>The following points are covered in this article:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Creation of Docker containers without connection to the default Docker bridge to avoid NAT and any other improper network connectivity.<\/li><li>Deployment of Microsoft Azure SONiC on leafs and spine switches.<\/li><li>Connectivity between the hosts implemented on IP layer.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Software version<\/h3>\n\n\n\n<p>The following software components are used in this lab.&nbsp;<\/p>\n\n\n\n<p>Management host:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>CentOS 7.5.1804&nbsp;<\/li><li>Bash<\/li><li>Docker-CE 18.09<\/li><\/ul>\n\n\n\n<p>The Data Centre Fabric:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Microsoft Azure SONiC <strong>[guest CNF]<\/strong><\/li><li>Ubuntu <strong>[guest CNF]<\/strong><\/li><\/ul>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>CNF stands for a containerised network function.<\/p><\/blockquote>\n\n\n\n<h3 class=\"wp-block-heading\">Topology<\/h3>\n\n\n\n<p>They physical topology is significantly changed since <a href=\"http:\/\/bit.ly\/2SktMT1\">the previous networking labs<\/a>, as this is the first time, we build a containerised network. Hence, all of the network functions and hosts are containerised: <\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/>16<br \/>17<br \/>18<br \/>19<br \/>20<br \/>21<br \/>22<br \/>23<br \/>24<br \/>25<br \/>26<br \/>27<br \/>28<br \/>29<br \/>30<br \/>31<br \/>32<br \/>33<br \/>34<br \/>35<br \/>36<br \/>37<br \/><\/div><\/td><td><div class=\"text codecolorer\">+-------------------------------------------------------------------------+<br \/>\n| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br \/>\n| &nbsp; &nbsp; +-----------+ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+-----------+ &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | sw_port1 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;sw_port0 | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; | &nbsp;spine11 &nbsp;+-------+ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+---------+ &nbsp;spine12 &nbsp;| &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; +-----+-----+ &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; +-----+-----+ &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp;sw_port0 | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; &nbsp;sw_port1 | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; +---+---+ &nbsp; &nbsp; +---+---+ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+---+---+ &nbsp; &nbsp; &nbsp; +---+---+ &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; |s11_l12| &nbsp; &nbsp; |s11_l12| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|s12_l11| &nbsp; &nbsp; &nbsp; |s12_l12| &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; +---+---+ &nbsp; &nbsp; +---+---+ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+---+---+ &nbsp; &nbsp; &nbsp; +---+---+ &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp;sw_port5 | &nbsp; &nbsp; &nbsp; &nbsp; +----------------------+ &nbsp; &nbsp; &nbsp;sw_port6 | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; +-----+-----+ &nbsp; | &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+-----+-----+ &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; | &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; | &nbsp;leaf11 &nbsp; +---+ &nbsp; +----------------------------+ &nbsp;leaf12 &nbsp; | &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | sw_port6 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;sw_port5 | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; +-----+-----+ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+-----+-----+ &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp;sw_port0 | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; sw_port0 | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; +---+---+ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+---+---+ &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; |l11_h11| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|l12_h12| &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; +---+---+ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+---+---+ &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp;eth1 | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; eth1 | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; +-----+-----+ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+-----+-----+ &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; | &nbsp;host11 &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp;host12 &nbsp; | &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; +-----------+ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+-----------+ &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br \/>\n| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br \/>\n| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;(c)2020, karneliuk.com \/\/ DC POD with 2x leaf, 2x spine &nbsp; &nbsp; &nbsp;|<br \/>\n| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; and 2x hosts. Leaf\/Spine run MS Azure SONiC &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br \/>\n| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br \/>\n+-------------------------------------------------------------------------+<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>If you want to learn more about server lab setup, <a href=\"http:\/\/bit.ly\/2V0lGxs\">refer to the corresponding article<\/a>.<\/p><\/blockquote>\n\n\n\n<p>As said earlier, all the elements of the lab are running as Docker containers. However, each of the has a full network stack (rather complicated with multiple namespaces, to be honest). Let\u2019s dig into the details.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Topology emulation build<\/h3>\n\n\n\n<p>I\u2019ve started with the downloading the official repository of <a href=\"https:\/\/github.com\/Azure\/SONiC\/wiki\/SONiC-P4-Software-Switch\">SONiC P4 Software Switch<\/a>. Its content is the following:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>2 switches running Microsoft Azure SONiC<\/li><li>2 hosts running Ubuntu<\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">#1. Configuration files for Microsoft Azure SONiC<\/h4>\n\n\n\n<p>The Ubuntu hosts are pretty simple, so we don\u2019t focus on them too much. On the other hand, each SONiC switch has configuration files stored locally on your host, which are to be mapped towards the container. The following files are available per the virtual instance:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/><\/div><\/td><td><div class=\"text codecolorer\">sonic_switch<br \/>\n&nbsp; +--etc<br \/>\n&nbsp; | &nbsp;+--config_db<br \/>\n&nbsp; | &nbsp;| &nbsp;+--\ufeffvlan_config.json<br \/>\n&nbsp; | &nbsp;+--quagga<br \/>\n&nbsp; | &nbsp;| &nbsp;+--\ufeffbgpd.conf<br \/>\n&nbsp; | &nbsp;| &nbsp;+--\ufeffdaemons<br \/>\n&nbsp; | &nbsp;| &nbsp;+--\ufeffzebra.conf<br \/>\n&nbsp; | &nbsp;+--swss<br \/>\n&nbsp; | &nbsp; &nbsp; +--\ufeffconfig.d<br \/>\n&nbsp; | &nbsp; &nbsp; &nbsp; &nbsp;+--\ufeff00-copp.config.json<br \/>\n&nbsp; +--scripts<br \/>\n&nbsp; &nbsp; &nbsp;+--\ufeffstartup.sh<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>We can review all the files. However, from our lab prospective we need to modify 3 following files:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>etc\/config_db\/vlan_config.json<\/strong> stores the configuration of the mapping between physical interfaces, VLANs and IP addresses from the Microsoft Azure SONiC perspective. <a href=\"http:\/\/bit.ly\/2SktMT1\">We configured that some time ago on Mellanox SN 2010<\/a>.<\/li><li><strong>etc\/quagga\/bgp.conf<\/strong> provides the BGP configuration necessary to build the data centre fabric as well as the definition of what prefixes are to be advertised.<\/li><li><strong>scrips\/startup.sh<\/strong> contains the shell script utilized upon the boot process of the SONiC P4 switch. Among other information, it contains the mapping of the system MAC address to the physical interfaces<\/li><\/ul>\n\n\n\n<p>We need to change the content of these files based on the topology shared above. Using the example of <strong>leaf11<\/strong> you should have:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/>16<br \/>17<br \/>18<br \/>19<br \/>20<br \/>21<br \/>22<br \/>23<br \/>24<br \/>25<br \/>26<br \/>27<br \/>28<br \/>29<br \/>30<br \/>31<br \/>32<br \/>33<br \/>34<br \/>35<br \/>36<br \/>37<br \/>38<br \/>39<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ cat infrastructure\/leaf11\/etc\/config_db\/vlan_config.json <br \/>\n&nbsp;{<br \/>\n&nbsp; &nbsp; &quot;VLAN&quot;: {<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &quot;Vlan130&quot;: {<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;members&quot;: &amp;#91;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;Ethernet0&quot;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ], <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;vlanid&quot;: &quot;130&quot;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; },<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &quot;Vlan131&quot;: {<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;members&quot;: &amp;#91;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;Ethernet5&quot;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ],<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;vlanid&quot;: &quot;131&quot;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; },<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &quot;Vlan132&quot;: {<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;members&quot;: &amp;#91;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;Ethernet6&quot;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ],<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;vlanid&quot;: &quot;132&quot;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; }<br \/>\n&nbsp; &nbsp; },<br \/>\n&nbsp; &nbsp; &quot;VLAN_MEMBER&quot;: {<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &quot;Vlan130|Ethernet0&quot;: {<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;tagging_mode&quot;: &quot;untagged&quot;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; },<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &quot;Vlan131|Ethernet5&quot;: {<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;tagging_mode&quot;: &quot;untagged&quot;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; },<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &quot;Vlan132|Ethernet6&quot;: {<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;tagging_mode&quot;: &quot;untagged&quot;<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; }<br \/>\n&nbsp; &nbsp; },<br \/>\n&nbsp; &nbsp; &quot;VLAN_INTERFACE&quot;: {<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &quot;Vlan130|192.168.1.1\/24&quot;: {},<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &quot;Vlan131|10.0.0.1\/31&quot;: {},<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &quot;Vlan132|10.0.0.5\/31&quot;: {}<br \/>\n&nbsp; &nbsp; }<br \/>\n}<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>And this one:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/>16<br \/>17<br \/>18<br \/>19<br \/>20<br \/>21<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ cat infrastructure\/leaf11\/etc\/quagga\/bgpd.conf <br \/>\nhostname bgpd<br \/>\npassword zebra<br \/>\nenable password zebra<br \/>\nlog file \/var\/log\/quagga\/bgpd.log<br \/>\n!<br \/>\nrouter bgp 65111<br \/>\n&nbsp; bgp router-id 10.1.1.1<br \/>\n&nbsp;\ufeff bgp bestpath as-path multipath-relax<br \/>\n&nbsp; network 192.168.1.0 mask 255.255.255.0<br \/>\n&nbsp; neighbor 10.0.0.0 remote-as 65101<br \/>\n&nbsp; neighbor 10.0.0.0 timers 1 3<br \/>\n&nbsp; neighbor 10.0.0.0 send-community<br \/>\n&nbsp; neighbor 10.0.0.0 allowas-in<br \/>\n&nbsp; neighbor 10.0.0.4 remote-as 65102<br \/>\n&nbsp; neighbor 10.0.0.4 timers 1 3<br \/>\n&nbsp; neighbor 10.0.0.4 send-community<br \/>\n&nbsp; neighbor 10.0.0.4 allowas-in<br \/>\n&nbsp; maximum-paths 64<br \/>\n!<br \/>\naccess-list all permit any<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>And also this one:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/>16<br \/>17<br \/>18<br \/>19<br \/>20<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ cat infrastructure\/leaf11\/scripts\/startup.sh <br \/>\n&amp;#91; -d \/etc\/sonic ] || mkdir -p \/etc\/sonic<br \/>\n<br \/>\nSYSTEM_MAC_ADDRESS=00:dc:5e:01:01:01<br \/>\nip link add eth0 addr $SYSTEM_MAC_ADDRESS type dummy<br \/>\n<br \/>\nif &amp;#91; -f \/etc\/sonic\/config_db.json ]; then<br \/>\n&nbsp; &nbsp; sonic-cfggen -j \/etc\/sonic\/config_db.json -j \/sonic\/scripts\/vlan_config.json --print-data &amp;gt; \/tmp\/config_db.json<br \/>\n&nbsp; &nbsp; mv \/tmp\/config_db.json \/etc\/sonic\/config_db.json<br \/>\nelse<br \/>\n&nbsp; &nbsp; sonic-cfggen -j \/sonic\/etc\/config_db\/vlan_config.json --print-data &amp;gt; \/etc\/sonic\/config_db.json<br \/>\nfi<br \/>\n<br \/>\n#chmod +x \/usr\/bin\/config_bm.sh # TODO: remove this line<br \/>\ncp -f \/sonic\/etc\/swss\/config.d\/00-copp.config.json \/etc\/swss\/config.d\/default_config.json<br \/>\ncp -rf \/sonic\/etc\/quagga \/etc\/<br \/>\nip netns exec sw_net ip link set dev sw_port0 addr $SYSTEM_MAC_ADDRESS<br \/>\nip netns exec sw_net ip link set dev sw_port5 addr $SYSTEM_MAC_ADDRESS<br \/>\nip netns exec sw_net ip link set dev sw_port6 addr $SYSTEM_MAC_ADDRESS<br \/>\nsupervisord<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>As I\u2019m explaining my own experience, I don\u2019t pretend to be the hundred percent objective. First of all, there is little to zero documentation about the containerized Microsoft SONiC available. Therefore, I have to follow the \u201cplug-and-pray\u201d approach going almost blindly forward.&nbsp;<\/p>\n\n\n\n<p>From my experience, I didn\u2019t manage to get purely routed physical interfaces (e.g. Ethernet0) working. Regardless what I tried to do; they were all the time in the down state. That\u2019s why I stuck to the concept that each physical interface (e.g. Ethernet0) has a single untagged VLAN (e.g. VLAN130), whereas each VLAN has an SVI got an IP address (e.g. VLAN130 IP 192.168.1.1\/24) assigned.<\/p>\n\n\n\n<p>But that is only part of the tricks.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">#2. Logic behind the port mapping in the Docker container with Microsoft SONiC P4 switch<\/h4>\n\n\n\n<p>The next trick, which cost me quite a lot of time is the way how the container with the SONiC connects to the outside world. But, step by step.<\/p>\n\n\n\n<p>The first step challenge is that we are building the Docker container without the networking part, as we want to bypass the standard Docker bridge with NAT. Therefore, the container with the Microsoft Azure SONiC would be created as the following:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/><\/div><\/td><td><div class=\"text codecolorer\">\ufeffsudo docker run --net=none --privileged --entrypoint \/bin\/bash --name leaf11 -it -d -v $PWD\/infrastructure\/leaf11:\/sonic docker-sonic-p4:latest<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>The whole Bash file to build the lab is provided later in this blogpost.<\/p><\/blockquote>\n\n\n\n<p>This command would launch the Docker container mapping the directory with the configuration files mentioned in the previous point. As you see, we launch it with the disabled networking <strong>&#8211;net=none<\/strong>. Therefore, we will need to the network interfaces later.<\/p>\n\n\n\n<p>The Docker container is running as a process inside the Linux, and it has its associated process ID (PID). We need to get this PID in order to be able to connect our container somewhere:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/><\/div><\/td><td><div class=\"text codecolorer\">LEAF11=$(sudo docker inspect --format '{{ .State.Pid }}' leaf11)<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>The Linux\u2019s PID is stored in the variable having the hostname of the device. Now we are able to configure the networking between the P4 software switches running Microsoft Azure SONiC or Ubuntu-based Linux hosts. We connect them between each other using the built-in Linux network bridges, what is a way different to the original SONiC P4 repo:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/><\/div><\/td><td><div class=\"text codecolorer\">sudo brctl addbr s11_l11<br \/>\nsudo ip link set s11_l11 up<br \/>\nsudo ip link add sw_port0 type veth<br \/>\nsudo ip link set veth0 up<br \/>\nsudo brctl addif s11_l11 veth0<br \/>\nsudo ip link set netns ${SPINE11} dev sw_port0<br \/>\nsudo ip link add sw_port5 type veth<br \/>\nsudo ip link set veth1 up<br \/>\nsudo brctl addif s11_l11 veth1<br \/>\nsudo ip link set netns ${LEAF11} dev sw_port5<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>Using the <strong>veth<\/strong> type of the interface, we can map the interfaces inside the container. The naming convention of the interfaces with the Docker container with SONiC, how we connect them to the host is <strong>sw_portX<\/strong>, where <strong>X<\/strong> is the sequence interface number. The mapping between these interfaces and the SONiC\u2019s one are the following:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>\u201cphysical\u201d interrace<\/strong><\/td><td><strong>\u201cSONiC\u201d interface<\/strong><\/td><\/tr><tr><td>sw_port0<\/td><td>Ethernet0<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p>Within the Microsoft SONiC P4 switch we don\u2019t interact directly with these \u201cphysical\u201d interfaces. Therefore, we need to do some additional job inside the launched Docker container to make these interfaces working:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/><\/div><\/td><td><div class=\"text codecolorer\">\ufeff\ufeff\ufeff\ufeffsudo docker exec -d leaf11 ip netns add sw_net<br \/>\nsudo docker exec -d leaf11 ip link set dev sw_port0 netns sw_net<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net sysctl net.ipv6.conf.sw_port0.disable_ipv6=1<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net ip link set sw_port0 up<br \/>\nsudo docker exec -d leaf11 ip link set dev sw_port5 netns sw_net<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net sysctl net.ipv6.conf.sw_port5.disable_ipv6=1<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net ip link set sw_port5 up<br \/>\nsudo docker exec -d leaf11 ip link set dev sw_port6 netns sw_net<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net sysctl net.ipv6.conf.sw_port6.disable_ipv6=1<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net ip link set sw_port6 up<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>The snippet above create the interfaces with the proper names inside the Docker container and associated them with the namespace <strong>sw_net<\/strong>, which exists inside that container. In a nutshell, this namespace is something the Microsoft SONiC P4 switch uses to communicate outside the world, whereas internally we deal with the standard namespace upon the configuration of the SONiC itself.<\/p>\n\n\n\n<p>I might be wrong, but my understanding is that this is a way, how the P4 switch works in general (link to Mellanox P4). I haven\u2019t managed to get to the bottom of this complexity, but I\u2019ve managed to get that working.<\/p>\n\n\n\n<p>The last modification of the container we need to do after it is launched, is to run the script, which we have mapped to it:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/><\/div><\/td><td><div class=\"text codecolorer\">sudo docker exec -d leaf11 sh \/sonic\/scripts\/startup.sh<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>This script, as shared in the previous point, makes sure that the SONiC configuration is applied and the MAC addresses are set properly.<\/p>\n\n\n\n<p>The overall launch file contains all the necessary information such as Docker container details, the Linux bridges to interconnect the devices and configuration of the respective namespace network in the containers with Microsoft Azure SONiC. I took the original one<a href=\"https:\/\/github.com\/Azure\/SONiC\/wiki\/SONiC-P4-Software-Switch\"> from the Microsoft Azure SONiC P4 software switch GitHub repo<\/a> and modified it as follows:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/>16<br \/>17<br \/>18<br \/>19<br \/>20<br \/>21<br \/>22<br \/>23<br \/>24<br \/>25<br \/>26<br \/>27<br \/>28<br \/>29<br \/>30<br \/>31<br \/>32<br \/>33<br \/>34<br \/>35<br \/>36<br \/>37<br \/>38<br \/>39<br \/>40<br \/>41<br \/>42<br \/>43<br \/>44<br \/>45<br \/>46<br \/>47<br \/>48<br \/>49<br \/>50<br \/>51<br \/>52<br \/>53<br \/>54<br \/>55<br \/>56<br \/>57<br \/>58<br \/>59<br \/>60<br \/>61<br \/>62<br \/>63<br \/>64<br \/>65<br \/>66<br \/>67<br \/>68<br \/>69<br \/>70<br \/>71<br \/>72<br \/>73<br \/>74<br \/>75<br \/>76<br \/>77<br \/>78<br \/>79<br \/>80<br \/>81<br \/>82<br \/>83<br \/>84<br \/>85<br \/>86<br \/>87<br \/>88<br \/>89<br \/>90<br \/>91<br \/>92<br \/>93<br \/>94<br \/>95<br \/>96<br \/>97<br \/>98<br \/>99<br \/>100<br \/>101<br \/>102<br \/>103<br \/>104<br \/>105<br \/>106<br \/>107<br \/>108<br \/>109<br \/>110<br \/>111<br \/>112<br \/>113<br \/>114<br \/>115<br \/>116<br \/>117<br \/>118<br \/>119<br \/>120<br \/>121<br \/>122<br \/>123<br \/>124<br \/>125<br \/>126<br \/>127<br \/>128<br \/>129<br \/>130<br \/>131<br \/>132<br \/>133<br \/>134<br \/>135<br \/>136<br \/>137<br \/>138<br \/>139<br \/>140<br \/>141<br \/>142<br \/>143<br \/>144<br \/>145<br \/>146<br \/>147<br \/>148<br \/>149<br \/>150<br \/>151<br \/>152<br \/>153<br \/>154<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ cat build.sh <br \/>\n#!\/bin\/bash<br \/>\n<br \/>\necho &quot;Launching Docker: starting&quot;<br \/>\nsudo systemctl start docker.service<br \/>\necho &quot;Launching Docker: done&quot;<br \/>\n<br \/>\necho &quot;Creating the containers: starting&quot;<br \/>\nsudo docker run --net=none --privileged --entrypoint \/bin\/bash --name leaf11 -it -d -v $PWD\/infrastructure\/leaf11:\/sonic docker-sonic-p4:latest<br \/>\nsudo docker run --net=none --privileged --entrypoint \/bin\/bash --name leaf12 -it -d -v $PWD\/infrastructure\/leaf12:\/sonic docker-sonic-p4:latest<br \/>\nsudo docker run --net=none --privileged --entrypoint \/bin\/bash --name spine11 -it -d -v $PWD\/infrastructure\/spine11:\/sonic docker-sonic-p4:latest<br \/>\nsudo docker run --net=none --privileged --entrypoint \/bin\/bash --name spine12 -it -d -v $PWD\/infrastructure\/spine12:\/sonic docker-sonic-p4:latest<br \/>\nsudo docker run --net=none --privileged --entrypoint \/bin\/bash --name host11 -it -d ubuntu:14.04 <br \/>\nsudo docker run --net=none --privileged --entrypoint \/bin\/bash --name host12 -it -d ubuntu:14.04<br \/>\n<br \/>\nLEAF11=$(sudo docker inspect --format '{{ .State.Pid }}' leaf11)<br \/>\nLEAF12=$(sudo docker inspect --format '{{ .State.Pid }}' leaf12)<br \/>\nSPINE11=$(sudo docker inspect --format '{{ .State.Pid }}' spine11)<br \/>\nSPINE12=$(sudo docker inspect --format '{{ .State.Pid }}' spine12)<br \/>\nHOST11=$(sudo docker inspect --format '{{ .State.Pid }}' host11)<br \/>\nHOST12=$(sudo docker inspect --format '{{ .State.Pid }}' host12)<br \/>\necho &quot;Creating the containers: done&quot;<br \/>\n<br \/>\necho &quot;Creating the network connectivity: starting&quot;<br \/>\nsudo brctl addbr s11_l11<br \/>\nsudo ip link set s11_l11 up<br \/>\nsudo ip link add sw_port0 type veth<br \/>\nsudo ip link set veth0 up<br \/>\nsudo brctl addif s11_l11 veth0<br \/>\nsudo ip link set netns ${SPINE11} dev sw_port0<br \/>\nsudo ip link add sw_port5 type veth<br \/>\nsudo ip link set veth1 up<br \/>\nsudo brctl addif s11_l11 veth1<br \/>\nsudo ip link set netns ${LEAF11} dev sw_port5<br \/>\n<br \/>\nsudo brctl addbr s12_l11<br \/>\nsudo ip link set s12_l11 up<br \/>\nsudo ip link add sw_port0 type veth<br \/>\nsudo ip link set veth2 up<br \/>\nsudo brctl addif s12_l11 veth2<br \/>\nsudo ip link set netns ${SPINE12} dev sw_port0<br \/>\nsudo ip link add sw_port6 type veth<br \/>\nsudo ip link set veth3 up<br \/>\nsudo brctl addif s12_l11 veth3<br \/>\nsudo ip link set netns ${LEAF11} dev sw_port6<br \/>\n<br \/>\nsudo brctl addbr s11_l12<br \/>\nsudo ip link set s11_l12 up<br \/>\nsudo ip link add sw_port1 type veth<br \/>\nsudo ip link set veth4 up<br \/>\nsudo brctl addif s11_l12 veth4<br \/>\nsudo ip link set netns ${SPINE11} dev sw_port1<br \/>\nsudo ip link add sw_port5 type veth<br \/>\nsudo ip link set veth5 up<br \/>\nsudo brctl addif s11_l12 veth5<br \/>\nsudo ip link set netns ${LEAF12} dev sw_port5<br \/>\n<br \/>\nsudo brctl addbr s12_l12<br \/>\nsudo ip link set s12_l12 up<br \/>\nsudo ip link add sw_port1 type veth<br \/>\nsudo ip link set veth6 up<br \/>\nsudo brctl addif s12_l12 veth6<br \/>\nsudo ip link set netns ${SPINE12} dev sw_port1<br \/>\nsudo ip link add sw_port6 type veth<br \/>\nsudo ip link set veth7 up<br \/>\nsudo brctl addif s12_l12 veth7<br \/>\nsudo ip link set netns ${LEAF12} dev sw_port6<br \/>\n<br \/>\nsudo brctl addbr host11_leaf11<br \/>\nsudo ip link set host11_leaf11 up<br \/>\nsudo ip link add sw_port0 type veth<br \/>\nsudo ip link set veth8 up<br \/>\nsudo brctl addif host11_leaf11 veth8<br \/>\nsudo ip link set netns ${LEAF11} dev sw_port0<br \/>\nsudo ip link add eth1 type veth<br \/>\nsudo ip link set veth9 up<br \/>\nsudo brctl addif host11_leaf11 veth9<br \/>\nsudo ip link set netns ${HOST11} dev eth1<br \/>\n<br \/>\nsudo brctl addbr host12_leaf12<br \/>\nsudo ip link set host12_leaf12 up<br \/>\nsudo ip link add sw_port0 type veth<br \/>\nsudo ip link set veth10 up<br \/>\nsudo brctl addif host12_leaf12 veth10<br \/>\nsudo ip link set netns ${LEAF12} dev sw_port0<br \/>\nsudo ip link add eth1 type veth<br \/>\nsudo ip link set veth11 up<br \/>\nsudo brctl addif host12_leaf12 veth11<br \/>\nsudo ip link set netns ${HOST12} eth1<br \/>\necho &quot;Creating the network connectivity: done&quot;<br \/>\n<br \/>\necho &quot;Configuring hosts: starting&quot;<br \/>\nsudo docker exec -d host11 sysctl net.ipv6.conf.eth0.disable_ipv6=1<br \/>\nsudo docker exec -d host11 sysctl net.ipv6.conf.eth1.disable_ipv6=1<br \/>\nsudo docker exec -d host12 sysctl net.ipv6.conf.eth0.disable_ipv6=1<br \/>\nsudo docker exec -d host12 sysctl net.ipv6.conf.eth1.disable_ipv6=1<br \/>\n<br \/>\nsudo docker exec -d host11 ifconfig eth1 192.168.1.2\/24 mtu 1400<br \/>\nsudo docker exec -d host11 ip route replace default via 192.168.1.1<br \/>\nsudo docker exec -d host12 ifconfig eth1 192.168.2.2\/24 mtu 1400<br \/>\nsudo docker exec -d host12 ip route replace default via 192.168.2.1<br \/>\necho &quot;Configuring hosts: done&quot;<br \/>\n<br \/>\necho &quot;Configuring switches: starting&quot;<br \/>\nsudo docker exec -d leaf11 ip netns add sw_net<br \/>\nsudo docker exec -d leaf11 ip link set dev sw_port0 netns sw_net<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net sysctl net.ipv6.conf.sw_port0.disable_ipv6=1<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net ip link set sw_port0 up<br \/>\nsudo docker exec -d leaf11 ip link set dev sw_port5 netns sw_net<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net sysctl net.ipv6.conf.sw_port5.disable_ipv6=1<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net ip link set sw_port5 up<br \/>\nsudo docker exec -d leaf11 ip link set dev sw_port6 netns sw_net<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net sysctl net.ipv6.conf.sw_port6.disable_ipv6=1<br \/>\nsudo docker exec -d leaf11 ip netns exec sw_net ip link set sw_port6 up<br \/>\n<br \/>\nsudo docker exec -d leaf12 ip netns add sw_net<br \/>\nsudo docker exec -d leaf12 ip link set dev sw_port0 netns sw_net<br \/>\nsudo docker exec -d leaf12 ip netns exec sw_net sysctl net.ipv6.conf.sw_port0.disable_ipv6=1<br \/>\nsudo docker exec -d leaf12 ip netns exec sw_net ip link set sw_port0 up<br \/>\nsudo docker exec -d leaf12 ip link set dev sw_port5 netns sw_net<br \/>\nsudo docker exec -d leaf12 ip netns exec sw_net sysctl net.ipv6.conf.sw_port5.disable_ipv6=1<br \/>\nsudo docker exec -d leaf12 ip netns exec sw_net ip link set sw_port5 up<br \/>\nsudo docker exec -d leaf12 ip link set dev sw_port6 netns sw_net<br \/>\nsudo docker exec -d leaf12 ip netns exec sw_net sysctl net.ipv6.conf.sw_port6.disable_ipv6=1<br \/>\nsudo docker exec -d leaf12 ip netns exec sw_net ip link set sw_port6 up<br \/>\n<br \/>\nsudo docker exec -d spine11 ip netns add sw_net<br \/>\nsudo docker exec -d spine11 ip link set dev sw_port0 netns sw_net<br \/>\nsudo docker exec -d spine11 ip netns exec sw_net sysctl net.ipv6.conf.sw_port0.disable_ipv6=1<br \/>\nsudo docker exec -d spine11 ip netns exec sw_net ip link set sw_port0 up<br \/>\nsudo docker exec -d spine11 ip link set dev sw_port1 netns sw_net<br \/>\nsudo docker exec -d spine11 ip netns exec sw_net sysctl net.ipv6.conf.sw_port1.disable_ipv6=1<br \/>\nsudo docker exec -d spine11 ip netns exec sw_net ip link set sw_port1 up<br \/>\n<br \/>\nsudo docker exec -d spine12 ip netns add sw_net<br \/>\nsudo docker exec -d spine12 ip link set dev sw_port0 netns sw_net<br \/>\nsudo docker exec -d spine12 ip netns exec sw_net sysctl net.ipv6.conf.sw_port0.disable_ipv6=1<br \/>\nsudo docker exec -d spine12 ip netns exec sw_net ip link set sw_port0 up<br \/>\nsudo docker exec -d spine12 ip link set dev sw_port1 netns sw_net<br \/>\nsudo docker exec -d spine12 ip netns exec sw_net sysctl net.ipv6.conf.sw_port1.disable_ipv6=1<br \/>\nsudo docker exec -d spine12 ip netns exec sw_net ip link set sw_port1 up<br \/>\necho &quot;Configuring switches: done&quot;<br \/>\n<br \/>\necho &quot;Booting switches, please wait ~1 minute for switches to load: starting&quot;<br \/>\nsudo docker exec -d leaf11 sh \/sonic\/scripts\/startup.sh<br \/>\nsudo docker exec -d leaf12 sh \/sonic\/scripts\/startup.sh<br \/>\nsudo docker exec -d spine11 sh \/sonic\/scripts\/startup.sh<br \/>\nsudo docker exec -d spine12 sh \/sonic\/scripts\/startup.sh<br \/>\nsleep 70<br \/>\necho &quot;Booting switches, please wait ~1 minute for switches to load: done&quot;<br \/>\n<br \/>\necho &quot;Fixing iptables firewall: starting&quot;<br \/>\nsudo iptables -I FORWARD 1 -s 10.0.0.0\/24 -d 10.0.0.0\/24 -j ACCEPT<br \/>\nsudo iptables -I FORWARD 1 -s 192.168.0.0\/16 -d 192.168.0.0\/16 -j ACCEPT<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>Launching this Bash script, we can bring the topology up.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>Join our network automation training (link), if you are interested in learning how to build the proper Bash or Python script to rock your network with automation.<\/p><\/blockquote>\n\n\n\n<h4 class=\"wp-block-heading\">#3. Launching the Docker containers with Microsoft SONiC P4 switches and Ubuntu hosts<\/h4>\n\n\n\n<p>However, before we bring the topology up, we need to download the SONiC image. Based on the original repo, you can do it like this:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/><\/div><\/td><td><div class=\"text codecolorer\">\ufeff$ cat load_image.sh <br \/>\nwget https:\/\/sonic-jenkins.westus2.cloudapp.azure.com\/job\/p4\/job\/buildimage-p4-all\/543\/artifact\/target\/docker-sonic-p4.gz<br \/>\nsudo docker load &amp;lt; docker-sonic-p4.gz<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>This link downloads the image from the corresponding webpage and adds it to your Docker images repository.<\/p>\n\n\n\n<p>Therefore, step number is one is to make sure that the Docker service on your host is up and running: <\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo systemctl start docker.service<br \/>\n<br \/>\n<br \/>\n\ufeff$ sudo systemctl status docker.service <br \/>\n\u25cf docker.service - Docker Application Container Engine<br \/>\n&nbsp; &nbsp;Loaded: loaded (\/usr\/lib\/systemd\/system\/docker.service; disabled; vendor preset: disabled)<br \/>\n&nbsp; &nbsp;Active: active (running) since Sun 2020-03-15 19:49:07 GMT; 1min 20s ago<br \/>\n&nbsp; &nbsp; &nbsp;Docs: https:\/\/docs.docker.com<br \/>\n&nbsp;Main PID: 5607 (dockerd)<br \/>\n&nbsp; &nbsp; Tasks: 11<br \/>\n&nbsp; &nbsp;Memory: 146.8M<br \/>\n&nbsp; &nbsp;CGroup: \/system.slice\/docker.service<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\u2514\u25005607 \/usr\/bin\/dockerd -H fd:\/\/ --containerd=\/run\/containerd\/containerd.sock<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>The second step is to launch the script mentioned above:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ .\/load_image.sh <br \/>\n--2020-03-15 19:48:17-- &nbsp;https:\/\/sonic-jenkins.westus2.cloudapp.azure.com\/job\/p4\/job\/buildimage-p4-all\/543\/artifact\/target\/docker-sonic-p4.gz<br \/>\nResolving sonic-jenkins.westus2.cloudapp.azure.com (sonic-jenkins.westus2.cloudapp.azure.com)... 52.250.106.22<br \/>\nConnecting to sonic-jenkins.westus2.cloudapp.azure.com (sonic-jenkins.westus2.cloudapp.azure.com)|52.250.106.22|:443... connected.<br \/>\nHTTP request sent, awaiting response... 200 OK<br \/>\nLength: 164639463 (157M) &amp;#91;application\/x-gzip]<br \/>\nSaving to: \u2018docker-sonic-p4.gz\u2019<br \/>\n<br \/>\n100%&amp;#91;====================================================================================================================&amp;gt;] 164,639,463 1.05MB\/s &nbsp; in 2m 44s <br \/>\n<br \/>\n2020-03-15 19:51:03 (978 KB\/s) - \u2018docker-sonic-p4.gz\u2019 saved &amp;#91;164639463\/164639463]<br \/>\n<br \/>\nLoaded image: docker-sonic-p4:latest<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>The third step is to verify the image is added properly to the Docker images:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker image ls<br \/>\nREPOSITORY &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; TAG &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; IMAGE ID &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;CREATED &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; SIZE<br \/>\nubuntu &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 14.04 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 6e4f1fe62ff1 &nbsp; &nbsp; &nbsp; &nbsp;2 months ago &nbsp; &nbsp; &nbsp; &nbsp;197MB<br \/>\nakarneliuk\/dcf_ftp &nbsp; latest &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;aeca611f7115 &nbsp; &nbsp; &nbsp; &nbsp;10 months ago &nbsp; &nbsp; &nbsp; 8MB<br \/>\ndocker-sonic-p4 &nbsp; &nbsp; &nbsp;latest &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;a62359e719f0 &nbsp; &nbsp; &nbsp; &nbsp;2 years ago &nbsp; &nbsp; &nbsp; &nbsp; 445MB<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>Now we can bring our topology with Microsoft Azure SONiC up:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/>16<br \/>17<br \/>18<br \/>19<br \/>20<br \/>21<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ .\/build.sh <br \/>\nLaunching Docker: starting<br \/>\nLaunching Docker: done<br \/>\nCreating the containers: starting<br \/>\n1cd17d8872a3d9127493a0af2d94a5472a4dbdfc5912967771322e05b306d529<br \/>\n229b87ba9d116c9ecef4a0190eeb14418d02ce6daa185b74fb3929fce6754a8b<br \/>\n37215b3837199b084f48500883a0dd16db9cbd6cd65a54402593cad3818fb60f<br \/>\n35bca165764f98747020eea1345c58e483dd896ddf7ff317e320c110f5734cec<br \/>\ndd851bd9ee34c8c7ea180edb078dad3961c7a602c1e5edb18e0e0df0a5206988<br \/>\n6b1ec0df840a6bfe841a22c8a3817897b14c1b4b5c1f50a8453790f1e129a1a7<br \/>\nCreating the containers: done<br \/>\nCreating the network connectivity: starting<br \/>\nCreating the network connectivity: done<br \/>\nConfiguring hosts: starting<br \/>\nConfiguring hosts: done<br \/>\nConfiguring switches: starting<br \/>\nConfiguring switches: done<br \/>\nBooting switches, please wait ~1 minute for switches to load: starting<br \/>\nBooting switches, please wait ~1 minute for switches to load: done<br \/>\nFixing iptables firewall: starting<br \/>\nFixing iptables firewall: done<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>Now we can get some brew and wait for some minutes to allow the containers to boot and the topology to converge.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Topology verification<\/h3>\n\n\n\n<p>The first step we need to check in our created hyper-scaler is whether our containers are booted properly:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker container ls<br \/>\n&amp;#91;sudo] password for aaa: <br \/>\nCONTAINER ID &nbsp; &nbsp; &nbsp; &nbsp;IMAGE &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;COMMAND &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; CREATED &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; STATUS &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;PORTS &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; NAMES<br \/>\n6b1ec0df840a &nbsp; &nbsp; &nbsp; &nbsp;ubuntu:14.04 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;\/bin\/bash&quot; &nbsp; &nbsp; &nbsp; &nbsp; 7 minutes ago &nbsp; &nbsp; &nbsp; Up 7 minutes &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;host12<br \/>\ndd851bd9ee34 &nbsp; &nbsp; &nbsp; &nbsp;ubuntu:14.04 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;\/bin\/bash&quot; &nbsp; &nbsp; &nbsp; &nbsp; 7 minutes ago &nbsp; &nbsp; &nbsp; Up 7 minutes &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;host11<br \/>\n35bca165764f &nbsp; &nbsp; &nbsp; &nbsp;docker-sonic-p4:latest &nbsp; &quot;\/bin\/bash&quot; &nbsp; &nbsp; &nbsp; &nbsp; 7 minutes ago &nbsp; &nbsp; &nbsp; Up 7 minutes &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;spine12<br \/>\n37215b383719 &nbsp; &nbsp; &nbsp; &nbsp;docker-sonic-p4:latest &nbsp; &quot;\/bin\/bash&quot; &nbsp; &nbsp; &nbsp; &nbsp; 7 minutes ago &nbsp; &nbsp; &nbsp; Up 7 minutes &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;spine11<br \/>\n229b87ba9d11 &nbsp; &nbsp; &nbsp; &nbsp;docker-sonic-p4:latest &nbsp; &quot;\/bin\/bash&quot; &nbsp; &nbsp; &nbsp; &nbsp; 7 minutes ago &nbsp; &nbsp; &nbsp; Up 7 minutes &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;leaf12<br \/>\n1cd17d8872a3 &nbsp; &nbsp; &nbsp; &nbsp;docker-sonic-p4:latest &nbsp; &quot;\/bin\/bash&quot; &nbsp; &nbsp; &nbsp; &nbsp; 7 minutes ago &nbsp; &nbsp; &nbsp; Up 7 minutes &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;leaf11<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>Once that is done, we can check the interfaces inside the container with Microsoft Azure SONiC P4 switch:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/>16<br \/>17<br \/>18<br \/>19<br \/>20<br \/>21<br \/>22<br \/>23<br \/>24<br \/>25<br \/>26<br \/>27<br \/>28<br \/>29<br \/>30<br \/>31<br \/>32<br \/>33<br \/>34<br \/>35<br \/>36<br \/>37<br \/>38<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker container exec -it leaf11 ip link show<br \/>\n&amp;#91;sudo] password for aaa: <br \/>\n1: lo: &amp;lt;LOOPBACK,UP,LOWER_UP&amp;gt; mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00<br \/>\n2: eth0: &amp;lt;BROADCAST,NOARP&amp;gt; mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 00:dc:5e:01:01:01 brd ff:ff:ff:ff:ff:ff<br \/>\n3: host_port1@if4: &amp;lt;NO-CARRIER,BROADCAST,MULTICAST,UP&amp;gt; mtu 1500 qdisc noqueue state LOWERLAYERDOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 5a:da:c3:34:cf:d4 brd ff:ff:ff:ff:ff:ff link-netnsid 1<br \/>\n5: host_port2@if6: &amp;lt;NO-CARRIER,BROADCAST,MULTICAST,UP&amp;gt; mtu 1500 qdisc noqueue state LOWERLAYERDOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 66:16:8f:5f:b5:1b brd ff:ff:ff:ff:ff:ff link-netnsid 1ago &nbsp; &nbsp; &nbsp; Up 7 minutes &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;leaf11<br \/>\n!<br \/>\n! Some output is truncated for brevity<br \/>\n!<br \/>\n\ufeff67: Ethernet0: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 9100 qdisc pfifo_fast master Bridge state UNKNOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether b6:91:5b:15:4d:20 brd ff:ff:ff:ff:ff:ff<br \/>\n68: Ethernet1: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 8a:66:6c:84:12:e4 brd ff:ff:ff:ff:ff:ff<br \/>\n69: Ethernet2: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether ae:d0:7d:40:58:a4 brd ff:ff:ff:ff:ff:ff<br \/>\n70: Ethernet3: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 96:53:90:81:55:e6 brd ff:ff:ff:ff:ff:ff<br \/>\n71: Ethernet4: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether d2:1c:49:7b:28:c9 brd ff:ff:ff:ff:ff:ff<br \/>\n72: Ethernet5: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 9100 qdisc pfifo_fast master Bridge state UNKNOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 3a:c9:4e:77:ba:81 brd ff:ff:ff:ff:ff:ff<br \/>\n73: Ethernet6: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 9100 qdisc pfifo_fast master Bridge state UNKNOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 5a:ba:ac:07:9a:4b brd ff:ff:ff:ff:ff:ff<br \/>\n!<br \/>\n! Some output is truncated for brevity<br \/>\n!<br \/>\n\ufeff99: Bridge: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 9100 qdisc noqueue state UP mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 3a:c9:4e:77:ba:81 brd ff:ff:ff:ff:ff:ff<br \/>\n100: Vlan130@Bridge: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 00:dc:5e:01:01:01 brd ff:ff:ff:ff:ff:ff<br \/>\n101: Vlan131@Bridge: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 00:dc:5e:01:01:01 brd ff:ff:ff:ff:ff:ff<br \/>\n102: Vlan132@Bridge: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 00:dc:5e:01:01:01 brd ff:ff:ff:ff:ff:ff<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>There are multiple interfaces down. However, the ones, which we use, Ethernet0, Ethernet5 and Ethernet6 are up. So that the VLAN interfaces, which we have created. What was the point of mentioning the namespace <strong>sw_net<\/strong> earlier? Let\u2019s take a look here:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/>16<br \/>17<br \/>18<br \/>19<br \/>20<br \/>21<br \/>22<br \/>23<br \/>24<br \/>25<br \/>26<br \/>27<br \/>28<br \/>29<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker container exec -it spine11 ip netns exec sw_net ip link show<br \/>\n1: lo: &amp;lt;LOOPBACK&amp;gt; mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00<br \/>\n2: sw_port5@if9: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 22:b9:ea:49:ff:6b brd ff:ff:ff:ff:ff:ff link-netnsid 1<br \/>\n3: sw_port10@if19: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether ea:4a:d9:88:2b:9d brd ff:ff:ff:ff:ff:ff link-netnsid 1<br \/>\n4: sw_port2@if3: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 56:6f:02:9e:2d:2b brd ff:ff:ff:ff:ff:ff link-netnsid 1<br \/>\n6: sw_port3@if5: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether f2:fa:22:b5:c2:35 brd ff:ff:ff:ff:ff:ff link-netnsid 1<br \/>\n8: sw_port4@if7: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether c6:6f:2e:78:4c:4c brd ff:ff:ff:ff:ff:ff link-netnsid 1<br \/>\n10: sw_port0@if9: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 00:dc:5e:01:00:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0<br \/>\n12: sw_port6@if11: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000<br \/>\n!<br \/>\n! Some output is truncated for brevity<br \/>\n!<br \/>\n\ufeff62: sw_port31@if61: &amp;lt;BROADCAST,MULTICAST&amp;gt; mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether d6:45:43:47:b3:5d brd ff:ff:ff:ff:ff:ff link-netnsid 1<br \/>\n64: cpu_port@if63: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether c2:5d:81:d8:8b:11 brd ff:ff:ff:ff:ff:ff link-netnsid 1<br \/>\n65: router_port1@router_port0: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether 76:32:87:29:78:cf brd ff:ff:ff:ff:ff:ff<br \/>\n66: router_port0@router_port1: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether f2:a1:66:5d:53:99 brd ff:ff:ff:ff:ff:ff<br \/>\n68: router_cpu_port@if67: &amp;lt;BROADCAST,MULTICAST,UP,LOWER_UP&amp;gt; mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000<br \/>\n&nbsp; &nbsp; link\/ether de:60:a4:19:78:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 1<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>In this namespace you can see the ports <strong>sw_port0<\/strong>, <strong>sw_port5<\/strong> and <strong>sw_port6<\/strong>, which we have created in the original Bash script. The mapping of the interfaces are explained above, so we can don\u2019t focus on that now.<\/p>\n\n\n\n<p>Putting aside the complexity with all the multiple namespaces caused by the Software Switch P4, we can focus on the SONiC itself.<\/p>\n\n\n\n<p>Let\u2019s check the Microsoft Azure SONiC version, <a href=\"http:\/\/bit.ly\/2SktMT1\">like we did earlier on the Mellanox SN2010 switches<\/a>:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker container exec -it leaf11 show version<br \/>\nTraceback (most recent call last):<br \/>\n&nbsp; File &quot;\/usr\/local\/bin\/sonic-cfggen&quot;, line 220, in &amp;lt;module&amp;gt;<br \/>\n&nbsp; &nbsp; main()<br \/>\n&nbsp; File &quot;\/usr\/local\/bin\/sonic-cfggen&quot;, line 176, in main<br \/>\n&nbsp; &nbsp; with open(yaml_file, 'r') as stream:<br \/>\nIOError: &amp;#91;Errno 2] No such file or directory: '\/etc\/sonic\/sonic_version.yml'<br \/>\n<br \/>\nDocker images:<br \/>\n\ufeff\/bin\/sh: 1: sudo: not found<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>It looks like the implementation is not complete (or not fully complete). However, we have seen the VLAN interfaces, what means that the configuration was applied. Let\u2019s try to reach the spine switches and the host:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/>16<br \/>17<br \/>18<br \/>19<br \/>20<br \/>21<br \/>22<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker container exec -it leaf11 ping 10.0.0.0 -c 1<br \/>\nPING 10.0.0.0 (10.0.0.0): 56 data bytes<br \/>\n64 bytes from 10.0.0.0: icmp_seq=0 ttl=64 time=6.771 ms<br \/>\n--- 10.0.0.0 ping statistics ---<br \/>\n1 packets transmitted, 1 packets received, 0% packet loss<br \/>\nround-trip min\/avg\/max\/stddev = 6.771\/6.771\/6.771\/0.000 ms<br \/>\n<br \/>\n<br \/>\n$ sudo docker container exec -it leaf11 ping 10.0.0.4 -c 1<br \/>\nPING 10.0.0.4 (10.0.0.4): 56 data bytes<br \/>\n64 bytes from 10.0.0.4: icmp_seq=0 ttl=64 time=8.064 ms<br \/>\n--- 10.0.0.4 ping statistics ---<br \/>\n1 packets transmitted, 1 packets received, 0% packet loss<br \/>\nround-trip min\/avg\/max\/stddev = 8.064\/8.064\/8.064\/0.000 ms<br \/>\n<br \/>\n<br \/>\n$ sudo docker container exec -it leaf11 ping 192.168.1.2 -c 1<br \/>\nPING 192.168.1.2 (192.168.1.2): 56 data bytes<br \/>\n64 bytes from 192.168.1.2: icmp_seq=0 ttl=64 time=9.607 ms<br \/>\n--- 192.168.1.2 ping statistics ---<br \/>\n1 packets transmitted, 1 packets received, 0% packet loss<br \/>\nround-trip min\/avg\/max\/stddev = 9.607\/9.607\/9.607\/0.000 ms<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>The reachability check is successful, which means that our networking part works correct.<\/p>\n\n\n\n<p>The next point is to check the BGP routing:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker container exec -it leaf11 vtysh -c &quot;show bgp ipv4 unicast&quot;<br \/>\n&amp;#91;sudo] password for aaa: <br \/>\nBGP table version is 0, local router ID is 10.1.1.1<br \/>\nStatus codes: s suppressed, d damped, h history, * valid, &amp;gt; best, = multipath,<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; i internal, r RIB-failure, S Stale, R Removed<br \/>\nOrigin codes: i - IGP, e - EGP, ? - incomplete<br \/>\n<br \/>\n&nbsp; &nbsp;Network &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Next Hop &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Metric LocPrf Weight Path<br \/>\n*&amp;gt; 192.168.1.0 &nbsp; &nbsp; &nbsp;0.0.0.0 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;0 &nbsp; &nbsp; &nbsp; &nbsp; 32768 i<br \/>\n*= 192.168.2.0 &nbsp; &nbsp; &nbsp;10.0.0.0 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0 65101 65112 i<br \/>\n*&amp;gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;10.0.0.4 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0 65102 65112 i<br \/>\n<br \/>\nTotal number of prefixes 2<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>Per our idea, we don\u2019t need to advertise the transit links, therefore we have only customer subnets. On the other hand, as you know, the BGP RIB is a BGP control plane, what might deviate from the routing plane. In Linux you can check the routing table as follows:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker container exec -it leaf11 ip route show<br \/>\n10.0.0.0\/31 dev Vlan131 proto kernel scope link src 10.0.0.1 <br \/>\n10.0.0.4\/31 dev Vlan132 proto kernel scope link src 10.0.0.5 <br \/>\n192.168.1.0\/24 dev Vlan130 proto kernel scope link src 192.168.1.1 <br \/>\n192.168.2.0\/24 proto zebra <br \/>\n&nbsp; &nbsp; nexthop via 10.0.0.4 &nbsp;dev Vlan132 weight 1<br \/>\n&nbsp; &nbsp; nexthop via 10.0.0.0 &nbsp;dev Vlan131 weight 1<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>So, it looks like everything is ready to start sending the traffic between the Ubuntu hosts.&nbsp;<\/p>\n\n\n\n<p>Let\u2019s take a look on the routing table at the host11:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker container exec -it host11 ip route show<br \/>\ndefault via 192.168.1.1 dev eth1 <br \/>\n192.168.1.0\/24 dev eth1 &nbsp;proto kernel &nbsp;scope link &nbsp;src 192.168.1.2<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>It looks accurate and we can try to ping host12:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker container exec -it host11 ping 192.168.2.2 -c 1<br \/>\nPING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.<br \/>\n^C<br \/>\n--- 192.168.2.2 ping statistics ---<br \/>\n1 packets transmitted, 0 received, 100% packet loss, time 0ms<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>I don&#8217;t want create too much mystery, there is a bug explained in the <a href=\"https:\/\/github.com\/Azure\/SONiC\/wiki\/SONiC-P4-Software-Switch\">original test file<\/a>:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ cat test.sh <br \/>\n#!\/bin\/bash<br \/>\n# First ping from host 2 to switch 2 - this is a patch:<br \/>\n# currently there is a bug with miss on neighbor table (does not trap by default as should)<br \/>\n# When fixed, we can remove it<br \/>\nsudo docker exec -it host12 ping 192.168.2.1 -c1<br \/>\nsleep 2<br \/>\nsudo docker exec -it host11 ping 192.168.2.2<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>Therefore, we replicate this solution:<\/p>\n\n\n\n<pre class=\"wp-block-code\">\n\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/><\/div><\/td><td><div class=\"text codecolorer\">$ sudo docker container exec -it host12 ping 192.168.2.1 -c 1<br \/>\nPING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.<br \/>\n64 bytes from 192.168.2.1: icmp_seq=1 ttl=64 time=3.76 ms<br \/>\n<br \/>\n--- 192.168.2.1 ping statistics ---<br \/>\n1 packets transmitted, 1 received, 0% packet loss,<br \/>\n<br \/>\n<br \/>\n\ufeff$ sudo docker container exec -it host11 ping 192.168.2.2 -c 1<br \/>\nPING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.<br \/>\n64 bytes from 192.168.2.2: icmp_seq=1 ttl=61 time=16.6 ms<br \/>\n<br \/>\n--- 192.168.2.2 ping statistics ---<br \/>\n1 packets transmitted, 1 received, 0% packet loss, time 0ms<br \/>\nrtt min\/avg\/max\/mdev = 16.605\/16.605\/16.605\/0.000 ms<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n\n<\/pre>\n\n\n\n<p>The end to end connectivity between the Ubuntu hosts is established and the traffic is passing our leaf\/spine micro hyper-scale fabric running Microsoft Azure SONiC. Feeling like we\u2019ve just built our own Azure Cloud \ud83d\ude09<\/p>\n\n\n\n<p>You can find the topology and the relevant files <a href=\"https:\/\/bit.ly\/2ysFymx\">on our GitHub page<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Lessons learned<\/h3>\n\n\n\n<p>It took me quite a while to dig into the Linux networking on a level deeper than Linux bridges and ip link\/addr\/route. There were a lot of things, which were not working from the beginning and required fixing. One of the biggest issues is that there is almost no documentation from the usage standpoint (only for the HW\/SW developers, what is not an easy read for the network engineers). Therefore, the biggest lessons learned is that it is required to have cross functional skills (network, programming, Linux) to succeed in the world of cloud builders.<br><\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"http:\/\/bit.ly\/2mP3SJy\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3-1024x576.png\" alt=\"\" class=\"wp-image-3695\" srcset=\"https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3-1024x576.png 1024w, https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3-300x169.png 300w, https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3-768x432.png 768w, https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3-1536x864.png 1536w, https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3-711x400.png 711w, https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3-409x230.png 409w, https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3-600x338.png 600w, https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3-1422x800.png 1422w, https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3-817x460.png 817w, https:\/\/karneliuk.com\/wp-content\/uploads\/2019\/11\/nat-us-3.png 1800w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Conclusion<\/h3>\n\n\n\n<p>We have made a significant step into an understanding of the operations of the Microsoft Azure SONiC and built a local environment with a leaf\/spine topology. We know that SONiC doesn\u2019t support EVPN\/VXLAN, which is necessary for an enterprise data centres, therefore we can\u2019t test those technologies. However, we will try to build a mixed environment with multiple vendors, as SONiC could perfectly be a spines, aggregation spines and A-Z switches. More to come on Microsoft Azure SONiC automation as well, hopefully soon. Take care and goodbye!&nbsp;<br><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Support us<\/h3>\n\n\n\n<p><style>\r\n        .wpedon-container .wpedon-select,\r\n        .wpedon-container .wpedon-input {\r\n            width: 171px;\r\n            min-width: 171px;\r\n            max-width: 171px;\r\n        }\r\n    <\/style><div class='wpedon-container wpedon-align-center'><label id='wpedon-1919-name-label'>Support new interop and automation articles at karneliuk.com<br \/><span class='price'><\/span>EUR<\/label><br \/>\r\n\t\t<script>\r\n\t\tjQuery(document).ready(function(){\r\n\t\t\tjQuery('#dd_88889c56ba45c146e61200ba6f741249').on('change', function() {\r\n\t\t\t  jQuery('#amount_88889c56ba45c146e61200ba6f741249').val(this.value);\r\n              jQuery('#price_88889c56ba45c146e61200ba6f741249').val(this.value);\r\n\t\t\t});\r\n\t\t});\r\n\t\t<\/script>\r\n\t\t<br \/><label style='font-size:11pt !important;'>I want to support with:<\/label><br \/><select class='wpedon-select' name='dd_88889c56ba45c146e61200ba6f741249' id='dd_88889c56ba45c146e61200ba6f741249'><option value='9.99'>9.99 EUR<\/option><option value='4.99'>4.99 EUR<\/option><option value='24.99'>24.99 EUR<\/option><option value='49.99'>49.99 EUR<\/option><option value='99.99'>99.99 EUR<\/option><option value='199.99'>199.99 EUR<\/option><\/select><br \/><br \/><form target='_blank' action='https:\/\/www.paypal.com\/cgi-bin\/webscr' method='post' class='wpedon-form'><input type='hidden' name='cmd' value='_donations' \/><input type='hidden' name='business' value='MZVY3WH2X7HPN' \/><input type='hidden' name='currency_code' value='EUR' \/><input type='hidden' name='notify_url' value='https:\/\/karneliuk.com\/wp-admin\/admin-post.php?action=add_wpedon_button_ipn'><input type='hidden' name='lc' value='en_US'><input type='hidden' name='bn' value='WPPlugin_SP'><input type='hidden' name='return' value='http:\/\/karneliuk.com\/thanks\/' \/><input type='hidden' name='cancel_return' value='' \/><input class='wpedon_paypalbuttonimage' type='image' src='https:\/\/www.paypal.com\/en_US\/i\/btn\/btn_donateCC_LG.gif' border='0' name='submit' alt='Make your payments with PayPal. It is free, secure, effective.' style='border: none;'><img alt='' border='0' style='border:none;display:none;' src='https:\/\/www.paypal.com\/en_US\/i\/scr\/pixel.gif' width='1' height='1'><input type='hidden' name='amount' id='amount_88889c56ba45c146e61200ba6f741249' value='9.99' \/><input type='hidden' name='price' id='price_88889c56ba45c146e61200ba6f741249' value='9.99' \/><input type='hidden' name='item_number' value='karneliuk.com-general' \/><input type='hidden' name='item_name' value='Support new interop and automation articles at karneliuk.com' \/><input type='hidden' name='name' value='Support new interop and automation articles at karneliuk.com' \/><input type='hidden' name='custom' value='1919'><input type='hidden' name='no_shipping' value='1'><input type='hidden' name='no_note' value='0'><input type='hidden' name='currency_code' value='EUR'><\/form><\/div><br><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">P.S.<\/h3>\n\n\n\n<p>If you have further questions or you need help with your networks, I\u2019m happy to assist you, just send me message (<a href=\"http:\/\/karneliuk.com\/contact\/\">http:\/\/karneliuk.com\/contact\/<\/a>). Also don\u2019t forget to share the article on your social media, if you like it.<\/p>\n\n\n\n<p>BR,<\/p>\n\n\n\n<p>Anton Karneliuk <\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hello my friend, In the previous article from the networking series we have started the discussion about SONiC (Software for Open Networking in Clouds), which is a network infrastructure behind Microsoft Azure cloud. Today we continue this discussion from slightly different angle: we will emulate the whole data centre infrastructure end-to-end with leafs, spines and&hellip;<\/p>\n","protected":false},"author":1,"featured_media":3380,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[16],"tags":[29,141,175,178,43,177],"class_list":{"0":"post-3373","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-network","8":"tag-bgp","9":"tag-data-center","10":"tag-mellanox","11":"tag-microsoft","12":"tag-sdn","13":"tag-sonic"},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/posts\/3373","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/comments?post=3373"}],"version-history":[{"count":12,"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/posts\/3373\/revisions"}],"predecessor-version":[{"id":4700,"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/posts\/3373\/revisions\/4700"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/media\/3380"}],"wp:attachment":[{"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/media?parent=3373"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/categories?post=3373"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/karneliuk.com\/wp-json\/wp\/v2\/tags?post=3373"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}