Skip to content

[continuous_link_flap.yml]: Continuous link flap test.#810

Merged
qiluo-msft merged 4 commits intosonic-net:masterfrom
praveen-li:continous_link_flap_merge
Jul 8, 2019
Merged

[continuous_link_flap.yml]: Continuous link flap test.#810
qiluo-msft merged 4 commits intosonic-net:masterfrom
praveen-li:continous_link_flap_merge

Conversation

@praveen-li
Copy link
Copy Markdown
Member

This is continuous link flap test. In this test,
1.) Flap all interfaces one by one to cause BGP Flaps on DUT (3 iterations).
2.) Flap all interfaces on peer (FanOutLeaf) one by one to cause BGP Flaps (3 iterations).
3.) Watch for memory (show system-memory) ,orchagent CPU Utilization and Redis_memory.

Pass Criteria: All routes must be re-learned with < 5% increase in Redis memory
and with Orchagent CPU consumption below 10% after 90 secs of stopping flaps.

Signed-off-by: Praveen Chaudharypchaudhary@linkedin.com

Description of PR

[continuous_link_flap.yml]: Continuous link flap test.

This is a new test case to check Redis memory increase for link flaps.
Test also watches for orchagent cpu usage and system memory, but not necessary include them while deciding pass criteria.

Summary:
This is continuous link flap test. In this test,
1.) Flap all interfaces one by one to cause BGP Flaps on DUT (3 iterations).
2.) Flap all interfaces on peer (FanOutLeaf) one by one to cause BGP Flaps (3 iterations).
3.) Watch for memory (show system-memory) ,orchagent CPU Utilization and Redis_memory.
Fixes # (issue)

Type of change

  • [] Bug fix
  • [] Testbed and Framework(new/improvement)
  • [-] Test case(new/improvement)

Approach

How did you do it?

In this test:
First:
1.) I record Redis Memory and ipv4 routes at start.
2.) Make sure orchagent consuming then than 10% cpu before starting the test.
3.) I flap all interfaces one by one to cause BGP Flaps on DUT (3 iterations).
4.) then I flap all interfaces on peer (FanOutLeaf) one by one to cause BGP Flaps (3 iterations).
5.) Test continuously watches for memory (show system-memory) ,orchagent CPU Utilization and Redis_memory.
6.) At last compares Total Redis memory difference and ipv6 routes.

Pass Criteria is based on:
1.) All routes must be re-learned
2.) with < 5% increase in Redis memory
3.) and with Orchagent CPU consumption below 10% after 90 secs of stopping flaps.

How did you verify/test it?

Ran on T0 and T1.
On t0 Redis memory consumption is less.
On T1 Redis memory consumption increases by > 30%

Output Snippet:

TASK [test : run test case continuous_link_flap.yml file] **********************

Wednesday 13 February 2019  17:53:09 +0000 (0:00:00.136)       0:00:23.695 **** 

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap.yml for falco-test-dut01


TASK [test : Record  memory status at start] ***********************************

Wednesday 13 February 2019  17:53:10 +0000 (0:00:00.605)       0:00:24.301 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:11 +0000 (0:00:00.955)       0:00:25.257 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       955M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : Record Redis Memory at start] *************************************

Wednesday 13 February 2019  17:53:11 +0000 (0:00:00.112)       0:00:25.369 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:12 +0000 (0:00:00.664)       0:00:26.034 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory 13.71 M"
}


TASK [test : Record ipv4 route counts at start] ********************************

Wednesday 13 February 2019  17:53:12 +0000 (0:00:00.127)       0:00:26.161 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:13 +0000 (0:00:01.349)       0:00:27.511 **** 

ok: [falco-test-dut01] => {
    "msg": "IPv4 routes at start 6012"
}


TASK [test : Make Sure orchagent CPU utilization is less that 10 before link flap] ***

Wednesday 13 February 2019  17:53:13 +0000 (0:00:00.129)       0:00:27.640 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:14 +0000 (0:00:01.142)       0:00:28.783 **** 

ok: [falco-test-dut01] => {
    "msg": "First Iteration flap all interfaces one by one on DUT"
}


TASK [test : Gathering lab graph facts about the device] ***********************

Wednesday 13 February 2019  17:53:14 +0000 (0:00:00.125)       0:00:28.908 **** 

ok: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:15 +0000 (0:00:00.267)       0:00:29.176 **** 

ok: [falco-test-dut01] => {
    "msg": {
        "Ethernet0": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet1/1", 
            "speed": "40000"
        }, 
        "Ethernet100": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet26/1", 
            "speed": "40000"
        }, 
        "Ethernet104": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet27/1", 
            "speed": "40000"
        }, 
        "Ethernet108": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet28/1", 
            "speed": "40000"
        }, 
        "Ethernet112": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet29/1", 
            "speed": "40000"
        }, 
        "Ethernet116": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet30/1", 
            "speed": "40000"
        }, 
        "Ethernet12": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet4/1", 
            "speed": "40000"
        }, 
        "Ethernet120": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet31/1", 
            "speed": "40000"
        }, 
        "Ethernet124": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet32/1", 
            "speed": "40000"
        }, 
        "Ethernet16": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet5/1", 
            "speed": "40000"
        }, 
        "Ethernet20": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet6/1", 
            "speed": "40000"
        }, 
        "Ethernet24": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet7/1", 
            "speed": "40000"
        }, 
        "Ethernet28": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet8/1", 
            "speed": "40000"
        }, 
        "Ethernet32": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet9/1", 
            "speed": "40000"
        }, 
        "Ethernet36": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet10/1", 
            "speed": "40000"
        }, 
        "Ethernet4": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet2/1", 
            "speed": "40000"
        }, 
        "Ethernet40": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet11/1", 
            "speed": "40000"
        }, 
        "Ethernet44": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet12/1", 
            "speed": "40000"
        }, 
        "Ethernet48": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet13/1", 
            "speed": "40000"
        }, 
        "Ethernet52": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet14/1", 
            "speed": "40000"
        }, 
        "Ethernet56": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet15/1", 
            "speed": "40000"
        }, 
        "Ethernet60": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet16/1", 
            "speed": "40000"
        }, 
        "Ethernet64": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet17/1", 
            "speed": "40000"
        }, 
        "Ethernet68": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet18/1", 
            "speed": "40000"
        }, 
        "Ethernet72": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet19/1", 
            "speed": "40000"
        }, 
        "Ethernet76": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet20/1", 
            "speed": "40000"
        }, 
        "Ethernet8": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet3/1", 
            "speed": "40000"
        }, 
        "Ethernet80": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet21/1", 
            "speed": "40000"
        }, 
        "Ethernet84": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet22/1", 
            "speed": "40000"
        }, 
        "Ethernet88": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet23/1", 
            "speed": "40000"
        }, 
        "Ethernet92": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet24/1", 
            "speed": "40000"
        }, 
        "Ethernet96": {
            "peerdevice": "falco-test-fout01", 
            "peerport": "Ethernet25/1", 
            "speed": "40000"
        }
    }
}


TASK [test : include continuous_link_flap_helper.yml] **************************

Wednesday 13 February 2019  17:53:15 +0000 (0:00:00.135)       0:00:29.311 **** 

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:37 +0000 (0:00:22.587)       0:00:51.899 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet8"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:53:38 +0000 (0:00:00.175)       0:00:52.074 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:53:38 +0000 (0:00:00.556)       0:00:52.631 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:53:39 +0000 (0:00:01.168)       0:00:53.800 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:40 +0000 (0:00:00.981)       0:00:54.782 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       956M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:53:41 +0000 (0:00:00.168)       0:00:54.950 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:42 +0000 (0:00:01.154)       0:00:56.105 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:53:42 +0000 (0:00:00.157)       0:00:56.263 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:43 +0000 (0:00:00.682)       0:00:56.946 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.71M"
}


TASK [test : Bring up interface Ethernet8] *************************************

Wednesday 13 February 2019  17:53:43 +0000 (0:00:00.168)       0:00:57.114 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:43 +0000 (0:00:00.523)       0:00:57.638 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet0"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:53:43 +0000 (0:00:00.167)       0:00:57.806 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:53:44 +0000 (0:00:00.523)       0:00:58.329 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:53:45 +0000 (0:00:01.174)       0:00:59.504 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:46 +0000 (0:00:01.018)       0:01:00.522 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       956M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:53:46 +0000 (0:00:00.169)       0:01:00.692 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:47 +0000 (0:00:01.170)       0:01:01.862 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:53:48 +0000 (0:00:00.151)       0:01:02.013 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:48 +0000 (0:00:00.715)       0:01:02.729 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.71M"
}


TASK [test : Bring up interface Ethernet0] *************************************

Wednesday 13 February 2019  17:53:48 +0000 (0:00:00.156)       0:01:02.886 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:49 +0000 (0:00:00.492)       0:01:03.378 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet4"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:53:49 +0000 (0:00:00.152)       0:01:03.531 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:53:50 +0000 (0:00:00.500)       0:01:04.031 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:53:51 +0000 (0:00:01.148)       0:01:05.180 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:52 +0000 (0:00:01.028)       0:01:06.209 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       956M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:53:52 +0000 (0:00:00.154)       0:01:06.364 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:53 +0000 (0:00:01.204)       0:01:07.568 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:53:53 +0000 (0:00:00.154)       0:01:07.723 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:54 +0000 (0:00:00.702)       0:01:08.426 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.71M"
}


TASK [test : Bring up interface Ethernet4] *************************************

Wednesday 13 February 2019  17:53:54 +0000 (0:00:00.151)       0:01:08.577 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:55 +0000 (0:00:00.487)       0:01:09.065 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet108"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:53:55 +0000 (0:00:00.147)       0:01:09.212 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:53:55 +0000 (0:00:00.537)       0:01:09.750 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:53:56 +0000 (0:00:01.178)       0:01:10.929 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:57 +0000 (0:00:00.968)       0:01:11.897 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       957M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:53:58 +0000 (0:00:00.153)       0:01:12.051 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:53:59 +0000 (0:00:01.202)       0:01:13.253 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:53:59 +0000 (0:00:00.169)       0:01:13.423 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:00 +0000 (0:00:00.751)       0:01:14.175 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.78M"
}


TASK [test : Bring up interface Ethernet108] ***********************************

Wednesday 13 February 2019  17:54:00 +0000 (0:00:00.147)       0:01:14.322 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:00 +0000 (0:00:00.509)       0:01:14.831 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet100"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:01 +0000 (0:00:00.172)       0:01:15.004 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:54:01 +0000 (0:00:00.529)       0:01:15.533 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:54:02 +0000 (0:00:01.174)       0:01:16.708 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:03 +0000 (0:00:01.014)       0:01:17.722 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       957M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:54:03 +0000 (0:00:00.178)       0:01:17.900 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:05 +0000 (0:00:01.194)       0:01:19.095 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:54:05 +0000 (0:00:00.172)       0:01:19.268 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:06 +0000 (0:00:00.686)       0:01:19.955 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.71M"
}


TASK [test : Bring up interface Ethernet100] ***********************************

Wednesday 13 February 2019  17:54:06 +0000 (0:00:00.165)       0:01:20.121 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:06 +0000 (0:00:00.537)       0:01:20.658 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet104"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:06 +0000 (0:00:00.165)       0:01:20.824 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:54:07 +0000 (0:00:00.532)       0:01:21.356 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:54:08 +0000 (0:00:01.176)       0:01:22.533 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:09 +0000 (0:00:01.021)       0:01:23.554 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       959M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:54:09 +0000 (0:00:00.160)       0:01:23.715 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:10 +0000 (0:00:01.177)       0:01:24.893 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:54:11 +0000 (0:00:00.163)       0:01:25.056 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:11 +0000 (0:00:00.661)       0:01:25.719 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.67M"
}


TASK [test : Bring up interface Ethernet104] ***********************************

Wednesday 13 February 2019  17:54:11 +0000 (0:00:00.167)       0:01:25.886 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:12 +0000 (0:00:00.503)       0:01:26.390 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet68"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:12 +0000 (0:00:00.174)       0:01:26.565 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:54:13 +0000 (0:00:00.560)       0:01:27.125 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:54:14 +0000 (0:00:01.156)       0:01:28.281 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:15 +0000 (0:00:00.977)       0:01:29.259 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       959M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:54:15 +0000 (0:00:00.149)       0:01:29.408 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:16 +0000 (0:00:01.165)       0:01:30.574 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:54:16 +0000 (0:00:00.155)       0:01:30.729 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:17 +0000 (0:00:00.701)       0:01:31.430 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.74M"
}


TASK [test : Bring up interface Ethernet68] ************************************

Wednesday 13 February 2019  17:54:17 +0000 (0:00:00.172)       0:01:31.602 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:18 +0000 (0:00:00.539)       0:01:32.141 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet96"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:18 +0000 (0:00:00.172)       0:01:32.314 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:54:18 +0000 (0:00:00.549)       0:01:32.863 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:54:20 +0000 (0:00:01.187)       0:01:34.051 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:21 +0000 (0:00:01.021)       0:01:35.073 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       960M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:54:21 +0000 (0:00:00.167)       0:01:35.241 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:22 +0000 (0:00:01.173)       0:01:36.414 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:54:22 +0000 (0:00:00.151)       0:01:36.566 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:23 +0000 (0:00:00.695)       0:01:37.262 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.81M"
}


TASK [test : Bring up interface Ethernet96] ************************************

Wednesday 13 February 2019  17:54:23 +0000 (0:00:00.163)       0:01:37.425 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:23 +0000 (0:00:00.490)       0:01:37.915 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet124"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:24 +0000 (0:00:00.171)       0:01:38.087 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:54:24 +0000 (0:00:00.553)       0:01:38.641 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:54:25 +0000 (0:00:01.170)       0:01:39.811 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:26 +0000 (0:00:01.040)       0:01:40.852 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       960M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:54:27 +0000 (0:00:00.157)       0:01:41.009 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:28 +0000 (0:00:01.207)       0:01:42.217 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 66"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:54:28 +0000 (0:00:00.152)       0:01:42.369 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:29 +0000 (0:00:00.771)       0:01:43.141 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.50M"
}


TASK [test : Bring up interface Ethernet124] ***********************************

Wednesday 13 February 2019  17:54:29 +0000 (0:00:00.156)       0:01:43.297 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:29 +0000 (0:00:00.514)       0:01:43.812 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet92"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:30 +0000 (0:00:00.173)       0:01:43.985 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:54:30 +0000 (0:00:00.567)       0:01:44.553 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:54:31 +0000 (0:00:01.164)       0:01:45.717 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:32 +0000 (0:00:01.074)       0:01:46.792 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       968M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:54:32 +0000 (0:00:00.145)       0:01:46.937 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:34 +0000 (0:00:01.253)       0:01:48.191 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 66"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:54:34 +0000 (0:00:00.154)       0:01:48.345 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:35 +0000 (0:00:00.741)       0:01:49.087 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.47M"
}


TASK [test : Bring up interface Ethernet92] ************************************

Wednesday 13 February 2019  17:54:35 +0000 (0:00:00.154)       0:01:49.242 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:35 +0000 (0:00:00.499)       0:01:49.741 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet120"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:35 +0000 (0:00:00.164)       0:01:49.905 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:54:36 +0000 (0:00:00.536)       0:01:50.442 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:54:37 +0000 (0:00:01.159)       0:01:51.602 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:38 +0000 (0:00:01.092)       0:01:52.694 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       964M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:54:38 +0000 (0:00:00.170)       0:01:52.865 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:40 +0000 (0:00:01.246)       0:01:54.111 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 97"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:54:40 +0000 (0:00:00.162)       0:01:54.273 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:41 +0000 (0:00:00.741)       0:01:55.015 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:12.94M"
}


TASK [test : Bring up interface Ethernet120] ***********************************

Wednesday 13 February 2019  17:54:41 +0000 (0:00:00.170)       0:01:55.186 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:41 +0000 (0:00:00.551)       0:01:55.737 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet52"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:41 +0000 (0:00:00.162)       0:01:55.900 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:54:42 +0000 (0:00:00.537)       0:01:56.437 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:54:43 +0000 (0:00:01.174)       0:01:57.612 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:44 +0000 (0:00:01.077)       0:01:58.689 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       968M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:54:44 +0000 (0:00:00.167)       0:01:58.857 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:46 +0000 (0:00:01.262)       0:02:00.119 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 66"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:54:46 +0000 (0:00:00.168)       0:02:00.288 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:47 +0000 (0:00:00.764)       0:02:01.052 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.04M"
}


TASK [test : Bring up interface Ethernet52] ************************************

Wednesday 13 February 2019  17:54:47 +0000 (0:00:00.154)       0:02:01.207 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:47 +0000 (0:00:00.496)       0:02:01.704 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet56"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:47 +0000 (0:00:00.162)       0:02:01.866 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:54:48 +0000 (0:00:00.523)       0:02:02.390 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:54:49 +0000 (0:00:01.166)       0:02:03.556 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:50 +0000 (0:00:01.024)       0:02:04.580 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       960M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:54:50 +0000 (0:00:00.176)       0:02:04.757 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:52 +0000 (0:00:01.216)       0:02:05.974 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:54:52 +0000 (0:00:00.158)       0:02:06.132 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:52 +0000 (0:00:00.729)       0:02:06.862 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:12.90M"
}


TASK [test : Bring up interface Ethernet56] ************************************

Wednesday 13 February 2019  17:54:53 +0000 (0:00:00.176)       0:02:07.038 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:53 +0000 (0:00:00.485)       0:02:07.524 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet76"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:53 +0000 (0:00:00.151)       0:02:07.675 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:54:54 +0000 (0:00:00.554)       0:02:08.229 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:54:55 +0000 (0:00:01.169)       0:02:09.399 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:56 +0000 (0:00:00.997)       0:02:10.396 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       962M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:54:56 +0000 (0:00:00.160)       0:02:10.557 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:57 +0000 (0:00:01.179)       0:02:11.737 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:54:57 +0000 (0:00:00.167)       0:02:11.904 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:58 +0000 (0:00:00.686)       0:02:12.591 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:12.86M"
}


TASK [test : Bring up interface Ethernet76] ************************************

Wednesday 13 February 2019  17:54:58 +0000 (0:00:00.169)       0:02:12.761 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:54:59 +0000 (0:00:00.552)       0:02:13.314 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet72"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:54:59 +0000 (0:00:00.169)       0:02:13.484 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:00 +0000 (0:00:00.538)       0:02:14.022 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:55:01 +0000 (0:00:01.156)       0:02:15.179 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:02 +0000 (0:00:01.034)       0:02:16.213 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       962M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:55:02 +0000 (0:00:00.175)       0:02:16.388 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:03 +0000 (0:00:01.216)       0:02:17.605 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:55:03 +0000 (0:00:00.148)       0:02:17.753 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:04 +0000 (0:00:00.693)       0:02:18.447 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:12.90M"
}


TASK [test : Bring up interface Ethernet72] ************************************

Wednesday 13 February 2019  17:55:04 +0000 (0:00:00.185)       0:02:18.633 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:05 +0000 (0:00:00.542)       0:02:19.175 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet64"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:55:05 +0000 (0:00:00.167)       0:02:19.342 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:05 +0000 (0:00:00.557)       0:02:19.900 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:55:07 +0000 (0:00:01.180)       0:02:21.080 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:08 +0000 (0:00:01.027)       0:02:22.107 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       962M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:55:08 +0000 (0:00:00.149)       0:02:22.257 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:09 +0000 (0:00:01.188)       0:02:23.446 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:55:09 +0000 (0:00:00.160)       0:02:23.606 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:10 +0000 (0:00:00.706)       0:02:24.312 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:12.86M"
}


TASK [test : Bring up interface Ethernet64] ************************************

Wednesday 13 February 2019  17:55:10 +0000 (0:00:00.178)       0:02:24.491 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:11 +0000 (0:00:00.544)       0:02:25.036 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet32"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:55:11 +0000 (0:00:00.151)       0:02:25.187 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:11 +0000 (0:00:00.552)       0:02:25.739 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:55:12 +0000 (0:00:01.177)       0:02:26.917 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:14 +0000 (0:00:01.067)       0:02:27.984 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       965M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:55:14 +0000 (0:00:00.174)       0:02:28.159 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:15 +0000 (0:00:01.233)       0:02:29.393 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 60"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:55:15 +0000 (0:00:00.151)       0:02:29.544 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:16 +0000 (0:00:00.723)       0:02:30.268 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.19M"
}


TASK [test : Bring up interface Ethernet32] ************************************

Wednesday 13 February 2019  17:55:16 +0000 (0:00:00.167)       0:02:30.435 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:17 +0000 (0:00:00.521)       0:02:30.957 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet16"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:55:17 +0000 (0:00:00.173)       0:02:31.130 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:17 +0000 (0:00:00.520)       0:02:31.651 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:55:18 +0000 (0:00:01.170)       0:02:32.821 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:19 +0000 (0:00:01.027)       0:02:33.848 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       966M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:55:20 +0000 (0:00:00.174)       0:02:34.023 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:21 +0000 (0:00:01.181)       0:02:35.205 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:55:21 +0000 (0:00:00.162)       0:02:35.367 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:22 +0000 (0:00:00.689)       0:02:36.057 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.05M"
}


TASK [test : Bring up interface Ethernet16] ************************************

Wednesday 13 February 2019  17:55:22 +0000 (0:00:00.173)       0:02:36.231 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:22 +0000 (0:00:00.534)       0:02:36.766 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet36"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:55:22 +0000 (0:00:00.155)       0:02:36.921 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:23 +0000 (0:00:00.517)       0:02:37.439 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:55:24 +0000 (0:00:01.177)       0:02:38.617 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:25 +0000 (0:00:01.027)       0:02:39.644 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       967M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:55:25 +0000 (0:00:00.160)       0:02:39.805 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:27 +0000 (0:00:01.190)       0:02:40.995 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:55:27 +0000 (0:00:00.163)       0:02:41.159 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:27 +0000 (0:00:00.668)       0:02:41.827 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.08M"
}


TASK [test : Bring up interface Ethernet36] ************************************

Wednesday 13 February 2019  17:55:28 +0000 (0:00:00.159)       0:02:41.987 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:28 +0000 (0:00:00.546)       0:02:42.534 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet12"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:55:28 +0000 (0:00:00.169)       0:02:42.703 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:29 +0000 (0:00:00.551)       0:02:43.255 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:55:30 +0000 (0:00:01.167)       0:02:44.422 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:31 +0000 (0:00:01.065)       0:02:45.487 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       294M       1.1G\n-/+ buffers/cache:       967M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:55:31 +0000 (0:00:00.181)       0:02:45.668 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:32 +0000 (0:00:01.267)       0:02:46.936 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 54"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:55:33 +0000 (0:00:00.158)       0:02:47.094 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:33 +0000 (0:00:00.748)       0:02:47.842 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.18M"
}


TASK [test : Bring up interface Ethernet12] ************************************

Wednesday 13 February 2019  17:55:34 +0000 (0:00:00.175)       0:02:48.018 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:34 +0000 (0:00:00.535)       0:02:48.553 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet88"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:55:34 +0000 (0:00:00.175)       0:02:48.729 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:35 +0000 (0:00:00.555)       0:02:49.284 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:55:36 +0000 (0:00:01.175)       0:02:50.460 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:37 +0000 (0:00:01.022)       0:02:51.482 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       966M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:55:37 +0000 (0:00:00.165)       0:02:51.648 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:38 +0000 (0:00:01.225)       0:02:52.873 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 6"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:55:39 +0000 (0:00:00.155)       0:02:53.029 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:39 +0000 (0:00:00.702)       0:02:53.732 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.25M"
}


TASK [test : Bring up interface Ethernet88] ************************************

Wednesday 13 February 2019  17:55:39 +0000 (0:00:00.156)       0:02:53.889 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:40 +0000 (0:00:00.526)       0:02:54.415 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet116"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:55:40 +0000 (0:00:00.164)       0:02:54.579 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:41 +0000 (0:00:00.541)       0:02:55.121 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:55:42 +0000 (0:00:01.168)       0:02:56.290 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:43 +0000 (0:00:01.109)       0:02:57.399 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       968M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:55:43 +0000 (0:00:00.174)       0:02:57.574 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:44 +0000 (0:00:01.245)       0:02:58.819 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 60"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:55:45 +0000 (0:00:00.149)       0:02:58.969 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:45 +0000 (0:00:00.756)       0:02:59.726 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.35M"
}


TASK [test : Bring up interface Ethernet116] ***********************************

Wednesday 13 February 2019  17:55:45 +0000 (0:00:00.159)       0:02:59.885 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:46 +0000 (0:00:00.546)       0:03:00.431 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet80"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:55:46 +0000 (0:00:00.168)       0:03:00.600 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:47 +0000 (0:00:00.565)       0:03:01.166 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:55:48 +0000 (0:00:01.171)       0:03:02.338 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:49 +0000 (0:00:01.060)       0:03:03.398 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       968M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:55:49 +0000 (0:00:00.169)       0:03:03.568 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:50 +0000 (0:00:01.273)       0:03:04.841 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 12"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:55:51 +0000 (0:00:00.160)       0:03:05.002 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:51 +0000 (0:00:00.820)       0:03:05.823 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:14.51M"
}


TASK [test : Bring up interface Ethernet80] ************************************

Wednesday 13 February 2019  17:55:52 +0000 (0:00:00.259)       0:03:06.082 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:52 +0000 (0:00:00.733)       0:03:06.816 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet112"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:55:53 +0000 (0:00:00.151)       0:03:06.968 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:53 +0000 (0:00:00.547)       0:03:07.515 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:55:54 +0000 (0:00:01.170)       0:03:08.686 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:55 +0000 (0:00:01.019)       0:03:09.706 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       967M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:55:55 +0000 (0:00:00.193)       0:03:09.900 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:57 +0000 (0:00:01.240)       0:03:11.140 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 12"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:55:57 +0000 (0:00:00.132)       0:03:11.273 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:58 +0000 (0:00:00.788)       0:03:12.062 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.14M"
}


TASK [test : Bring up interface Ethernet112] ***********************************

Wednesday 13 February 2019  17:55:58 +0000 (0:00:00.155)       0:03:12.217 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:55:58 +0000 (0:00:00.539)       0:03:12.757 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet84"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:55:58 +0000 (0:00:00.152)       0:03:12.909 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:55:59 +0000 (0:00:00.517)       0:03:13.427 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:56:00 +0000 (0:00:01.162)       0:03:14.590 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:01 +0000 (0:00:01.051)       0:03:15.641 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       976M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:56:01 +0000 (0:00:00.175)       0:03:15.817 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:03 +0000 (0:00:01.295)       0:03:17.113 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 36"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:56:03 +0000 (0:00:00.162)       0:03:17.275 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:04 +0000 (0:00:00.760)       0:03:18.036 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.97M"
}


TASK [test : Bring up interface Ethernet84] ************************************

Wednesday 13 February 2019  17:56:04 +0000 (0:00:00.178)       0:03:18.215 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:04 +0000 (0:00:00.571)       0:03:18.786 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet48"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:56:05 +0000 (0:00:00.166)       0:03:18.953 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:56:05 +0000 (0:00:00.591)       0:03:19.544 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:56:06 +0000 (0:00:01.176)       0:03:20.720 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:07 +0000 (0:00:01.053)       0:03:21.774 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       981M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:56:07 +0000 (0:00:00.166)       0:03:21.940 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:09 +0000 (0:00:01.249)       0:03:23.190 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 66"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:56:09 +0000 (0:00:00.163)       0:03:23.354 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:10 +0000 (0:00:00.776)       0:03:24.130 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.45M"
}


TASK [test : Bring up interface Ethernet48] ************************************

Wednesday 13 February 2019  17:56:10 +0000 (0:00:00.163)       0:03:24.294 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:10 +0000 (0:00:00.540)       0:03:24.834 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet44"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:56:11 +0000 (0:00:00.180)       0:03:25.014 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:56:11 +0000 (0:00:00.567)       0:03:25.582 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:56:12 +0000 (0:00:01.171)       0:03:26.754 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:13 +0000 (0:00:01.092)       0:03:27.846 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       979M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:56:14 +0000 (0:00:00.178)       0:03:28.025 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:15 +0000 (0:00:01.249)       0:03:29.274 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 66"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:56:15 +0000 (0:00:00.155)       0:03:29.430 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:16 +0000 (0:00:00.723)       0:03:30.154 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.35M"
}


TASK [test : Bring up interface Ethernet44] ************************************

Wednesday 13 February 2019  17:56:16 +0000 (0:00:00.157)       0:03:30.311 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:16 +0000 (0:00:00.513)       0:03:30.825 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet40"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:56:17 +0000 (0:00:00.182)       0:03:31.007 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:56:17 +0000 (0:00:00.556)       0:03:31.563 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:56:18 +0000 (0:00:01.172)       0:03:32.735 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:19 +0000 (0:00:01.016)       0:03:33.752 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       974M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:56:19 +0000 (0:00:00.170)       0:03:33.923 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:21 +0000 (0:00:01.176)       0:03:35.099 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:56:21 +0000 (0:00:00.167)       0:03:35.267 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:22 +0000 (0:00:00.696)       0:03:35.963 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.11M"
}


TASK [test : Bring up interface Ethernet40] ************************************

Wednesday 13 February 2019  17:56:22 +0000 (0:00:00.160)       0:03:36.123 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:22 +0000 (0:00:00.528)       0:03:36.652 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet28"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:56:22 +0000 (0:00:00.173)       0:03:36.825 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:56:23 +0000 (0:00:00.556)       0:03:37.381 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:56:24 +0000 (0:00:01.177)       0:03:38.559 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:25 +0000 (0:00:01.030)       0:03:39.589 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       975M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:56:25 +0000 (0:00:00.162)       0:03:39.752 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:26 +0000 (0:00:01.193)       0:03:40.945 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:56:27 +0000 (0:00:00.159)       0:03:41.105 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:27 +0000 (0:00:00.754)       0:03:41.860 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.14M"
}


TASK [test : Bring up interface Ethernet28] ************************************

Wednesday 13 February 2019  17:56:28 +0000 (0:00:00.174)       0:03:42.035 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:28 +0000 (0:00:00.538)       0:03:42.573 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet60"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:56:28 +0000 (0:00:00.175)       0:03:42.749 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:56:29 +0000 (0:00:00.546)       0:03:43.296 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:56:30 +0000 (0:00:01.174)       0:03:44.470 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:31 +0000 (0:00:01.031)       0:03:45.501 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       976M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:56:31 +0000 (0:00:00.158)       0:03:45.660 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:32 +0000 (0:00:01.210)       0:03:46.870 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 6"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:56:33 +0000 (0:00:00.171)       0:03:47.042 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:33 +0000 (0:00:00.722)       0:03:47.765 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.14M"
}


TASK [test : Bring up interface Ethernet60] ************************************

Wednesday 13 February 2019  17:56:33 +0000 (0:00:00.176)       0:03:47.942 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:34 +0000 (0:00:00.548)       0:03:48.490 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet20"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:56:34 +0000 (0:00:00.165)       0:03:48.656 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:56:35 +0000 (0:00:00.507)       0:03:49.163 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:56:36 +0000 (0:00:01.179)       0:03:50.343 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:37 +0000 (0:00:01.042)       0:03:51.386 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       977M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:56:37 +0000 (0:00:00.164)       0:03:51.550 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:38 +0000 (0:00:01.186)       0:03:52.736 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:56:38 +0000 (0:00:00.154)       0:03:52.891 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:39 +0000 (0:00:00.748)       0:03:53.640 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.18M"
}


TASK [test : Bring up interface Ethernet20] ************************************

Wednesday 13 February 2019  17:56:39 +0000 (0:00:00.156)       0:03:53.796 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:40 +0000 (0:00:00.514)       0:03:54.311 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet24"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:56:40 +0000 (0:00:00.184)       0:03:54.495 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:56:41 +0000 (0:00:00.549)       0:03:55.045 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:56:42 +0000 (0:00:01.154)       0:03:56.199 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:43 +0000 (0:00:01.005)       0:03:57.205 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       977M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:56:43 +0000 (0:00:00.142)       0:03:57.348 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:44 +0000 (0:00:01.186)       0:03:58.535 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:56:44 +0000 (0:00:00.155)       0:03:58.691 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:45 +0000 (0:00:00.715)       0:03:59.406 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.18M"
}


TASK [test : Bring up interface Ethernet24] ************************************

Wednesday 13 February 2019  17:56:45 +0000 (0:00:00.157)       0:03:59.564 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:56:46 +0000 (0:00:00.502)       0:04:00.066 **** 

ok: [falco-test-dut01] => {
    "msg": "Second Iteration flap all interfaces one by one on DUT"
}


TASK [test : Gathering lab graph facts about the device] ***********************

Wednesday 13 February 2019  17:56:46 +0000 (0:00:00.097)       0:04:00.164 **** 

ok: [falco-test-dut01]


TASK [test : include continuous_link_flap_helper.yml] **************************

Wednesday 13 February 2019  17:56:46 +0000 (0:00:00.251)       0:04:00.416 **** 

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/continuous_link_flap/continuous_link_flap_helper.yml for falco-test-dut01


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:37 +0000 (0:01:51.211)       0:05:51.627 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet8"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:58:37 +0000 (0:00:00.181)       0:05:51.808 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:58:38 +0000 (0:00:00.668)       0:05:52.476 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:58:39 +0000 (0:00:01.185)       0:05:53.662 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:40 +0000 (0:00:01.179)       0:05:54.841 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       981M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:58:41 +0000 (0:00:00.174)       0:05:55.015 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:42 +0000 (0:00:01.326)       0:05:56.342 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:58:42 +0000 (0:00:00.166)       0:05:56.509 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:43 +0000 (0:00:00.845)       0:05:57.354 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.70M"
}


TASK [test : Bring up interface Ethernet8] *************************************

Wednesday 13 February 2019  17:58:43 +0000 (0:00:00.184)       0:05:57.538 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:44 +0000 (0:00:00.661)       0:05:58.200 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet0"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:58:44 +0000 (0:00:00.178)       0:05:58.378 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:58:45 +0000 (0:00:00.686)       0:05:59.064 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:58:46 +0000 (0:00:01.180)       0:06:00.244 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:47 +0000 (0:00:01.152)       0:06:01.397 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.3G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       980M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:58:47 +0000 (0:00:00.176)       0:06:01.574 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:48 +0000 (0:00:01.332)       0:06:02.906 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:58:49 +0000 (0:00:00.182)       0:06:03.089 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:50 +0000 (0:00:00.907)       0:06:03.996 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.70M"
}


TASK [test : Bring up interface Ethernet0] *************************************

Wednesday 13 February 2019  17:58:50 +0000 (0:00:00.185)       0:06:04.182 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:50 +0000 (0:00:00.667)       0:06:04.849 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet4"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:58:51 +0000 (0:00:00.166)       0:06:05.015 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:58:51 +0000 (0:00:00.686)       0:06:05.702 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:58:52 +0000 (0:00:01.167)       0:06:06.869 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:54 +0000 (0:00:01.147)       0:06:08.016 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       981M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:58:54 +0000 (0:00:00.169)       0:06:08.185 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:55 +0000 (0:00:01.325)       0:06:09.511 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:58:55 +0000 (0:00:00.174)       0:06:09.685 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:56 +0000 (0:00:00.876)       0:06:10.562 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.66M"
}


TASK [test : Bring up interface Ethernet4] *************************************

Wednesday 13 February 2019  17:58:56 +0000 (0:00:00.179)       0:06:10.742 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:58:57 +0000 (0:00:00.671)       0:06:11.413 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet108"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:58:57 +0000 (0:00:00.176)       0:06:11.590 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:58:58 +0000 (0:00:00.677)       0:06:12.267 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:58:59 +0000 (0:00:01.178)       0:06:13.446 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:00 +0000 (0:00:01.140)       0:06:14.586 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       982M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:59:00 +0000 (0:00:00.176)       0:06:14.763 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:02 +0000 (0:00:01.348)       0:06:16.112 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:59:02 +0000 (0:00:00.188)       0:06:16.300 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:03 +0000 (0:00:00.855)       0:06:17.155 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.66M"
}


TASK [test : Bring up interface Ethernet108] ***********************************

Wednesday 13 February 2019  17:59:03 +0000 (0:00:00.179)       0:06:17.335 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:04 +0000 (0:00:00.675)       0:06:18.011 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet100"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:59:04 +0000 (0:00:00.183)       0:06:18.194 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:59:04 +0000 (0:00:00.673)       0:06:18.868 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:59:06 +0000 (0:00:01.184)       0:06:20.052 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:07 +0000 (0:00:01.155)       0:06:21.208 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       982M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:59:07 +0000 (0:00:00.182)       0:06:21.390 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:08 +0000 (0:00:01.346)       0:06:22.736 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:59:08 +0000 (0:00:00.177)       0:06:22.914 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:09 +0000 (0:00:00.825)       0:06:23.739 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.70M"
}


TASK [test : Bring up interface Ethernet100] ***********************************

Wednesday 13 February 2019  17:59:09 +0000 (0:00:00.170)       0:06:23.910 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:10 +0000 (0:00:00.672)       0:06:24.582 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet104"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:59:10 +0000 (0:00:00.182)       0:06:24.765 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:59:11 +0000 (0:00:00.687)       0:06:25.452 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:59:12 +0000 (0:00:01.179)       0:06:26.632 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:13 +0000 (0:00:01.138)       0:06:27.770 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       983M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:59:14 +0000 (0:00:00.179)       0:06:27.950 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:15 +0000 (0:00:01.338)       0:06:29.289 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:59:15 +0000 (0:00:00.179)       0:06:29.468 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:16 +0000 (0:00:00.875)       0:06:30.344 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.70M"
}


TASK [test : Bring up interface Ethernet104] ***********************************

Wednesday 13 February 2019  17:59:16 +0000 (0:00:00.178)       0:06:30.522 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:17 +0000 (0:00:00.656)       0:06:31.179 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet68"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:59:17 +0000 (0:00:00.173)       0:06:31.352 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:59:18 +0000 (0:00:00.678)       0:06:32.031 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:59:19 +0000 (0:00:01.181)       0:06:33.212 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:20 +0000 (0:00:01.132)       0:06:34.344 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       984M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:59:20 +0000 (0:00:00.178)       0:06:34.523 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:21 +0000 (0:00:01.332)       0:06:35.855 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:59:22 +0000 (0:00:00.176)       0:06:36.032 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:22 +0000 (0:00:00.896)       0:06:36.929 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.70M"
}


TASK [test : Bring up interface Ethernet68] ************************************

Wednesday 13 February 2019  17:59:23 +0000 (0:00:00.182)       0:06:37.111 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:23 +0000 (0:00:00.663)       0:06:37.775 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet96"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:59:23 +0000 (0:00:00.160)       0:06:37.936 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:59:24 +0000 (0:00:00.685)       0:06:38.622 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:59:25 +0000 (0:00:01.181)       0:06:39.803 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:26 +0000 (0:00:01.125)       0:06:40.929 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       984M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:59:27 +0000 (0:00:00.149)       0:06:41.078 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:28 +0000 (0:00:01.326)       0:06:42.405 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:59:28 +0000 (0:00:00.181)       0:06:42.587 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:29 +0000 (0:00:00.822)       0:06:43.409 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.70M"
}


TASK [test : Bring up interface Ethernet96] ************************************

Wednesday 13 February 2019  17:59:29 +0000 (0:00:00.169)       0:06:43.578 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:30 +0000 (0:00:00.668)       0:06:44.246 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet124"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:59:30 +0000 (0:00:00.168)       0:06:44.415 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:59:31 +0000 (0:00:00.674)       0:06:45.089 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:59:32 +0000 (0:00:01.182)       0:06:46.272 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:33 +0000 (0:00:01.231)       0:06:47.504 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       986M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:59:33 +0000 (0:00:00.163)       0:06:47.667 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:35 +0000 (0:00:01.399)       0:06:49.066 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 60"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:59:35 +0000 (0:00:00.178)       0:06:49.245 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:36 +0000 (0:00:00.880)       0:06:50.125 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.81M"
}


TASK [test : Bring up interface Ethernet124] ***********************************

Wednesday 13 February 2019  17:59:36 +0000 (0:00:00.192)       0:06:50.318 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:37 +0000 (0:00:00.667)       0:06:50.985 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet92"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:59:37 +0000 (0:00:00.176)       0:06:51.161 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:59:37 +0000 (0:00:00.694)       0:06:51.856 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:59:39 +0000 (0:00:01.175)       0:06:53.032 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:40 +0000 (0:00:01.186)       0:06:54.218 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       988M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:59:40 +0000 (0:00:00.169)       0:06:54.388 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:41 +0000 (0:00:01.354)       0:06:55.742 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:59:41 +0000 (0:00:00.173)       0:06:55.916 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:42 +0000 (0:00:00.848)       0:06:56.764 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.14M"
}


TASK [test : Bring up interface Ethernet92] ************************************

Wednesday 13 February 2019  17:59:43 +0000 (0:00:00.185)       0:06:56.950 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:43 +0000 (0:00:00.667)       0:06:57.617 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet120"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:59:43 +0000 (0:00:00.180)       0:06:57.798 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:59:44 +0000 (0:00:00.673)       0:06:58.471 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:59:45 +0000 (0:00:01.175)       0:06:59.646 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:46 +0000 (0:00:01.181)       0:07:00.828 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       986M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:59:47 +0000 (0:00:00.182)       0:07:01.010 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:48 +0000 (0:00:01.397)       0:07:02.407 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 54"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:59:48 +0000 (0:00:00.181)       0:07:02.589 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:49 +0000 (0:00:00.919)       0:07:03.509 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.20M"
}


TASK [test : Bring up interface Ethernet120] ***********************************

Wednesday 13 February 2019  17:59:49 +0000 (0:00:00.187)       0:07:03.697 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:50 +0000 (0:00:00.667)       0:07:04.364 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet52"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:59:50 +0000 (0:00:00.164)       0:07:04.529 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:59:51 +0000 (0:00:00.691)       0:07:05.221 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:59:52 +0000 (0:00:01.173)       0:07:06.394 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:53 +0000 (0:00:01.213)       0:07:07.608 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       986M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  17:59:53 +0000 (0:00:00.180)       0:07:07.789 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:55 +0000 (0:00:01.346)       0:07:09.135 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  17:59:55 +0000 (0:00:00.183)       0:07:09.319 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:56 +0000 (0:00:00.851)       0:07:10.170 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:12.83M"
}


TASK [test : Bring up interface Ethernet52] ************************************

Wednesday 13 February 2019  17:59:56 +0000 (0:00:00.182)       0:07:10.352 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  17:59:57 +0000 (0:00:00.662)       0:07:11.015 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet56"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  17:59:57 +0000 (0:00:00.188)       0:07:11.204 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  17:59:57 +0000 (0:00:00.662)       0:07:11.866 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  17:59:59 +0000 (0:00:01.167)       0:07:13.033 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:00 +0000 (0:00:01.146)       0:07:14.180 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       985M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:00:00 +0000 (0:00:00.180)       0:07:14.360 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:01 +0000 (0:00:01.340)       0:07:15.700 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:00:01 +0000 (0:00:00.178)       0:07:15.879 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:02 +0000 (0:00:00.867)       0:07:16.746 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:12.90M"
}


TASK [test : Bring up interface Ethernet56] ************************************

Wednesday 13 February 2019  18:00:02 +0000 (0:00:00.178)       0:07:16.925 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:03 +0000 (0:00:00.671)       0:07:17.596 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet76"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:00:03 +0000 (0:00:00.178)       0:07:17.774 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:00:04 +0000 (0:00:00.689)       0:07:18.464 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:00:05 +0000 (0:00:01.182)       0:07:19.646 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:06 +0000 (0:00:01.153)       0:07:20.800 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       295M       1.1G\n-/+ buffers/cache:       986M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:00:07 +0000 (0:00:00.177)       0:07:20.977 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:08 +0000 (0:00:01.730)       0:07:22.707 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:00:08 +0000 (0:00:00.180)       0:07:22.888 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:09 +0000 (0:00:00.889)       0:07:23.778 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:12.90M"
}


TASK [test : Bring up interface Ethernet76] ************************************

Wednesday 13 February 2019  18:00:10 +0000 (0:00:00.182)       0:07:23.960 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:10 +0000 (0:00:00.657)       0:07:24.618 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet72"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:00:10 +0000 (0:00:00.178)       0:07:24.797 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:00:11 +0000 (0:00:00.670)       0:07:25.468 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:00:12 +0000 (0:00:01.181)       0:07:26.649 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:13 +0000 (0:00:01.192)       0:07:27.841 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       988M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:00:14 +0000 (0:00:00.176)       0:07:28.017 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:15 +0000 (0:00:01.384)       0:07:29.402 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 54"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:00:15 +0000 (0:00:00.170)       0:07:29.573 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:16 +0000 (0:00:00.950)       0:07:30.523 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.24M"
}


TASK [test : Bring up interface Ethernet72] ************************************

Wednesday 13 February 2019  18:00:16 +0000 (0:00:00.181)       0:07:30.705 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:17 +0000 (0:00:00.687)       0:07:31.392 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet64"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:00:17 +0000 (0:00:00.169)       0:07:31.561 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:00:18 +0000 (0:00:00.686)       0:07:32.248 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:00:19 +0000 (0:00:01.187)       0:07:33.436 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:20 +0000 (0:00:01.196)       0:07:34.633 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       990M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:00:20 +0000 (0:00:00.176)       0:07:34.809 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:22 +0000 (0:00:01.374)       0:07:36.183 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 60"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:00:22 +0000 (0:00:00.180)       0:07:36.364 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:23 +0000 (0:00:00.914)       0:07:37.278 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.28M"
}


TASK [test : Bring up interface Ethernet64] ************************************

Wednesday 13 February 2019  18:00:23 +0000 (0:00:00.186)       0:07:37.465 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:24 +0000 (0:00:00.674)       0:07:38.139 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet32"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:00:24 +0000 (0:00:00.185)       0:07:38.325 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:00:25 +0000 (0:00:00.692)       0:07:39.017 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:00:26 +0000 (0:00:01.181)       0:07:40.198 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:27 +0000 (0:00:01.151)       0:07:41.350 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       989M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:00:27 +0000 (0:00:00.172)       0:07:41.522 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:28 +0000 (0:00:01.329)       0:07:42.852 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:00:29 +0000 (0:00:00.174)       0:07:43.026 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:29 +0000 (0:00:00.854)       0:07:43.881 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.36M"
}


TASK [test : Bring up interface Ethernet32] ************************************

Wednesday 13 February 2019  18:00:30 +0000 (0:00:00.182)       0:07:44.063 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:30 +0000 (0:00:00.671)       0:07:44.735 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet16"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:00:30 +0000 (0:00:00.179)       0:07:44.914 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:00:31 +0000 (0:00:00.680)       0:07:45.595 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:00:32 +0000 (0:00:01.177)       0:07:46.772 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:33 +0000 (0:00:01.159)       0:07:47.931 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       990M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:00:34 +0000 (0:00:00.172)       0:07:48.104 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:35 +0000 (0:00:01.334)       0:07:49.439 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:00:35 +0000 (0:00:00.179)       0:07:49.618 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:36 +0000 (0:00:00.855)       0:07:50.474 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.18M"
}


TASK [test : Bring up interface Ethernet16] ************************************

Wednesday 13 February 2019  18:00:36 +0000 (0:00:00.154)       0:07:50.629 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:37 +0000 (0:00:00.675)       0:07:51.304 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet36"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:00:37 +0000 (0:00:00.175)       0:07:51.479 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:00:38 +0000 (0:00:00.671)       0:07:52.151 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:00:39 +0000 (0:00:01.174)       0:07:53.326 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:40 +0000 (0:00:01.154)       0:07:54.481 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       998M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:00:40 +0000 (0:00:00.186)       0:07:54.667 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:42 +0000 (0:00:01.357)       0:07:56.024 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:00:42 +0000 (0:00:00.176)       0:07:56.200 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:43 +0000 (0:00:00.875)       0:07:57.076 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.14M"
}


TASK [test : Bring up interface Ethernet36] ************************************

Wednesday 13 February 2019  18:00:43 +0000 (0:00:00.175)       0:07:57.251 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:43 +0000 (0:00:00.662)       0:07:57.914 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet12"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:00:44 +0000 (0:00:00.182)       0:07:58.096 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:00:44 +0000 (0:00:00.683)       0:07:58.780 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:00:46 +0000 (0:00:01.176)       0:07:59.957 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:47 +0000 (0:00:01.169)       0:08:01.126 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       992M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:00:47 +0000 (0:00:00.177)       0:08:01.303 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:48 +0000 (0:00:01.342)       0:08:02.646 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:00:48 +0000 (0:00:00.194)       0:08:02.841 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:49 +0000 (0:00:00.808)       0:08:03.649 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.14M"
}


TASK [test : Bring up interface Ethernet12] ************************************

Wednesday 13 February 2019  18:00:49 +0000 (0:00:00.188)       0:08:03.838 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:50 +0000 (0:00:00.673)       0:08:04.511 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet88"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:00:50 +0000 (0:00:00.182)       0:08:04.694 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:00:51 +0000 (0:00:00.691)       0:08:05.385 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:00:52 +0000 (0:00:01.170)       0:08:06.556 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:53 +0000 (0:00:01.154)       0:08:07.710 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       991M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:00:53 +0000 (0:00:00.170)       0:08:07.880 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:55 +0000 (0:00:01.311)       0:08:09.192 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:00:55 +0000 (0:00:00.187)       0:08:09.379 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:56 +0000 (0:00:00.866)       0:08:10.245 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.11M"
}


TASK [test : Bring up interface Ethernet88] ************************************

Wednesday 13 February 2019  18:00:56 +0000 (0:00:00.185)       0:08:10.431 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:00:57 +0000 (0:00:00.649)       0:08:11.080 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet116"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:00:57 +0000 (0:00:00.184)       0:08:11.264 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:00:57 +0000 (0:00:00.662)       0:08:11.927 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:00:59 +0000 (0:00:01.183)       0:08:13.111 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:00 +0000 (0:00:01.194)       0:08:14.305 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       992M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:01:00 +0000 (0:00:00.181)       0:08:14.487 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:01 +0000 (0:00:01.364)       0:08:15.852 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 42"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:01:02 +0000 (0:00:00.183)       0:08:16.036 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:03 +0000 (0:00:01.055)       0:08:17.091 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:16.91M"
}


TASK [test : Bring up interface Ethernet116] ***********************************

Wednesday 13 February 2019  18:01:03 +0000 (0:00:00.169)       0:08:17.260 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:03 +0000 (0:00:00.678)       0:08:17.939 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet80"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:01:04 +0000 (0:00:00.181)       0:08:18.120 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:01:04 +0000 (0:00:00.685)       0:08:18.806 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:01:06 +0000 (0:00:01.181)       0:08:19.988 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:07 +0000 (0:00:01.214)       0:08:21.202 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       1.0G       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:01:07 +0000 (0:00:00.189)       0:08:21.392 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:08 +0000 (0:00:01.371)       0:08:22.764 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 66"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:01:09 +0000 (0:00:00.184)       0:08:22.949 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:09 +0000 (0:00:00.888)       0:08:23.838 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.73M"
}


TASK [test : Bring up interface Ethernet80] ************************************

Wednesday 13 February 2019  18:01:10 +0000 (0:00:00.186)       0:08:24.024 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:10 +0000 (0:00:00.689)       0:08:24.713 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet112"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:01:10 +0000 (0:00:00.175)       0:08:24.889 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:01:11 +0000 (0:00:00.680)       0:08:25.570 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:01:12 +0000 (0:00:01.174)       0:08:26.744 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:14 +0000 (0:00:01.203)       0:08:27.948 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       993M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:01:14 +0000 (0:00:00.187)       0:08:28.135 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:15 +0000 (0:00:01.373)       0:08:29.509 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 54"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:01:15 +0000 (0:00:00.175)       0:08:29.684 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:16 +0000 (0:00:00.937)       0:08:30.621 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.12M"
}


TASK [test : Bring up interface Ethernet112] ***********************************

Wednesday 13 February 2019  18:01:16 +0000 (0:00:00.188)       0:08:30.810 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:17 +0000 (0:00:00.679)       0:08:31.489 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet84"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:01:17 +0000 (0:00:00.181)       0:08:31.671 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:01:18 +0000 (0:00:00.697)       0:08:32.369 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:01:19 +0000 (0:00:01.185)       0:08:33.555 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:20 +0000 (0:00:01.193)       0:08:34.748 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.4G        27M       296M       1.1G\n-/+ buffers/cache:       1.0G       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:01:20 +0000 (0:00:00.182)       0:08:34.931 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:22 +0000 (0:00:01.385)       0:08:36.316 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 36"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:01:22 +0000 (0:00:00.181)       0:08:36.497 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:23 +0000 (0:00:00.856)       0:08:37.353 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:14.08M"
}


TASK [test : Bring up interface Ethernet84] ************************************

Wednesday 13 February 2019  18:01:23 +0000 (0:00:00.176)       0:08:37.530 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:24 +0000 (0:00:00.658)       0:08:38.188 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet48"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:01:24 +0000 (0:00:00.155)       0:08:38.344 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:01:25 +0000 (0:00:00.691)       0:08:39.035 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:01:26 +0000 (0:00:01.185)       0:08:40.221 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:27 +0000 (0:00:01.183)       0:08:41.404 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.4G        27M       296M       1.1G\n-/+ buffers/cache:       1.0G       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:01:27 +0000 (0:00:00.175)       0:08:41.579 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:29 +0000 (0:00:01.366)       0:08:42.946 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 67"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:01:29 +0000 (0:00:00.177)       0:08:43.124 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:30 +0000 (0:00:00.889)       0:08:44.013 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.47M"
}


TASK [test : Bring up interface Ethernet48] ************************************

Wednesday 13 February 2019  18:01:30 +0000 (0:00:00.185)       0:08:44.198 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:30 +0000 (0:00:00.676)       0:08:44.875 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet44"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:01:31 +0000 (0:00:00.179)       0:08:45.054 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:01:31 +0000 (0:00:00.685)       0:08:45.740 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:01:32 +0000 (0:00:01.174)       0:08:46.915 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:34 +0000 (0:00:01.164)       0:08:48.079 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       997M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:01:34 +0000 (0:00:00.171)       0:08:48.251 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:35 +0000 (0:00:01.336)       0:08:49.588 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:01:35 +0000 (0:00:00.177)       0:08:49.766 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:36 +0000 (0:00:00.900)       0:08:50.666 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.21M"
}


TASK [test : Bring up interface Ethernet44] ************************************

Wednesday 13 February 2019  18:01:36 +0000 (0:00:00.183)       0:08:50.849 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:37 +0000 (0:00:00.657)       0:08:51.507 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet40"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:01:37 +0000 (0:00:00.176)       0:08:51.683 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:01:38 +0000 (0:00:00.678)       0:08:52.361 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:01:39 +0000 (0:00:01.174)       0:08:53.536 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:40 +0000 (0:00:01.219)       0:08:54.756 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.5G        27M       296M       1.1G\n-/+ buffers/cache:       996M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:01:40 +0000 (0:00:00.189)       0:08:54.945 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:42 +0000 (0:00:01.385)       0:08:56.331 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 54"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:01:42 +0000 (0:00:00.178)       0:08:56.509 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:43 +0000 (0:00:00.901)       0:08:57.411 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.64M"
}


TASK [test : Bring up interface Ethernet40] ************************************

Wednesday 13 February 2019  18:01:43 +0000 (0:00:00.155)       0:08:57.567 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:44 +0000 (0:00:00.658)       0:08:58.225 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet28"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:01:44 +0000 (0:00:00.177)       0:08:58.402 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:01:45 +0000 (0:00:00.696)       0:08:59.099 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:01:46 +0000 (0:00:01.180)       0:09:00.279 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:47 +0000 (0:00:01.213)       0:09:01.492 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.4G        27M       296M       1.1G\n-/+ buffers/cache:       998M       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:01:47 +0000 (0:00:00.160)       0:09:01.653 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:49 +0000 (0:00:01.346)       0:09:03.000 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 60"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:01:49 +0000 (0:00:00.179)       0:09:03.180 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:50 +0000 (0:00:00.852)       0:09:04.032 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.73M"
}


TASK [test : Bring up interface Ethernet28] ************************************

Wednesday 13 February 2019  18:01:50 +0000 (0:00:00.184)       0:09:04.217 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:50 +0000 (0:00:00.676)       0:09:04.893 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet60"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:01:51 +0000 (0:00:00.176)       0:09:05.070 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:01:51 +0000 (0:00:00.668)       0:09:05.738 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:01:52 +0000 (0:00:01.176)       0:09:06.915 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:54 +0000 (0:00:01.154)       0:09:08.069 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.4G        27M       296M       1.1G\n-/+ buffers/cache:       1.0G       2.9G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:01:54 +0000 (0:00:00.184)       0:09:08.254 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:55 +0000 (0:00:01.328)       0:09:09.582 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:01:55 +0000 (0:00:00.165)       0:09:09.748 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:56 +0000 (0:00:00.795)       0:09:10.543 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.66M"
}


TASK [test : Bring up interface Ethernet60] ************************************

Wednesday 13 February 2019  18:01:56 +0000 (0:00:00.179)       0:09:10.723 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:01:57 +0000 (0:00:00.668)       0:09:11.391 **** 

ok: [falco-test-dut01] => {
    "msg": "Interface to flap Ethernet20"
}


TASK [test : Shutting down interface] ******************************************

Wednesday 13 February 2019  18:01:57 +0000 (0:00:00.178)       0:09:11.570 **** 

changed: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:01:58 +0000 (0:00:00.683)       0:09:12.253 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:01:59 +0000 (0:00:01.184)       0:09:13.438 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:02:00 +0000 (0:00:01.154)       0:09:14.592 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.4G       1.4G        27M       296M       1.1G\n-/+ buffers/cache:       1.0G       2.9G\nSwap:           0B         0B         0B"
}

.............................
.............................

TASK [test : set_fact] *********************************************************

Wednesday 13 February 2019  18:56:40 +0000 (0:00:00.784)       1:03:54.357 **** 

ok: [falco-test-dut01]


TASK [test : conn_graph_facts] *************************************************

Wednesday 13 February 2019  18:56:41 +0000 (0:00:00.773)       1:03:55.130 **** 

ok: [falco-test-dut01]


TASK [test : set_fact] *********************************************************

Wednesday 13 February 2019  18:56:42 +0000 (0:00:01.270)       1:03:56.401 **** 

ok: [falco-test-dut01]


TASK [test : set_fact] *********************************************************

Wednesday 13 February 2019  18:56:43 +0000 (0:00:00.776)       1:03:57.177 **** 

ok: [falco-test-dut01]


TASK [test : Shut down neighbor interface Ethernet7/1 on 172.25.11.53] *********

Wednesday 13 February 2019  18:56:44 +0000 (0:00:00.768)       1:03:57.945 **** 

ok: [falco-test-dut01]


TASK [test : find interface name mapping] **************************************

Wednesday 13 February 2019  18:56:46 +0000 (0:00:02.598)       1:04:00.544 **** 

skipping: [falco-test-dut01]


TASK [test : Shutting down neighbor interface Ethernet7/1 on 172.25.11.53] *****

Wednesday 13 February 2019  18:56:47 +0000 (0:00:00.403)       1:04:00.948 **** 

skipping: [falco-test-dut01]


TASK [test : pause] ************************************************************

Wednesday 13 February 2019  18:56:47 +0000 (0:00:00.403)       1:04:01.351 **** 

Pausing for 1 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Watch memory status] **********************************************

Wednesday 13 February 2019  18:56:49 +0000 (0:00:01.794)       1:04:03.146 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:56:52 +0000 (0:00:03.033)       1:04:06.180 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.6G       1.3G        34M       303M       1.2G\n-/+ buffers/cache:       1.0G       2.8G\nSwap:           0B         0B         0B"
}


TASK [test : watch orchagent CPU utilization] **********************************

Wednesday 13 February 2019  18:56:52 +0000 (0:00:00.761)       1:04:06.941 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:56:56 +0000 (0:00:03.186)       1:04:10.128 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : watch Redis Memory] ***********************************************

Wednesday 13 February 2019  18:56:56 +0000 (0:00:00.757)       1:04:10.886 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:56:59 +0000 (0:00:02.720)       1:04:13.607 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory used_memory_human:13.70M"
}


TASK [test : Bring up neighbor interface Ethernet7/1 on 172.25.11.53] **********

Wednesday 13 February 2019  18:57:00 +0000 (0:00:00.762)       1:04:14.369 **** 

ok: [falco-test-dut01]


TASK [test : Bring up neighbor interface Ethernet7/1 on 172.25.11.53] **********

Wednesday 13 February 2019  18:57:03 +0000 (0:00:02.617)       1:04:16.986 **** 

skipping: [falco-test-dut01]


TASK [test : wait 60 secs so that BGP routes are relearned] ********************

Wednesday 13 February 2019  18:57:03 +0000 (0:00:00.387)       1:04:17.374 **** 

Pausing for 60 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : Record  memory status at end] *************************************

Wednesday 13 February 2019  18:58:04 +0000 (0:01:00.730)       1:05:18.105 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:58:07 +0000 (0:00:02.983)       1:05:21.089 **** 

ok: [falco-test-dut01] => {
    "msg": "Memory Status              total       used       free     shared    buffers     cached\nMem:          3.9G       2.6G       1.3G        34M       303M       1.2G\n-/+ buffers/cache:       1.0G       2.8G\nSwap:           0B         0B         0B"
}


TASK [test : Record orchagent CPU utilization at end] **************************

Wednesday 13 February 2019  18:58:07 +0000 (0:00:00.775)       1:05:21.864 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:58:11 +0000 (0:00:03.240)       1:05:25.104 **** 

ok: [falco-test-dut01] => {
    "msg": "Orchagent CPU Util 0"
}


TASK [test : Record Redis Memory at end] ***************************************

Wednesday 13 February 2019  18:58:11 +0000 (0:00:00.778)       1:05:25.883 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:58:14 +0000 (0:00:02.798)       1:05:28.681 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory 13.71 M"
}


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:58:15 +0000 (0:00:00.778)       1:05:29.460 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory 13.83 M"
}


TASK [test : calculate diff in Redis memory] ***********************************

Wednesday 13 February 2019  18:58:16 +0000 (0:00:00.806)       1:05:30.267 **** 

ok: [falco-test-dut01]


TASK [test : find absolute diff in Redis memory] *******************************

Wednesday 13 February 2019  18:58:17 +0000 (0:00:00.795)       1:05:31.062 **** 

ok: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:58:17 +0000 (0:00:00.798)       1:05:31.860 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis absolute  difference 0.12"
}


TASK [test : calculate decimal increase in Redis memory] ***********************

Wednesday 13 February 2019  18:58:18 +0000 (0:00:00.781)       1:05:32.642 **** 

ok: [falco-test-dut01]


TASK [test : calculate percentage increase in Redis memory] ********************

Wednesday 13 February 2019  18:58:19 +0000 (0:00:00.800)       1:05:33.442 **** 

ok: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:58:20 +0000 (0:00:00.777)       1:05:34.220 **** 

ok: [falco-test-dut01] => {
    "msg": "Redis Memory percentage Increase 0.875273522976"
}


TASK [test : Fail if memory increased more than 5 percent] *********************

Wednesday 13 February 2019  18:58:21 +0000 (0:00:00.777)       1:05:34.997 **** 

skipping: [falco-test-dut01]


TASK [test : Record ipv4 route counts at end] **********************************

Wednesday 13 February 2019  18:58:21 +0000 (0:00:00.415)       1:05:35.413 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:58:24 +0000 (0:00:03.425)       1:05:38.838 **** 

ok: [falco-test-dut01] => {
    "msg": "IPv4 routes at start 6012"
}


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  18:58:25 +0000 (0:00:00.781)       1:05:39.620 **** 

ok: [falco-test-dut01] => {
    "msg": "IPv4 routes at end 6012"
}


TASK [test : Make Sure all ipv4 routes are relearned with jitter of ~5] ********

Wednesday 13 February 2019  18:58:26 +0000 (0:00:00.782)       1:05:40.403 **** 

skipping: [falco-test-dut01]


TASK [test : wait 15 secs] *****************************************************

Wednesday 13 February 2019  18:58:26 +0000 (0:00:00.423)       1:05:40.827 **** 

Pausing for 15 seconds

(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

ok: [falco-test-dut01]


TASK [test : watch orchagent CPU utilization when it goes below 10] ************

Wednesday 13 February 2019  18:58:42 +0000 (0:00:15.768)       1:05:56.595 **** 

changed: [falco-test-dut01]


TASK [test : perform config reload] ********************************************

Wednesday 13 February 2019  18:58:45 +0000 (0:00:03.234)       1:05:59.830 **** 

changed: [falco-test-dut01]


TASK [test : do basic sanity check after each test] ****************************

Wednesday 13 February 2019  19:00:20 +0000 (0:01:34.170)       1:07:34.001 **** 

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/base_sanity.yml for falco-test-dut01


TASK [test : Get process information in syncd docker] **************************

Wednesday 13 February 2019  19:00:21 +0000 (0:00:01.432)       1:07:35.433 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  19:00:24 +0000 (0:00:02.660)       1:07:38.094 **** 

ok: [falco-test-dut01] => {
    "ps_out.stdout_lines": [
        "root        19  0.0  0.0  41584  2576 ?        Sl   19:00   0:00 /usr/bin/dsserve /usr/bin/syncd --diag -p /etc/sai.d/sai.profile", 
        "root        29 19.9  4.6 1058840 187192 ?      Sl   19:00   0:06 /usr/bin/syncd --diag -p /etc/sai.d/sai.profile", 
        "root        52  0.6  5.0 759372 202732 ?       S    19:00   0:00 /usr/bin/syncd --diag -p /etc/sai.d/sai.profile"
    ]
}


TASK [test : Get orchagent process information] ********************************

Wednesday 13 February 2019  19:00:24 +0000 (0:00:00.782)       1:07:38.876 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  19:00:27 +0000 (0:00:02.585)       1:07:41.462 **** 

ok: [falco-test-dut01] => {
    "orch_out.stdout_lines": [
        "31536 /usr/bin/orchagent -d /var/log/swss -b 8192 -m 00:e0:ec:5a:7c:ac"
    ]
}


TASK [test : reboot] ***********************************************************

Wednesday 13 February 2019  19:00:28 +0000 (0:00:00.788)       1:07:42.251 **** 

skipping: [falco-test-dut01]


TASK [test : Get process information in syncd docker] **************************

Wednesday 13 February 2019  19:00:28 +0000 (0:00:00.426)       1:07:42.677 **** 

skipping: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  19:00:29 +0000 (0:00:00.412)       1:07:43.090 **** 

skipping: [falco-test-dut01]


TASK [test : Verify that syncd process is running] *****************************

Wednesday 13 February 2019  19:00:29 +0000 (0:00:00.415)       1:07:43.505 **** 

skipping: [falco-test-dut01]


TASK [test : Get orchagent process information] ********************************

Wednesday 13 February 2019  19:00:29 +0000 (0:00:00.411)       1:07:43.916 **** 

skipping: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  19:00:30 +0000 (0:00:00.413)       1:07:44.330 **** 

skipping: [falco-test-dut01]


TASK [test : Verify that orchagent process is running] *************************

Wednesday 13 February 2019  19:00:30 +0000 (0:00:00.406)       1:07:44.736 **** 

skipping: [falco-test-dut01]


TASK [test : Get syslog error information] *************************************

Wednesday 13 February 2019  19:00:31 +0000 (0:00:00.410)       1:07:45.147 **** 

changed: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  19:00:33 +0000 (0:00:02.585)       1:07:47.733 **** 

ok: [falco-test-dut01] => {
    "syslog_out.stdout_lines": [
        "Feb 13 19:01:03.461315 falco-test-dut01 INFO ansible-command: Invoked with warn=True executable=None chdir=None _raw_params=cat /var/log/syslog |tail -n 5000 |grep -i error removes=None creates=None _uses_shell=True"
    ]
}


TASK [test : validate all interfaces are up after test] ************************

Wednesday 13 February 2019  19:00:34 +0000 (0:00:00.801)       1:07:48.535 **** 

included: /var/jenkins/sonic-mgmt/ansible/roles/test/tasks/interface.yml for falco-test-dut01


TASK [test : Gather minigraph facts about the device] **************************

Wednesday 13 February 2019  19:00:36 +0000 (0:00:01.516)       1:07:50.052 **** 

ok: [falco-test-dut01]


TASK [test : Get interface facts] **********************************************

Wednesday 13 February 2019  19:00:38 +0000 (0:00:02.520)       1:07:52.572 **** 

ok: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  19:00:45 +0000 (0:00:07.357)       1:07:59.929 **** 

skipping: [falco-test-dut01]


TASK [test : reboot] ***********************************************************

Wednesday 13 February 2019  19:00:46 +0000 (0:00:00.413)       1:08:00.343 **** 

skipping: [falco-test-dut01]


TASK [test : figure out fanout switch port in case it was down] ****************

Wednesday 13 February 2019  19:00:46 +0000 (0:00:00.402)       1:08:00.745 **** 

skipping: [falco-test-dut01]


TASK [test : set_fact] *********************************************************

Wednesday 13 February 2019  19:00:47 +0000 (0:00:00.408)       1:08:01.153 **** 

skipping: [falco-test-dut01]


TASK [test : include] **********************************************************

Wednesday 13 February 2019  19:00:47 +0000 (0:00:00.415)       1:08:01.569 **** 


TASK [test : pause and wait interface to be up] ********************************

Wednesday 13 February 2019  19:00:48 +0000 (0:00:00.405)       1:08:01.975 **** 

skipping: [falco-test-dut01]


TASK [test : Get interface facts] **********************************************

Wednesday 13 February 2019  19:00:48 +0000 (0:00:00.392)       1:08:02.367 **** 

skipping: [falco-test-dut01]


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  19:00:48 +0000 (0:00:00.401)       1:08:02.768 **** 

skipping: [falco-test-dut01]


TASK [test : Verify interfaces are up correctly] *******************************

Wednesday 13 February 2019  19:00:49 +0000 (0:00:00.420)       1:08:03.189 **** 

ok: [falco-test-dut01]


TASK [test : Verify port channel interfaces are up correctly] ******************

Wednesday 13 February 2019  19:00:50 +0000 (0:00:00.774)       1:08:03.964 **** 

ok: [falco-test-dut01] => (item=PortChannel0001)

ok: [falco-test-dut01] => (item=PortChannel0003)

ok: [falco-test-dut01] => (item=PortChannel0002)

ok: [falco-test-dut01] => (item=PortChannel0004)


TASK [test : Verify VLAN interfaces are up correctly] **************************

Wednesday 13 February 2019  19:00:54 +0000 (0:00:04.283)       1:08:08.247 **** 

ok: [falco-test-dut01] => (item=Vlan1000)


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  19:00:55 +0000 (0:00:01.368)       1:08:09.616 **** 

ok: [falco-test-dut01] => {
    "msg": "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
}


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  19:00:56 +0000 (0:00:00.767)       1:08:10.383 **** 

ok: [falco-test-dut01] => {
    "msg": "!!!!!!!!!!!!!!!!!!!! end running test continuous_link_flap !!!!!!!!!!!!!!!!!!!!"
}


TASK [test : debug] ************************************************************

Wednesday 13 February 2019  19:00:57 +0000 (0:00:00.783)       1:08:11.167 **** 

ok: [falco-test-dut01] => {
    "msg": "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
}


TASK [test : include] **********************************************************

Wednesday 13 February 2019  19:00:58 +0000 (0:00:00.784)       1:08:11.951 **** 

skipping: [falco-test-dut01]


PLAY ***************************************************************************

skipping: no hosts matched


PLAY RECAP *********************************************************************

falco-test-dut01           : ok=2587 changed=785  unreachable=0    failed=0   



Wednesday 13 February 2019  19:00:58 +0000 (0:00:00.422)       1:08:12.374 **** 

=============================================================================== 

TASK: test : include continuous_peer_link_flap_helper.yml ------------- 475.38s

TASK: test : include continuous_peer_link_flap_helper.yml ------------- 360.37s

TASK: test : include continuous_peer_link_flap_helper.yml ------------- 223.15s

TASK: test : include continuous_link_flap_helper.yml ------------------ 156.22s

TASK: test : include continuous_link_flap_helper.yml ------------------ 111.21s

TASK: test : perform config reload ------------------------------------- 94.17s

TASK: test : wait 60 secs so that BGP routes are relearned ------------- 60.73s

TASK: test : include continuous_link_flap_helper.yml ------------------- 22.59s

TASK: test : wait 15 secs ---------------------------------------------- 15.77s

TASK: test : Get interface facts ---------------------------------------- 7.36s

TASK: setup ------------------------------------------------------------- 5.95s

TASK: test : Get interface facts ---------------------------------------- 5.42s

TASK: test : Verify port channel interfaces are up correctly ------------ 4.28s

TASK: test : Record ipv4 route counts at end ---------------------------- 3.43s

TASK: test : watch orchagent CPU utilization ---------------------------- 3.29s

TASK: test : watch orchagent CPU utilization ---------------------------- 3.25s

TASK: test : watch orchagent CPU utilization ---------------------------- 3.24s

TASK: test : watch orchagent CPU utilization ---------------------------- 3.24s

TASK: test : Record orchagent CPU utilization at end -------------------- 3.24s

TASK: test : watch orchagent CPU utilization ---------------------------- 3.24s

Done

(3372, 0)
----------------------------------------------------------------------------------------------------
Summary:
----------------------------------------------------------------------------------------------------
TestSuite: continuous_link_flap Pass: 3372                 Fail: 0                   

----------------------------------------------------------------------------------------------------

Any platform specific information?

Supported testbed topology if it's a new test case?

Ran on T0 and T1.

Documentation

This is continuous link flap test. In this test,
 1.) Flap all interfaces one by one to cause BGP Flaps (3 iterations).
 2.) Flap all interfaces on peer (FanOutLeaf) one by one to cause BGP Flaps (3 iterations).
 3.) Watch for memory (show system-memory) ,orchagent CPU Utilization and Redis_memory.

Pass Criteria: All routes must be re-learned with < 5% increase in Redis memory
 and with Orchagent CPU consumption below 10% after 3 mins of stopping flaps.

Signed-off-by: Praveen Chaudhary<pchaudhary@linkedin.com>
@praveen-li
Copy link
Copy Markdown
Member Author

Kindly review, Let me know if any issue.

Copy link
Copy Markdown
Contributor

@qiluo-msft qiluo-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As comments

Praveen Chaudhary added 2 commits April 11, 2019 01:04
Signed-off-by: Praveen Chaudhary<pchaudhary@linkedin.com>
[testcases.yml]: run only on t1 to bring l3 interfaces down.

Signed-off-by: Praveen Chaudhary<pchaudhary@linkedin.com>
@praveen-li
Copy link
Copy Markdown
Member Author

Address all review comments, kindly review. Thx.

…k flap test.

Signed-off-by: Praveen Chaudhary<pchaudhary@linkedin.com>
@qiluo-msft qiluo-msft dismissed their stale review April 25, 2019 22:13

Unblocked. Still one minor comment.

@qiluo-msft
Copy link
Copy Markdown
Contributor

I heard the purpose of this test case is to identify unexpected memory increase during link flap. How can I know if it happens or not?

@praveen-li
Copy link
Copy Markdown
Member Author

unexpected memory increase during link flap.

We are watching redis db memory at start and end of the test. If it increases by 5% or more then test case fails.

@praveen-li
Copy link
Copy Markdown
Member Author

Kindly review and merge if we are still writing new tests in Ansible :). Thanks a lot.

@qiluo-msft qiluo-msft merged commit 45df992 into sonic-net:master Jul 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants