Skip to content

New IXIA PFC-WD test for low traffic and low XOFF rate#11091

Open
rraghav-cisco wants to merge 8 commits intosonic-net:masterfrom
rraghav-cisco:pfcwd-issue-snappi
Open

New IXIA PFC-WD test for low traffic and low XOFF rate#11091
rraghav-cisco wants to merge 8 commits intosonic-net:masterfrom
rraghav-cisco:pfcwd-issue-snappi

Conversation

@rraghav-cisco
Copy link
Copy Markdown
Contributor

@rraghav-cisco rraghav-cisco commented Dec 18, 2023

Summary
Modifying the existing pfcwd snappi test to reproduce a corner case in pfcwd.

Type of change
[] Bug fix
[X] Test case(new/improvement)
[] Testbed and Framework(new/improvement)

Back port request
[] 201911
[] 202012
[x] 202205
[x] 202305

Approach
What is the motivation for this PR?
This PR is to add a new test to cover the corner case in pfcwd (low number of pfc pause frames, and a very low traffic). This condition triggered pfcwd in cisco-8000 though it shouldn't have.

How did you do it?
I have added new parameters to the existing testcase. One set of parameters is same as the original test itself. The second set of parameters tests out the corner case as explained above.

How did you verify/test it?
Ran both tests with cisco-8000 platform, once without image fix, and once with the image fix.

Any platform specific information?
No.

Without the fix:
=========================================================================================================== PASSES ===========================================================================================================
________________________________________________________________________________ test_pfcwd_burst_storm_single_lossless_prio[95-None-None-1] _________________________________________________________________________________
------------------------------------------------------------------- generated xml file: /run_logs/ixia/8030/2023-12-23-01-26-43/tr_2023-12-23-01-26-43.xml -------------------------------------------------------------------
INFO:root:Can not get Allure report URL. Please check logs
--------------------------------------------------------------------------------------------------- live log sessionfinish ---------------------------------------------------------------------------------------------------
02:06:06 init.pytest_terminal_summary L0064 INFO | Can not get Allure report URL. Please check logs
================================================================================================== short test summary info ===================================================================================================
PASSED snappi_tests/pfcwd/test_pfcwd_burst_storm_with_snappi.py::test_pfcwd_burst_storm_single_lossless_prio[95-None-None-1]
FAILED snappi_tests/pfcwd/test_pfcwd_burst_storm_with_snappi.py::test_pfcwd_burst_storm_single_lossless_prio[2e-05-1000-5-10] - RuntimeError: Packet drops or other issues are seen in 10 attempts out of 10.
=========================================================================================== 1 failed, 1 passed in 2361.12 seconds ============================================================================================
AzDevOps@e200d6aa4ff5:/data/tests$

With the fix:
________________________________________________________________________________ test_pfcwd_burst_storm_single_lossless_prio[95-None-None-1] _________________________________________________________________________________
________________________________________________________________________________ test_pfcwd_burst_storm_single_lossless_prio[2e-05-1000-5-10] ________________________________________________________________________________
------------------------------------------------------------------- generated xml file: /run_logs/ixia/8030/2023-12-23-00-35-18/tr_2023-12-23-00-35-18.xml -------------------------------------------------------------------
INFO:root:Can not get Allure report URL. Please check logs
--------------------------------------------------------------------------------------------------- live log sessionfinish ---------------------------------------------------------------------------------------------------
01:14:35 init.pytest_terminal_summary L0064 INFO | Can not get Allure report URL. Please check logs
================================================================================================== short test summary info ===================================================================================================
PASSED snappi_tests/pfcwd/test_pfcwd_burst_storm_with_snappi.py::test_pfcwd_burst_storm_single_lossless_prio[95-None-None-1]
PASSED snappi_tests/pfcwd/test_pfcwd_burst_storm_with_snappi.py::test_pfcwd_burst_storm_single_lossless_prio[2e-05-1000-5-10]
================================================================================================ 2 passed in 2355.75 seconds =================================================================================================
AzDevOps@e200d6aa4ff5:/data/tests$

@rraghav-cisco
Copy link
Copy Markdown
Contributor Author

@alpeshspatel , @rbpittman for viz.

@rraghav-cisco rraghav-cisco marked this pull request as ready for review January 13, 2024 17:36
burst_cycle_sec = poll_interval_sec + detect_time_sec + restore_time_sec + 0.1
data_flow_dur_sec = ceil(burst_cycle_sec * BURST_EVENTS)
pause_flow_dur_sec = poll_interval_sec * 0.5
burst_cycle_sec = 2 * (poll_interval_sec + detect_time_sec + restore_time_sec + 0.1)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why 2 * ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will change to cisco-8000 only.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Original test must be run without changing any test behaviour. Add a multiplier only for the low-traffic scenario.

@rraghav-cisco rraghav-cisco changed the title Modifying the test to reproduce the pfcwd corner case. Parameterize IXIA PFC-WD test to test with low traffic and low XOFF rate Jan 31, 2024
@rraghav-cisco
Copy link
Copy Markdown
Contributor Author

Results:

Older image:

==========================
../../usr/local/lib/python3.8/dist-packages/paramiko/transport.py:236
  /usr/local/lib/python3.8/dist-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
    "class": algorithms.Blowfish,

tests/snappi_tests/pfcwd/test_pfcwd_burst_storm_with_snappi_low_rate.py::test_pfcwd_burst_storm_single_lossless_prio_low_rate
  /usr/local/lib/python3.8/dist-packages/snappi_ixnetwork/device/utils.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
    from collections import namedtuple, Mapping

tests/snappi_tests/pfcwd/test_pfcwd_burst_storm_with_snappi_low_rate.py::test_pfcwd_burst_storm_single_lossless_prio_low_rate
tests/snappi_tests/pfcwd/test_pfcwd_burst_storm_with_snappi_low_rate.py::test_pfcwd_burst_storm_single_lossless_prio_low_rate
  /home/sonic/.local/lib/python3.8/site-packages/ixnetwork_restpy/testplatform/sessions/sessions.py:66: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    elif LooseVersion(build_number) < LooseVersion("8.52"):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
------------------------------------------------------------------- generated xml file: /run_logs/ixia/8030/2024-04-05-02-08-59/tr_2024-04-05-02-08-59.xml -------------------------------------------------------------------
INFO:root:Can not get Allure report URL. Please check logs
--------------------------------------------------------------------------------------------------- live log sessionfinish ---------------------------------------------------------------------------------------------------
INFO     root:__init__.py:64 Can not get Allure report URL. Please check logs
================================================================================================== short test summary info ===================================================================================================
ERROR snappi_tests/pfcwd/test_pfcwd_burst_storm_with_snappi_low_rate.py::test_pfcwd_burst_storm_single_lossless_prio_low_rate - Failed: Processes "['__dut_reload--<MultiAsicSonicHost rdma-m64>']" failed with exit code "1"
FAILED snappi_tests/pfcwd/test_pfcwd_burst_storm_with_snappi_low_rate.py::test_pfcwd_burst_storm_single_lossless_prio_low_rate - Failed: No data packet should be dropped
===================================================================================== 1 failed, 4 warnings, 1 error in 991.74s (0:16:31) =====================================================================================
sonic@snappi-202305-sonic-mgmt:/data/tests$

Newer images:

=========================================================================================================== PASSES ===========================================================================================================
____________________________________________________________________________________ test_pfcwd_burst_storm_single_lossless_prio_low_rate ____________________________________________________________________________________
------------------------------------------------------------------- generated xml file: /run_logs/ixia/8030/2024-04-05-04-45-37/tr_2024-04-05-04-45-37.xml -------------------------------------------------------------------
INFO:root:Can not get Allure report URL. Please check logs
--------------------------------------------------------------------------------------------------- live log sessionfinish ---------------------------------------------------------------------------------------------------
INFO     root:__init__.py:64 Can not get Allure report URL. Please check logs
================================================================================================== short test summary info ===================================================================================================
PASSED snappi_tests/pfcwd/test_pfcwd_burst_storm_with_snappi_low_rate.py::test_pfcwd_burst_storm_single_lossless_prio_low_rate
========================================================================================= 1 passed, 4 warnings in 691.02s (0:11:31) ==========================================================================================
sonic@snappi-202305-sonic-mgmt:/data/tests$

@rraghav-cisco rraghav-cisco changed the title Parameterize IXIA PFC-WD test to test with low traffic and low XOFF rate New IXIA PFC-WD test for low traffic and low XOFF rate Apr 18, 2024
@abdosi
Copy link
Copy Markdown
Contributor

abdosi commented Jun 21, 2024

let's have multi-dut version also for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants