Enhance qos tests to support single-asic, multi-asic, and multi-dut testing#7556
Merged
vmittal-msft merged 1 commit intosonic-net:202205from Mar 31, 2023
Merged
Conversation
bf6281b to
a3a1a01
Compare
6 tasks
f16f831 to
2d2cb9d
Compare
2d2cb9d to
e324453
Compare
dbc6c26 to
32c9564
Compare
301eae0 to
d0d660e
Compare
vmittal-msft
approved these changes
Mar 30, 2023
…esting
The existing QoS (test_qos_sai.py) is written to accomodate a single asic on a single Dut.
But, we require the same tests to be executed against a T2 chassis (with single/multi-asic linecards) and multi-asic pizza boxes.
All the test cases create a list of src and dst ports. For the different modes, here is the distribution of the src and dst ports:
- single_asic: The src and dst ports are on the same asic on the same linecard.
- single_dut_multi_asic: On a multi-asic DUT/linecard, the src port is on an asic, while the dst ports are on another asic on the same DUT/linecard
- multi_dut: The src port is on an asic on one of the DUT/linecards, and the dst port is on another asic on another DUT/linecard. This is currently only required for T2 topology
Approach to accomplish this is the following:
- All the tests have to parameterized for the 3 modes defined above.
- This is done using the 'select_src_and_dst_dut_and_asic' fixture that is parameterized for 'single_asic', 'single_dut_multi_asic', 'multi_dut'
Based on the mode, it sets the src_dut_index, dst_dut_index, src_asic_index and dst_asic_index
- Added fixture 'get_src_dst_asic_and_duts' that returns dictionary of the src_dut_index, dst_dut_index, src_asic_index, and dst_asic_index,
and the src_dut and dst_dut (instances of MultiAsicSonicHost), src_asic and dst_asic (instances of Asic), and also a list of all DUTs and all Asics
- dutConfig is modified such that testPortIds and testPortIps are collecting from all the duts and asics involved and stored in a dictionary with key being the dutIndex and value being a dictionary per asic index.
- __buildTestPorts then sets the src and dst ports based on the src_dut_index, dst_dut_index, src_asic_index and dst_asic_index
- All the other fixtures and tests, we use 'get_src_dst_asci_and_duts' fixture instead of enum_rand_one_frontend_hostname and enum_frontend_index.
- The code instead the fixtures and tests is modified to the actions on the correct src/dst dut or asic.
For example:
- swap_syncd fixture would swap syncd docker on all DUT's (both src and dst) instead of just one DUT as before.
- stopServices - do it all_duts (src and dst duts)
- Similarly, changes to saitests involved dealing with multiple DUTs (and thus multiple sai clients) and modifying other data structure
like 'interface_to_front_mapping' in sai_base_test.py and port_list, sai_port_list, front_port_list in switch.py
to deal with multiple duts (modified to be dictionary with key being 'src' and 'dst')
- tests in sai_qos_tests.py pass src_dut_index, src_asic_index, dst_dut_index and dst_asic_index in the testParams.
- The saitests classes then use this to do the actions on the right client and ports.
Assumptions:
- For multi-dut, we are assuming that hwsku for all the cards are same.
d0d660e to
65308cd
Compare
This was referenced Apr 3, 2023
Contributor
|
@sanmalho-git looks like this PR break some function which depends on original format of testPortIds @stephenxs before this PR: After this PR: your function "collect_qos_configurations()" in tests\qos\files\mellanox\qos_param_generator.py was broken, a raise below exception: Can you help to update your affected function? |
Contributor
|
and testPortIps related operations are broken as well. revert this PR. |
XuChen-MSFT
added a commit
to XuChen-MSFT/sonic-mgmt
that referenced
this pull request
Apr 4, 2023
…ti-dut testing (sonic-net#7556)" This reverts commit 01ca468.
6 tasks
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary:
Fixes # (issue)
This is same as PR #6946 from 'master' branch that can't be cherry-picked without merge conflicts into '202205' branch.
The existing QoS (test_qos_sai.py) is written to accomodata a single asic on a single Dut. But, we require the same tests to be executed against a T2 chassis (with single/multi-asic linecards) and multi-asic pizza boxes.
Type of change
Back port request
Approach
What is the motivation for this PR?
All the test cases create a list of src and dst ports. For the different modes, here is the distribution of the src and dst ports:
How did you do it?
Approach to accomplish this is the following:
All the tests have to parameterized for the 3 modes defined above.
dutConfig is modified such that testPortIds and testPortIps are collecting from all the duts and asics involved and stored in a dictionary with key being the dutIndex and value being a dictionary per asic index.
All the other fixtures and tests, we use 'get_src_dst_asci_and_duts' fixture instead of enum_rand_one_frontend_hostname and enum_frontend_index.
Similarly, changes to saitests involved dealing with multiple DUTs (and thus multiple sai clients) and modifying other data structure like 'interface_to_front_mapping' in sai_base_test.py and port_list, sai_port_list, front_port_list in switch.py to deal with multiple duts (modified to be dictionary with key being 'src' and 'dst')
Assumptions:
How did you verify/test it?
This is verified on T0, T1, dual ToR as well as t2 topologies.
Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation