Skip to content

Enhance qos tests to support single-asic, multi-asic, and multi-dut testing#7556

Merged
vmittal-msft merged 1 commit intosonic-net:202205from
sanmalho-git:qos_new_2205
Mar 31, 2023
Merged

Enhance qos tests to support single-asic, multi-asic, and multi-dut testing#7556
vmittal-msft merged 1 commit intosonic-net:202205from
sanmalho-git:qos_new_2205

Conversation

@sanmalho-git
Copy link
Copy Markdown
Contributor

@sanmalho-git sanmalho-git commented Feb 23, 2023

Description of PR

Summary:
Fixes # (issue)

This is same as PR #6946 from 'master' branch that can't be cherry-picked without merge conflicts into '202205' branch.

The existing QoS (test_qos_sai.py) is written to accomodata a single asic on a single Dut. But, we require the same tests to be executed against a T2 chassis (with single/multi-asic linecards) and multi-asic pizza boxes.

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 201911
  • 202012
  • 202205

Approach

What is the motivation for this PR?

All the test cases create a list of src and dst ports. For the different modes, here is the distribution of the src and dst ports:

  • single_asic: The src and dst ports are on the same asic on the same linecard.
  • single_dut_multi_asic: On a multi-asic DUT/linecard, the src port is on an asic, while the dst ports are on another asic on the same DUT/linecard
  • multi_dut: The src port is on an asic on one of the DUT/linecards, and the dst port is on another asic on another DUT/linecard. This is currently only required for T2 topology

How did you do it?

Approach to accomplish this is the following:

  • All the tests have to parameterized for the 3 modes defined above.

    • This is done using the 'select_src_and_dst_dut_and_asic' fixture that is parameterized for 'single_asic', 'single_dut_multi_asic', 'multi_dut' - Based on the mode, it sets the src_dut_index, dst_dut_index, src_asic_index and dst_asic_index
    • Added fixture 'get_src_dst_asic_and_duts' that returns dictionary of the src_dut_index, dst_dut_index, src_asic_index, and dst_asic_index, and the src_dut and dst_dut (instances of MultiAsicSonicHost), src_asic and dst_asic (instances of Asic), and also a list of all DUTs and all Asics
  • dutConfig is modified such that testPortIds and testPortIps are collecting from all the duts and asics involved and stored in a dictionary with key being the dutIndex and value being a dictionary per asic index.

    • __buildTestPorts then sets the src and dst ports based on the src_dut_index, dst_dut_index, src_asic_index and dst_asic_index
  • All the other fixtures and tests, we use 'get_src_dst_asci_and_duts' fixture instead of enum_rand_one_frontend_hostname and enum_frontend_index.

    • The code instead the fixtures and tests is modified to the actions on the correct src/dst dut or asic. For example: - swap_syncd fixture would swap syncd docker on all DUT's (both src and dst) instead of just one DUT as before. - stopServices - do it all_duts (src and dst duts)
  • Similarly, changes to saitests involved dealing with multiple DUTs (and thus multiple sai clients) and modifying other data structure like 'interface_to_front_mapping' in sai_base_test.py and port_list, sai_port_list, front_port_list in switch.py to deal with multiple duts (modified to be dictionary with key being 'src' and 'dst')

    • tests in sai_qos_tests.py pass src_dut_index, src_asic_index, dst_dut_index and dst_asic_index in the testParams.
      • The saitests classes then use this to do the actions on the right client and ports.

Assumptions:

  • For multi-dut, we are assuming that hwsku for all the cards are same.

How did you verify/test it?

This is verified on T0, T1, dual ToR as well as t2 topologies.

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

…esting

The existing QoS (test_qos_sai.py) is written to accomodate a single asic on a single Dut.
But, we require the same tests to be executed against a T2 chassis (with single/multi-asic linecards) and multi-asic pizza boxes.

All the test cases create a list of src and dst ports. For the different modes, here is the distribution of the src and dst ports:
- single_asic: The src and dst ports are on the same asic on the same linecard.
- single_dut_multi_asic: On a multi-asic DUT/linecard, the src port is on an asic, while the dst ports are on another asic on the same DUT/linecard
- multi_dut: The src port is on an asic on one of the DUT/linecards, and the dst port is on another asic on another DUT/linecard. This is currently only required for T2 topology

Approach to accomplish this is the following:
- All the tests have to parameterized for the 3 modes defined above.
  - This is done using the 'select_src_and_dst_dut_and_asic' fixture that is parameterized for 'single_asic', 'single_dut_multi_asic', 'multi_dut'
    Based on the mode, it sets the src_dut_index, dst_dut_index, src_asic_index and dst_asic_index

  - Added fixture 'get_src_dst_asic_and_duts' that returns dictionary of the src_dut_index, dst_dut_index, src_asic_index, and dst_asic_index,
    and the src_dut and dst_dut (instances of MultiAsicSonicHost), src_asic and dst_asic (instances of Asic), and also a list of all DUTs and all Asics
  - dutConfig is modified such that testPortIds and testPortIps are collecting from all the duts and asics involved and stored in a dictionary with key being the dutIndex and value being a dictionary per asic index.
     - __buildTestPorts then sets the src and dst ports based on the src_dut_index, dst_dut_index, src_asic_index and dst_asic_index
     - All the other fixtures and tests, we use 'get_src_dst_asci_and_duts' fixture instead of enum_rand_one_frontend_hostname and enum_frontend_index.
     - The code instead the fixtures and tests is modified to the actions on the correct src/dst dut or asic.
       For example:
         - swap_syncd fixture would swap syncd docker on all DUT's (both src and dst) instead of just one DUT as before.
         - stopServices - do it all_duts (src and dst duts)

  - Similarly, changes to saitests involved dealing with multiple DUTs (and thus multiple sai clients) and modifying other data structure
    like 'interface_to_front_mapping' in sai_base_test.py and port_list, sai_port_list, front_port_list in switch.py
    to deal with multiple duts (modified to be dictionary with key being 'src' and 'dst')
      - tests in sai_qos_tests.py pass src_dut_index, src_asic_index, dst_dut_index and dst_asic_index in the testParams.
         - The saitests classes then use this to do the actions on the right client and ports.

Assumptions:
  - For multi-dut, we are assuming that hwsku for all the cards are same.
@XuChen-MSFT
Copy link
Copy Markdown
Contributor

XuChen-MSFT commented Apr 4, 2023

@sanmalho-git looks like this PR break some function which depends on original format of testPortIds

@stephenxs
testPortIds' data model changed, as below example:

before this PR:

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24] IPs:{1: {'peer_addr': '192.168.0.3'}, 2: {'peer_addr': '192.168.0.4'}, 3: {'peer_addr': '192.168.0.5'}, 4: {'peer_addr': '192.168.0.6'}, 5: {'peer_addr': '192.168.0.7'}, 6: {'peer_addr': '192.168.0.8'}, 7: {'peer_addr': '192.168.0.9'}, 8: {'peer_addr': '192.168.0.10'}, 9: {'peer_addr': '192.168.0.11'}, 10: {'peer_addr': '192.168.0.12'}, 11: {'peer_addr': '192.168.0.13'}, 12: {'peer_addr': '192.168.0.14'}, 13: {'peer_addr': '192.168.0.15'}, 14: {'peer_addr': '192.168.0.16'}, 15: {'peer_addr': '192.168.0.17'}, 16: {'peer_addr': '192.168.0.18'}, 17: {'peer_addr': '192.168.0.19'}, 18: {'peer_addr': '192.168.0.20'}, 19: {'peer_addr': '192.168.0.21'}, 20: {'peer_addr': '192.168.0.22'}, 21: {'peer_addr': '192.168.0.23'}, 22: {'peer_addr': '192.168.0.24'}, 23: {'peer_addr': '192.168.0.25'}, 24: {'peer_addr': '192.168.0.26'}}

After this PR:

{0: {0: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]}} IPs:{0: {0: {1: {'peer_addr': '192.168.0.3'}, 2: {'peer_addr': '192.168.0.4'}, 3: {'peer_addr': '192.168.0.5'}, 4: {'peer_addr': '192.168.0.6'}, 5: {'peer_addr': '192.168.0.7'}, 6: {'peer_addr': '192.168.0.8'}, 7: {'peer_addr': '192.168.0.9'}, 8: {'peer_addr': '192.168.0.10'}, 9: {'peer_addr': '192.168.0.11'}, 10: {'peer_addr': '192.168.0.12'}, 11: {'peer_addr': '192.168.0.13'}, 12: {'peer_addr': '192.168.0.14'}, 13: {'peer_addr': '192.168.0.15'}, 14: {'peer_addr': '192.168.0.16'}, 15: {'peer_addr': '192.168.0.17'}, 16: {'peer_addr': '192.168.0.18'}, 17: {'peer_addr': '192.168.0.19'}, 18: {'peer_addr': '192.168.0.20'}, 19: {'peer_addr': '192.168.0.21'}, 20: {'peer_addr': '192.168.0.22'}, 21: {'peer_addr': '192.168.0.23'}, 22: {'peer_addr': '192.168.0.24'}, 23: {'peer_addr': '192.168.0.25'}, 24: {'peer_addr': '192.168.0.26'}}}}

your function "collect_qos_configurations()" in tests\qos\files\mellanox\qos_param_generator.py was broken, a raise below exception:

            for i in range(1, ingress_ports_num_shp):
                for j in range(pgs_per_port):
                    pkts_num_trig_pfc_shp.append(occupancy_per_port + xon + hysteresis)
                    occupancy_per_port /= 2
>               ingress_ports_list_shp.append(testPortIds[i])
E               KeyError: 1

Can you help to update your affected function?

@XuChen-MSFT
Copy link
Copy Markdown
Contributor

and testPortIps related operations are broken as well.

        testParams.update({
            "dscp": qosConfig[LosslessVoqProfile]["dscp"],
            "ecn": qosConfig[LosslessVoqProfile]["ecn"],
            "pg": qosConfig[LosslessVoqProfile]["pg"],
            "dst_port_id": dst_port_id,
            "dst_port_ip": dst_port_ip,
            "src_port_1_id": src_port_1_id,
            "src_port_1_ip": testPortIps[src_port_1_id]['peer_addr'],    >>>>>  break here
            "src_port_2_id": src_port_2_id,
            "src_port_2_ip": testPortIps[src_port_2_id]['peer_addr'],   >>>>>  break here

revert this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants