Reduce the number of ifconfig calls made in add_topo by ccroy-arista · Pull Request #19119 · sonic-net/sonic-mgmt

ccroy-arista · 2025-06-20T21:25:53Z

Description of PR

Within VMTopology.init, ifconfig -a is called for every VM. For topologies with several hundred VMs, each call to ifconfig -a can take an appreciable amount of time (from seconds to minutes, depending on exact number of VMs and processing power of the testbed server). Multiplying this across hundreds of VMs results in add_topo taking a significant amount of time to complete.

This change decreases the add_topo setup time by reducing the number of ifconfig -a calls within VMTopology.init from to 1.

This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations.

Summary:
Fixes # (issue)

Type of change

Back port request

Approach

What is the motivation for this PR?

To reduce the setup time for add_topo, particularly for topologies with a larger number of VMs.

How did you do it?

Reduced the number of ifconfig -a calls within VMTopology.init(...) from to 1.

How did you verify/test it?

Ran add_topo on a t1-isolated topology and confirmed that the setup time was reduced.

Any platform specific information?

This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations.

`ifconfig -a` is called for every VM in a topology within ansible/roles/vm_set/library/vm_topology.py. For topologies with a couple hundred VMs, this can take anywhere from seconds to minutes for each call to `ifconfig -a`. Multiplying this across the number of VMs results in these calls taking up the majority of time for add_topo to complete.

mssonicbld · 2025-06-20T21:26:00Z

/azp run

azure-pipelines · 2025-06-20T21:26:13Z

Azure Pipelines successfully started running 1 pipeline(s).

wsycqyz · 2025-06-23T03:13:34Z

The pre-commit check failed with:

flake8...................................................................Failed

hook id: flake8
exit code: 1

ansible/roles/vm_set/library/vm_topology.py:304:31: E201 whitespace after '['
ansible/roles/vm_set/library/vm_topology.py:304:70: E202 whitespace before ']'

@ccroy-arista can you help fix the issue?

saiarcot895 · 2025-06-23T21:53:16Z

+            # dualtor, hence why the former approach is preserved above for that case.
+            else:
+                # Get the intf information in a tabulated format. Discard the header row.
+                intf_rows = VMTopology.cmd('ifconfig -a -s').split('\n')[1:]


Can ip -j addr (JSON output of ip addr) be used? That way, manual string parsing isn't needed, and it also replaces one use of ifconfig with ip.

Updated to use ip -j addr.

@saiarcot895 I'm looking into using a sysfs approach with os.listdir, which should be an even more portable and general solution; I will provide an update once I have tested this approach, and if it works well then I'll update the PR as well.

@saiarcot895 I've updated the changes to use the os.listdir approach now.

Fix formatting errors encountered during pre-commit check.

mssonicbld · 2025-06-23T22:09:26Z

/azp run

azure-pipelines · 2025-06-23T22:09:40Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-06-24T14:40:48Z

/azp run

azure-pipelines · 2025-06-24T14:41:04Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-06-26T19:00:02Z

/azp run

azure-pipelines · 2025-06-26T19:00:18Z

Azure Pipelines successfully started running 1 pipeline(s).

StormLiangMS · 2025-07-03T02:30:44Z

/azp run

azure-pipelines · 2025-07-03T02:30:58Z

Azure Pipelines successfully started running 1 pipeline(s).

r12f · 2025-07-03T17:42:04Z

hi @ccroy-arista , anything happened to this PR?

ccroy-arista · 2025-07-03T17:45:26Z

Hi @r12f, I am updating this PR to a more portable and robust solution per: #19119 (comment)

I will mark the PR active again once the change is updated.

r12f · 2025-07-03T17:48:14Z

Ah, I see. Got it and no worries. Feel free to leave the PR in open state. It is just a regular "review-and-address-comment" cycles :D

ccroy-arista · 2025-07-03T17:50:19Z

Thanks Riff, will do :)

`ifconfig -a` is called for every VM in a topology within ansible/roles/vm_set/library/vm_topology.py. For topologies with a couple hundred VMs, this can take anywhere from seconds to minutes for each call to `ifconfig -a`. Multiplying this across the number of VMs results in these calls taking up the majority of time for add_topo to complete.

mssonicbld · 2025-07-08T23:32:06Z

/azp run

azure-pipelines · 2025-07-08T23:32:19Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-07-08T23:38:04Z

/azp run

azure-pipelines · 2025-07-09T20:07:32Z

Azure Pipelines successfully started running 1 pipeline(s).

StormLiangMS · 2025-07-10T05:02:59Z

hi @saiarcot895 would you help to review again for the update?

StormLiangMS

LGTM

StormLiangMS · 2025-07-17T05:03:08Z

hi @ccroy-arista could you check on the conflict to 202412?

ccroy-arista · 2025-07-17T14:49:43Z

Manual cherry-pick to 202412 to resolve conflict: Azure/sonic-mgmt.msft#557

What is the motivation for this PR? To reduce the setup time for add_topo, particularly for topologies with a larger number of VMs. How did you do it? Reduced the number of ifconfig -a calls within VMTopology.init(...) from to 1. How did you verify/test it? Ran add_topo on a t1-isolated topology and confirmed that the setup time was reduced. Any platform specific information? This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations.

mssonicbld · 2025-07-18T00:35:13Z

Cherry-pick PR to 202505: #19693

What is the motivation for this PR? To reduce the setup time for add_topo, particularly for topologies with a larger number of VMs. How did you do it? Reduced the number of ifconfig -a calls within VMTopology.init(...) from to 1. How did you verify/test it? Ran add_topo on a t1-isolated topology and confirmed that the setup time was reduced. Any platform specific information? This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations.

### Description of PR This is a manual cherry-pick of sonic-net/sonic-mgmt#19119 to 202412. Within `VMTopology.init`, `ifconfig -a` is called for every VM. For topologies with several hundred VMs, each call to `ifconfig -a` can take an appreciable amount of time (from seconds to minutes, depending on exact number of VMs and processing power of the testbed server). Multiplying this across hundreds of VMs results in `add_topo` taking a significant amount of time to complete. This change decreases the `add_topo` setup time by reducing the number of `ifconfig -a` calls within `VMTopology.init` from <number of VMs in the topology> to 1. This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations. Summary: Fixes # (issue) ### Type of change - [ ] Bug fix - [x] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [x] 202412 ### Approach #### What is the motivation for this PR? To reduce the setup time for `add_topo`, particularly for topologies with a larger number of VMs. #### How did you do it? Reduced the number of `ifconfig -a` calls within `VMTopology.init(...)` from <number of VMs in the topology> to 1. #### How did you verify/test it? Ran `add_topo` on a t1-isolated topology and confirmed that the setup time was reduced. #### Any platform specific information? This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations.

What is the motivation for this PR? To reduce the setup time for add_topo, particularly for topologies with a larger number of VMs. How did you do it? Reduced the number of ifconfig -a calls within VMTopology.init(...) from to 1. How did you verify/test it? Ran add_topo on a t1-isolated topology and confirmed that the setup time was reduced. Any platform specific information? This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations. Signed-off-by: opcoder0 <110003254+opcoder0@users.noreply.github.com>

What is the motivation for this PR? To reduce the setup time for add_topo, particularly for topologies with a larger number of VMs. How did you do it? Reduced the number of ifconfig -a calls within VMTopology.init(...) from to 1. How did you verify/test it? Ran add_topo on a t1-isolated topology and confirmed that the setup time was reduced. Any platform specific information? This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations. Signed-off-by: Guy Shemesh <gshemesh@nvidia.com>

What is the motivation for this PR? To reduce the setup time for add_topo, particularly for topologies with a larger number of VMs. How did you do it? Reduced the number of ifconfig -a calls within VMTopology.init(...) from to 1. How did you verify/test it? Ran add_topo on a t1-isolated topology and confirmed that the setup time was reduced. Any platform specific information? This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations.

What is the motivation for this PR? To reduce the setup time for add_topo, particularly for topologies with a larger number of VMs. How did you do it? Reduced the number of ifconfig -a calls within VMTopology.init(...) from to 1. How did you verify/test it? Ran add_topo on a t1-isolated topology and confirmed that the setup time was reduced. Any platform specific information? This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations. Signed-off-by: Guy Shemesh <gshemesh@nvidia.com>

What is the motivation for this PR? To reduce the setup time for add_topo, particularly for topologies with a larger number of VMs. How did you do it? Reduced the number of ifconfig -a calls within VMTopology.init(...) from to 1. How did you verify/test it? Ran add_topo on a t1-isolated topology and confirmed that the setup time was reduced. Any platform specific information? This change reduces setup time for non-multi-DUT configurations only; it does not impact the setup time for multi-DUT configurations. Signed-off-by: Yael Tzur <ytzur@nvidia.com>

wsycqyz added the Request for 202505 branch label Jun 23, 2025

saiarcot895 reviewed Jun 23, 2025

View reviewed changes

Fix E201/E202 python formatting errors

d938a62

Fix formatting errors encountered during pre-commit check.

Replace 'ifconfig -a -s' with 'ip -j addr' in check_bridge.

e646b47

Merge branch 'sonic-net:master' into fix-excess-ifconfigs

e00fb20

r12f added the Request for msft-202412 Branch label Jul 3, 2025

ccroy-arista marked this pull request as draft July 3, 2025 16:48

ccroy-arista marked this pull request as ready for review July 3, 2025 17:50

mssonicbld added the Cherry Pick Conflict_msft-202412 label Jul 8, 2025

sonic-mgmt: restore location of assignment of self._is_multi_duts

ec7a913

saiarcot895 approved these changes Jul 15, 2025

View reviewed changes

StormLiangMS approved these changes Jul 17, 2025

View reviewed changes

StormLiangMS merged commit 96bfe4d into sonic-net:master Jul 17, 2025
18 checks passed

ccroy-arista mentioned this pull request Jul 17, 2025

Reduce the number of ifconfig calls made in add_topo Azure/sonic-mgmt.msft#557

Merged

11 tasks

yejianquan added the Approved for 202505 branch label Jul 18, 2025

mssonicbld added the Created PR to 202505 branch label Jul 18, 2025

mssonicbld mentioned this pull request Jul 18, 2025

[action] [PR:19119] Reduce the number of ifconfig calls made in add_topo #19693

Merged

13 tasks

mssonicbld added Included in 202505 branch and removed Created PR to 202505 branch labels Jul 18, 2025

ccroy-arista deleted the fix-excess-ifconfigs branch July 31, 2025 05:42

StormLiangMS added the Included in msft-202412 Branch label Aug 7, 2025

Conversation

ccroy-arista commented Jun 20, 2025

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Uh oh!

mssonicbld commented Jun 20, 2025

Uh oh!

azure-pipelines Bot commented Jun 20, 2025

Uh oh!

wsycqyz commented Jun 23, 2025

Uh oh!

saiarcot895 Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ccroy-arista Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

ccroy-arista Jun 27, 2025

Choose a reason for hiding this comment

Uh oh!

ccroy-arista Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Jun 23, 2025

Uh oh!

azure-pipelines Bot commented Jun 23, 2025

Uh oh!

mssonicbld commented Jun 24, 2025

Uh oh!

azure-pipelines Bot commented Jun 24, 2025

Uh oh!

mssonicbld commented Jun 26, 2025

Uh oh!

azure-pipelines Bot commented Jun 26, 2025

Uh oh!

StormLiangMS commented Jul 3, 2025

Uh oh!

azure-pipelines Bot commented Jul 3, 2025

Uh oh!

r12f commented Jul 3, 2025

Uh oh!

ccroy-arista commented Jul 3, 2025

Uh oh!

r12f commented Jul 3, 2025

Uh oh!

ccroy-arista commented Jul 3, 2025

Uh oh!

mssonicbld commented Jul 8, 2025

Uh oh!

azure-pipelines Bot commented Jul 8, 2025

Uh oh!

mssonicbld commented Jul 8, 2025

Uh oh!

azure-pipelines Bot commented Jul 9, 2025

Uh oh!

StormLiangMS commented Jul 10, 2025

Uh oh!

StormLiangMS left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

StormLiangMS commented Jul 17, 2025

Uh oh!

ccroy-arista commented Jul 17, 2025

Uh oh!

mssonicbld commented Jul 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

saiarcot895 Jun 23, 2025 •

edited

Loading