Fix 1 test plan fail will cancel all peer test plans issue by xwjiang-ms · Pull Request #21138 · sonic-net/sonic-mgmt

xwjiang-ms · 2025-10-29T23:58:00Z

Description of PR

Summary:
Fixes # (issue)

Type of change

Back port request

Approach

What is the motivation for this PR?

When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by #12978 using set -o and failure counter, but set -o was changed back by #15618.

How did you do it?

Change set -e back to set -o to make failure counter work again.

How did you verify/test it?

Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

mssonicbld · 2025-10-29T23:58:08Z

/azp run

azure-pipelines · 2025-10-29T23:58:23Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-10-30T00:39:02Z

/azp run

azure-pipelines · 2025-10-30T00:39:15Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-10-30T01:18:42Z

/azp run

azure-pipelines · 2025-10-30T01:18:58Z

Azure Pipelines successfully started running 1 pipeline(s).

StormLiangMS

LGTM

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: vikumarks <vikumar7ks@gmail.com>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: Alberto Villarreal <alberto.villarreal@keysight.com>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: opcoder0 <110003254+opcoder0@users.noreply.github.com>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: selldinesh <dinesh.sellappan@keysight.com>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: Saravanan <saravanan@nexthop.ai>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: Guy Shemesh <gshemesh@nvidia.com>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: Aharon Malkin <amalkin@nvidia.com>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: Guy Shemesh <gshemesh@nvidia.com>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: YiFan Wang <yifan@nexthop.ai>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: Guy Shemesh <gshemesh@nvidia.com>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: Lakshmi Yarramaneni <lakshmi@nexthop.ai>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: Yael Tzur <ytzur@nvidia.com>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>

…#21138) What is the motivation for this PR? When we use TEST_PLAN_NUM and .azure-pipelines/run-test-elastictest-template.yml to create a batch of test plans, if 1 test plan fails, the others will be cancelled because test_plan.py will raise exception then run sys.exit. Previously this issue was fixed by sonic-net#12978 using set -o and failure counter, but set -o was changed back by sonic-net#15618. How did you do it? Change set -e back to set -o to make failure counter work again. How did you verify/test it? Verified in pipeline https://dev.azure.com/mssonic/build/_build/results?buildId=971751&view=logs&j=9657d7a6-3fe9-53eb-d67f-701685efa9c8&t=316819a7-603a-55dc-2941-0b950b5989c2 image

Fix 1 test plan fail will cancel all issue

72c0a6b

xwjiang-ms requested a review from wangxin as a code owner October 29, 2025 23:58

xwjiang-ms marked this pull request as draft October 29, 2025 23:58

fix

3bc9f9e

fix

2fc59db

xwjiang-ms marked this pull request as ready for review October 30, 2025 01:35

xwjiang-ms changed the title ~~Fix 1 test plan fail will cancel all issue~~ Fix 1 test plan fail will cancel all peer test plans issue Oct 30, 2025

auspham approved these changes Oct 30, 2025

View reviewed changes

StormLiangMS approved these changes Oct 31, 2025

View reviewed changes

StormLiangMS merged commit 3defdc9 into sonic-net:master Oct 31, 2025
19 checks passed

xwjiang-ms deleted the fix_1_test_plan_cancel_all branch October 31, 2025 04:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix 1 test plan fail will cancel all peer test plans issue#21138

Fix 1 test plan fail will cancel all peer test plans issue#21138
StormLiangMS merged 3 commits intosonic-net:masterfrom
xwjiang-ms:fix_1_test_plan_cancel_all

xwjiang-ms commented Oct 29, 2025 •

edited

Loading

Uh oh!

mssonicbld commented Oct 29, 2025

Uh oh!

azure-pipelines bot commented Oct 29, 2025

Uh oh!

mssonicbld commented Oct 30, 2025

Uh oh!

azure-pipelines bot commented Oct 30, 2025

Uh oh!

mssonicbld commented Oct 30, 2025

Uh oh!

azure-pipelines bot commented Oct 30, 2025

Uh oh!

StormLiangMS left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

xwjiang-ms commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

mssonicbld commented Oct 29, 2025

Uh oh!

azure-pipelines bot commented Oct 29, 2025

Uh oh!

mssonicbld commented Oct 30, 2025

Uh oh!

azure-pipelines bot commented Oct 30, 2025

Uh oh!

mssonicbld commented Oct 30, 2025

Uh oh!

azure-pipelines bot commented Oct 30, 2025

Uh oh!

StormLiangMS left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

xwjiang-ms commented Oct 29, 2025 •

edited

Loading