[Reclaim buffer] Common infrastructure update for reclaiming buffer by stephenxs · Pull Request #9133 · sonic-net/sonic-buildimage

stephenxs · 2021-11-01T08:36:46Z

Why I did it

This is to update the common sonic-buildimage infra for reclaiming buffer.

How I did it

Render zero_profiles.j2 to zero_profiles.json for vendors that support reclaiming buffer
The zero profiles will be referenced in PR [Reclaim buffer] Reclaim unused buffers by applying zero buffer profiles #8768 on Mellanox platforms and there will be test cases to verify the behavior there.
Rendering is done here for passing azure pipeline.
Load zero_profiles.json when the dynamic buffer manager starts
Generate inactive port list to reclaim buffer

It depends on PR sonic-net/sonic-swss#1996 and advancing submodule #9288

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

A picture of a cute animal (not mandatory but encouraged)

stephenxs · 2021-11-10T11:12:06Z

/azpw run

mssonicbld · 2021-11-10T11:12:08Z

/AzurePipelines run

azure-pipelines · 2021-11-10T11:12:13Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

stephenxs · 2021-11-12T01:14:57Z

/azpw run

mssonicbld · 2021-11-12T01:14:59Z

/AzurePipelines run

azure-pipelines · 2021-11-12T01:15:04Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

stephenxs · 2021-11-15T02:02:39Z

/azpw run vs

mssonicbld · 2021-11-15T02:03:09Z

/AzurePipelines run vs

azure-pipelines · 2021-11-15T02:03:14Z

No pipelines are associated with this pull request.

stephenxs · 2021-11-15T02:03:44Z

/azpw run Azure.sonic-buildimage

mssonicbld · 2021-11-15T02:04:42Z

/AzurePipelines run Azure.sonic-buildimage

azure-pipelines · 2021-11-15T02:04:54Z

Azure Pipelines successfully started running 1 pipeline(s).

….sh accordingly Signed-off-by: Stephen Sun <stephens@nvidia.com>

Signed-off-by: Stephen Sun <stephens@nvidia.com>

stephenxs · 2021-11-17T00:27:06Z

More than one PRs of mine failed due to the following error.
Looks like it's a community issue.
@lguohan @qiluo-msft @yxieca @prsunny is there anyone from the community who can help? thanks

2021-11-16T14:04:36.8871692Z =================================== FAILURES ===================================
2021-11-16T14:04:36.8872321Z ___________________ Test_SonicYang.test_validate_yang_models ___________________
2021-11-16T14:04:36.8872612Z 
2021-11-16T14:04:36.8873051Z self = <test_sonic_yang.Test_SonicYang object at 0x7fdeaa38c790>
2021-11-16T14:04:36.8874476Z sonic_yang_data = {'syc': <sonic_yang.SonicYang object at 0x7fdeaadf3ac0>, 'test_file': '../sonic-yang-models/tests/files/sample_config_db.json', 'yang_dir': '../sonic-yang-models/yang-models/'}
2021-11-16T14:04:36.8875089Z 
2021-11-16T14:04:36.8875578Z     def test_validate_yang_models(self, sonic_yang_data):
2021-11-16T14:04:36.8876463Z         '''
2021-11-16T14:04:36.8876915Z         In this test, we validate yang models
2021-11-16T14:04:36.8877439Z         a.) by converting the config as per RFC 7951 using YANG Models,
2021-11-16T14:04:36.8923505Z         b.) by creating data tree using new YANG models and
2021-11-16T14:04:36.8924186Z         c.) by validating config against YANG models.
2021-11-16T14:04:36.8924913Z         Successful execution of these steps can be treated as
2021-11-16T14:04:36.8925512Z         validation of new Yang models.
2021-11-16T14:04:36.8926839Z         '''
2021-11-16T14:04:36.8927520Z         test_file = sonic_yang_data['test_file']
2021-11-16T14:04:36.8928262Z         syc = sonic_yang_data['syc']
2021-11-16T14:04:36.8928740Z         # Currently only 3 YANG files are not directly related to config
2021-11-16T14:04:36.8929758Z         # which are: sonic-extension.yang, sonic-types.yang and sonic-bgp-common.yang. Hard coding
2021-11-16T14:04:36.8930312Z         # it right now.
2021-11-16T14:04:36.8930810Z         # If any more such helper yang files are added, we need to update here.
2021-11-16T14:04:36.8931320Z         NON_CONFIG_YANG_FILES = 3
2021-11-16T14:04:36.8931712Z         # read config
2021-11-16T14:04:36.8932615Z         jIn = self.readIjsonInput(test_file, 'SAMPLE_CONFIG_DB_JSON')
2021-11-16T14:04:36.8933477Z         jIn = json.loads(jIn)
2021-11-16T14:04:36.8933910Z         numTables = len(jIn)
2021-11-16T14:04:36.8934334Z         # load config and create Data tree
2021-11-16T14:04:36.8934769Z         syc.loadData(jIn)
2021-11-16T14:04:36.8935258Z         # check all tables are loaded and config related to all Yang Models is
2021-11-16T14:04:36.8935776Z         # loaded in Data tree.
2021-11-16T14:04:36.8936181Z >       assert len(syc.jIn) == numTables
2021-11-16T14:04:36.8936661Z E       AssertionError: assert 76 == 78
2021-11-16T14:04:36.8938415Z E        +  where 76 = len({'AAA': {'accounting': {'login': 'local'}, 'authentication': {'login': 'local'}, 'authorization': {'login': 'local'}},...nterval': '600', 'state': 'enabled'}}, 'BGP_GLOBALS': {'default': {'local_asn': '65001', 'router_id': '5.5.5.5'}}, ...})
2021-11-16T14:04:36.8940896Z E        +    where {'AAA': {'accounting': {'login': 'local'}, 'authentication': {'login': 'local'}, 'authorization': {'login': 'local'}},...nterval': '600', 'state': 'enabled'}}, 'BGP_GLOBALS': {'default': {'local_asn': '65001', 'router_id': '5.5.5.5'}}, ...} = <sonic_yang.SonicYang object at 0x7fdeaadf3ac0>.jIn
2021-11-16T14:04:36.8941820Z 
2021-11-16T14:04:36.8942558Z tests/libyang-python-tests/test_sonic_yang.py:308: AssertionError
2021-11-16T14:04:36.8943075Z ----------------------------- Captured stdout call -----------------------------
2021-11-16T14:04:36.8943401Z  Read JSON Section: SAMPLE_CONFIG_DB_JSON
2021-11-16T14:04:36.8943856Z Note: Below table(s) have no YANG models:
2021-11-16T14:04:36.8944118Z ACL_RULE, ACL_TABLE, 
2021-11-16T14:04:36.8944827Z _____________________ Test_SonicYang.test_xlate_rev_xlate ______________________
2021-11-16T14:04:36.8945029Z 
2021-11-16T14:04:36.8945317Z self = <test_sonic_yang.Test_SonicYang object at 0x7fdeaa323ca0>
2021-11-16T14:04:36.8946222Z sonic_yang_data = {'syc': <sonic_yang.SonicYang object at 0x7fdeaadf3ac0>, 'test_file': '../sonic-yang-models/tests/files/sample_config_db.json', 'yang_dir': '../sonic-yang-models/yang-models/'}
2021-11-16T14:04:36.8946653Z 
2021-11-16T14:04:36.8949843Z     def test_xlate_rev_xlate(self, sonic_yang_data):
2021-11-16T14:04:36.8950461Z         # In this test, xlation and revXlation is tested with latest Sonic
2021-11-16T14:04:36.8951334Z         # YANG model.
2021-11-16T14:04:36.8952397Z         test_file = sonic_yang_data['test_file']
2021-11-16T14:04:36.8953415Z         syc = sonic_yang_data['syc']
2021-11-16T14:04:36.8953819Z     
2021-11-16T14:04:36.8955049Z         jIn = self.readIjsonInput(test_file, 'SAMPLE_CONFIG_DB_JSON')
2021-11-16T14:04:36.8955735Z         jIn = json.loads(jIn)
2021-11-16T14:04:36.8956322Z         numTables = len(jIn)
2021-11-16T14:04:36.8956690Z     
2021-11-16T14:04:36.8957028Z         syc.loadData(jIn)
2021-11-16T14:04:36.8957866Z         # check all tables are loaded and no tables is without Yang Models
2021-11-16T14:04:36.8958555Z >       assert len(syc.jIn) == numTables
2021-11-16T14:04:36.8959043Z E       AssertionError: assert 76 == 78
2021-11-16T14:04:36.8960895Z E        +  where 76 = len({'AAA': {'accounting': {'login': 'local'}, 'authentication': {'login': 'local'}, 'authorization': {'login': 'local'}},...nterval': '600', 'state': 'enabled'}}, 'BGP_GLOBALS': {'default': {'local_asn': '65001', 'router_id': '5.5.5.5'}}, ...})
2021-11-16T14:04:36.8963078Z E        +    where {'AAA': {'accounting': {'login': 'local'}, 'authentication': {'login': 'local'}, 'authorization': {'login': 'local'}},...nterval': '600', 'state': 'enabled'}}, 'BGP_GLOBALS': {'default': {'local_asn': '65001', 'router_id': '5.5.5.5'}}, ...} = <sonic_yang.SonicYang object at 0x7fdeaadf3ac0>.jIn
2021-11-16T14:04:36.8963683Z 
2021-11-16T14:04:36.8964324Z tests/libyang-python-tests/test_sonic_yang.py:334: AssertionError
2021-11-16T14:04:36.8965025Z ----------------------------- Captured stdout call -----------------------------
2021-11-16T14:04:36.8965369Z  Read JSON Section: SAMPLE_CONFIG_DB_JSON
2021-11-16T14:04:36.8965793Z Note: Below table(s) have no YANG models:
2021-11-16T14:04:36.8966031Z ACL_RULE, ACL_TABLE, 
2021-11-16T14:04:36.8966870Z =============================== warnings summary ===============================

neethajohn · 2021-11-17T01:35:20Z

More than one PRs of mine failed due to the following error. Looks like it's a community issue. @lguohan @qiluo-msft @yxieca @prsunny is there anyone from the community who can help? thanks

2021-11-16T14:04:36.8871692Z =================================== FAILURES ===================================
2021-11-16T14:04:36.8872321Z ___________________ Test_SonicYang.test_validate_yang_models ___________________
2021-11-16T14:04:36.8872612Z 
2021-11-16T14:04:36.8873051Z self = <test_sonic_yang.Test_SonicYang object at 0x7fdeaa38c790>
2021-11-16T14:04:36.8874476Z sonic_yang_data = {'syc': <sonic_yang.SonicYang object at 0x7fdeaadf3ac0>, 'test_file': '../sonic-yang-models/tests/files/sample_config_db.json', 'yang_dir': '../sonic-yang-models/yang-models/'}
2021-11-16T14:04:36.8875089Z 
2021-11-16T14:04:36.8875578Z     def test_validate_yang_models(self, sonic_yang_data):
2021-11-16T14:04:36.8876463Z         '''
2021-11-16T14:04:36.8876915Z         In this test, we validate yang models
2021-11-16T14:04:36.8877439Z         a.) by converting the config as per RFC 7951 using YANG Models,
2021-11-16T14:04:36.8923505Z         b.) by creating data tree using new YANG models and
2021-11-16T14:04:36.8924186Z         c.) by validating config against YANG models.
2021-11-16T14:04:36.8924913Z         Successful execution of these steps can be treated as
2021-11-16T14:04:36.8925512Z         validation of new Yang models.
2021-11-16T14:04:36.8926839Z         '''
2021-11-16T14:04:36.8927520Z         test_file = sonic_yang_data['test_file']
2021-11-16T14:04:36.8928262Z         syc = sonic_yang_data['syc']
2021-11-16T14:04:36.8928740Z         # Currently only 3 YANG files are not directly related to config
2021-11-16T14:04:36.8929758Z         # which are: sonic-extension.yang, sonic-types.yang and sonic-bgp-common.yang. Hard coding
2021-11-16T14:04:36.8930312Z         # it right now.
2021-11-16T14:04:36.8930810Z         # If any more such helper yang files are added, we need to update here.
2021-11-16T14:04:36.8931320Z         NON_CONFIG_YANG_FILES = 3
2021-11-16T14:04:36.8931712Z         # read config
2021-11-16T14:04:36.8932615Z         jIn = self.readIjsonInput(test_file, 'SAMPLE_CONFIG_DB_JSON')
2021-11-16T14:04:36.8933477Z         jIn = json.loads(jIn)
2021-11-16T14:04:36.8933910Z         numTables = len(jIn)
2021-11-16T14:04:36.8934334Z         # load config and create Data tree
2021-11-16T14:04:36.8934769Z         syc.loadData(jIn)
2021-11-16T14:04:36.8935258Z         # check all tables are loaded and config related to all Yang Models is
2021-11-16T14:04:36.8935776Z         # loaded in Data tree.
2021-11-16T14:04:36.8936181Z >       assert len(syc.jIn) == numTables
2021-11-16T14:04:36.8936661Z E       AssertionError: assert 76 == 78
2021-11-16T14:04:36.8938415Z E        +  where 76 = len({'AAA': {'accounting': {'login': 'local'}, 'authentication': {'login': 'local'}, 'authorization': {'login': 'local'}},...nterval': '600', 'state': 'enabled'}}, 'BGP_GLOBALS': {'default': {'local_asn': '65001', 'router_id': '5.5.5.5'}}, ...})
2021-11-16T14:04:36.8940896Z E        +    where {'AAA': {'accounting': {'login': 'local'}, 'authentication': {'login': 'local'}, 'authorization': {'login': 'local'}},...nterval': '600', 'state': 'enabled'}}, 'BGP_GLOBALS': {'default': {'local_asn': '65001', 'router_id': '5.5.5.5'}}, ...} = <sonic_yang.SonicYang object at 0x7fdeaadf3ac0>.jIn
2021-11-16T14:04:36.8941820Z 
2021-11-16T14:04:36.8942558Z tests/libyang-python-tests/test_sonic_yang.py:308: AssertionError
2021-11-16T14:04:36.8943075Z ----------------------------- Captured stdout call -----------------------------
2021-11-16T14:04:36.8943401Z  Read JSON Section: SAMPLE_CONFIG_DB_JSON
2021-11-16T14:04:36.8943856Z Note: Below table(s) have no YANG models:
2021-11-16T14:04:36.8944118Z ACL_RULE, ACL_TABLE, 
2021-11-16T14:04:36.8944827Z _____________________ Test_SonicYang.test_xlate_rev_xlate ______________________
2021-11-16T14:04:36.8945029Z 
2021-11-16T14:04:36.8945317Z self = <test_sonic_yang.Test_SonicYang object at 0x7fdeaa323ca0>
2021-11-16T14:04:36.8946222Z sonic_yang_data = {'syc': <sonic_yang.SonicYang object at 0x7fdeaadf3ac0>, 'test_file': '../sonic-yang-models/tests/files/sample_config_db.json', 'yang_dir': '../sonic-yang-models/yang-models/'}
2021-11-16T14:04:36.8946653Z 
2021-11-16T14:04:36.8949843Z     def test_xlate_rev_xlate(self, sonic_yang_data):
2021-11-16T14:04:36.8950461Z         # In this test, xlation and revXlation is tested with latest Sonic
2021-11-16T14:04:36.8951334Z         # YANG model.
2021-11-16T14:04:36.8952397Z         test_file = sonic_yang_data['test_file']
2021-11-16T14:04:36.8953415Z         syc = sonic_yang_data['syc']
2021-11-16T14:04:36.8953819Z     
2021-11-16T14:04:36.8955049Z         jIn = self.readIjsonInput(test_file, 'SAMPLE_CONFIG_DB_JSON')
2021-11-16T14:04:36.8955735Z         jIn = json.loads(jIn)
2021-11-16T14:04:36.8956322Z         numTables = len(jIn)
2021-11-16T14:04:36.8956690Z     
2021-11-16T14:04:36.8957028Z         syc.loadData(jIn)
2021-11-16T14:04:36.8957866Z         # check all tables are loaded and no tables is without Yang Models
2021-11-16T14:04:36.8958555Z >       assert len(syc.jIn) == numTables
2021-11-16T14:04:36.8959043Z E       AssertionError: assert 76 == 78
2021-11-16T14:04:36.8960895Z E        +  where 76 = len({'AAA': {'accounting': {'login': 'local'}, 'authentication': {'login': 'local'}, 'authorization': {'login': 'local'}},...nterval': '600', 'state': 'enabled'}}, 'BGP_GLOBALS': {'default': {'local_asn': '65001', 'router_id': '5.5.5.5'}}, ...})
2021-11-16T14:04:36.8963078Z E        +    where {'AAA': {'accounting': {'login': 'local'}, 'authentication': {'login': 'local'}, 'authorization': {'login': 'local'}},...nterval': '600', 'state': 'enabled'}}, 'BGP_GLOBALS': {'default': {'local_asn': '65001', 'router_id': '5.5.5.5'}}, ...} = <sonic_yang.SonicYang object at 0x7fdeaadf3ac0>.jIn
2021-11-16T14:04:36.8963683Z 
2021-11-16T14:04:36.8964324Z tests/libyang-python-tests/test_sonic_yang.py:334: AssertionError
2021-11-16T14:04:36.8965025Z ----------------------------- Captured stdout call -----------------------------
2021-11-16T14:04:36.8965369Z  Read JSON Section: SAMPLE_CONFIG_DB_JSON
2021-11-16T14:04:36.8965793Z Note: Below table(s) have no YANG models:
2021-11-16T14:04:36.8966031Z ACL_RULE, ACL_TABLE, 
2021-11-16T14:04:36.8966870Z =============================== warnings summary ===============================

This is been looked into

neethajohn

Can you update the description to specify that #8768 will have the associated test cases for the mellanox specific template rendering? Also mention if there are merge dependencies

dockers/docker-orchagent/buffermgrd.sh

platform/vs/docker-sonic-vs/buffermgrd.sh

stephenxs · 2021-11-19T00:53:46Z

Can you update the description to specify that #8768 will have the associated test cases for the mellanox specific template rendering? Also mention if there are merge dependencies

Done.

stephenxs · 2021-11-19T10:26:15Z

/azpw run

mssonicbld · 2021-11-19T10:26:17Z

/AzurePipelines run

azure-pipelines · 2021-11-19T10:26:22Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

stephenxs · 2021-11-19T13:31:18Z

/azpw run azure.sonic-buildimage

mssonicbld · 2021-11-19T13:31:19Z

/AzurePipelines run azure.sonic-buildimage

azure-pipelines · 2021-11-19T13:31:30Z

Azure Pipelines successfully started running 1 pipeline(s).

stephenxs · 2021-11-20T08:14:21Z

/azpw run azure.sonic-buildimage

mssonicbld · 2021-11-20T08:14:23Z

/AzurePipelines run azure.sonic-buildimage

azure-pipelines · 2021-11-20T08:14:34Z

Azure Pipelines successfully started running 1 pipeline(s).

stephenxs · 2021-11-20T08:33:12Z

It can be cherry-picked to 202106 and 202012 branches based on the following commit respectively. It depends on the following PR to be merged to 202012 and 202106 before cherry picking: sonic-net/sonic-swss#2038 which is to back port sonic-net/sonic-swss#1996
Labels added.
202106

commit 1ebe52847ac1655a002fe2e3eb716b80a4400ffd
Author: shlomibitton <60430976+shlomibitton@users.noreply.github.com>
Date:   Thu Nov 18 19:40:48 2021 +0200

    [DHCPv6 relay] [202106] Fix DHCPv6 design to support multiple VLANS (#9163)

202012

commit d6ab40970998a6c1025c7fcba51c3180dcec4d3c
Author: Prince Sunny <prince.sunny@microsoft.com>
Date:   Thu Nov 18 12:48:20 2021 -0800

    [202012] td2/td3 change cpu cos num to 10 (#9311)

    Cherry-pick from #9301

…9133) - Why I did it This is to update the common sonic-buildimage infra for reclaiming buffer. - How I did it Render zero_profiles.j2 to zero_profiles.json for vendors that support reclaiming buffer The zero profiles will be referenced in PR [Reclaim buffer] Reclaim unused buffers by applying zero buffer profiles #8768 on Mellanox platforms and there will be test cases to verify the behavior there. Rendering is done here for passing azure pipeline. Load zero_profiles.json when the dynamic buffer manager starts Generate inactive port list to reclaim buffer Signed-off-by: Stephen Sun <stephens@nvidia.com>

stephenxs requested a review from lguohan as a code owner November 1, 2021 08:36

stephenxs requested a review from neethajohn November 1, 2021 08:37

stephenxs mentioned this pull request Nov 1, 2021

Reclaim reserved buffer for unused ports sonic-net/SONiC#831

Merged

keboliu previously approved these changes Nov 15, 2021

View reviewed changes

stephenxs dismissed keboliu’s stale review via 1efa718 November 16, 2021 06:18

stephenxs added 5 commits November 16, 2021 19:00

Add template for zero profiles in dynamic model and update buffermgrd…

818ff4d

….sh accordingly Signed-off-by: Stephen Sun <stephens@nvidia.com>

Update common buffer template to support zero profiles

0396e3d

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Make it possible to run vstest for zero profiles

08dbd36

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Supply zero_profile.json for vs image

7597cb9

Signed-off-by: Stephen Sun <stephens@nvidia.com>

VS image: load zero profiles for dynamic model only

8f24901

Signed-off-by: Stephen Sun <stephens@nvidia.com>

stephenxs force-pushed the zero-profiles-base branch from 1efa718 to 8f24901 Compare November 16, 2021 11:01

neethajohn reviewed Nov 17, 2021

View reviewed changes

dockers/docker-orchagent/buffermgrd.sh Show resolved Hide resolved

platform/vs/docker-sonic-vs/buffermgrd.sh Show resolved Hide resolved

stephenxs mentioned this pull request Nov 17, 2021

[Reclaim buffer] Reclaim unused buffers by applying zero buffer profiles #8768

Merged

5 tasks

neethajohn approved these changes Nov 19, 2021

View reviewed changes

stephenxs added Request for 202012 Branch Request for 202106 Branch labels Nov 20, 2021

stephenxs requested a review from keboliu November 24, 2021 09:11

keboliu approved these changes Nov 24, 2021

View reviewed changes

liat-grozovik merged commit b3ccef9 into sonic-net:master Nov 24, 2021

stephenxs deleted the zero-profiles-base branch November 24, 2021 13:17

This was referenced Nov 24, 2021

[Reclaiming buffer] Test cases for reclaiming buffer on both traditional and dynamic model sonic-net/sonic-mgmt#4561

Merged

[Reclaim buffer][202012] Reclaim unused buffer for dynamic buffer model sonic-net/sonic-swss#1985

Merged

qiluo-msft added the Included in 202012 Branch label Dec 1, 2021

judyjoseph added the Included in 202106 Branch label Dec 1, 2021

Conversation

stephenxs commented Nov 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

A picture of a cute animal (not mandatory but encouraged)

Uh oh!

stephenxs commented Nov 10, 2021

Uh oh!

mssonicbld commented Nov 10, 2021

Uh oh!

azure-pipelines bot commented Nov 10, 2021

Uh oh!

stephenxs commented Nov 12, 2021

Uh oh!

mssonicbld commented Nov 12, 2021

Uh oh!

azure-pipelines bot commented Nov 12, 2021

Uh oh!

stephenxs commented Nov 15, 2021

Uh oh!

mssonicbld commented Nov 15, 2021

Uh oh!

azure-pipelines bot commented Nov 15, 2021

Uh oh!

stephenxs commented Nov 15, 2021

Uh oh!

mssonicbld commented Nov 15, 2021

Uh oh!

azure-pipelines bot commented Nov 15, 2021

Uh oh!

stephenxs commented Nov 17, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

neethajohn commented Nov 17, 2021

Uh oh!

neethajohn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

stephenxs commented Nov 19, 2021

Uh oh!

stephenxs commented Nov 19, 2021

Uh oh!

mssonicbld commented Nov 19, 2021

Uh oh!

azure-pipelines bot commented Nov 19, 2021

Uh oh!

stephenxs commented Nov 19, 2021

Uh oh!

mssonicbld commented Nov 19, 2021

Uh oh!

azure-pipelines bot commented Nov 19, 2021

Uh oh!

stephenxs commented Nov 20, 2021

Uh oh!

mssonicbld commented Nov 20, 2021

Uh oh!

azure-pipelines bot commented Nov 20, 2021

Uh oh!

stephenxs commented Nov 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

stephenxs commented Nov 1, 2021 •

edited

Loading

stephenxs commented Nov 17, 2021 •

edited

Loading

stephenxs commented Nov 20, 2021 •

edited

Loading