Skip to content

Increase post-PDU state change delay to allow all PSUs to react#13793

Merged
kevinskwang merged 2 commits intosonic-net:masterfrom
spilkey-cisco:spilkey/pdu_delay
Jul 26, 2024
Merged

Increase post-PDU state change delay to allow all PSUs to react#13793
kevinskwang merged 2 commits intosonic-net:masterfrom
spilkey-cisco:spilkey/pdu_delay

Conversation

@spilkey-cisco
Copy link
Copy Markdown
Contributor

Description of PR

Increase delay between PDU state change and reading PSU details to 10 seconds (previously 5).

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 202012
  • 202205
  • 202305
  • 202311
  • 202405

Approach

What is the motivation for this PR?

10 second delay between PDU state change and accessing PSU details is not always enough. The time needed depends on PDU hardware, psud update interval, and PSU hardware. Increasing to 20 seconds allows extra time for the PDU to apply a state change and PSU to reflect it, which MSFT has validated and found sufficient.

How did you do it?

Increase post-PDU state change delays to 20 seconds (previously 10 seconds). This is especially important anywhere the PDU state is changed to "on".

How did you verify/test it?

MSFT verified by setting the delay to 20 seconds since Cisco was unable to reproduce with our PDU hardware.

@spilkey-cisco spilkey-cisco requested a review from prgeor as a code owner July 24, 2024 18:36
Copy link
Copy Markdown
Contributor

@wsycqyz wsycqyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am ok with this PR

for outlet in all_outlet_status:
if not outlet["outlet_on"]:
pdu_ctrl.turn_on_outlet(outlet)
time.sleep(5)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need to change here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was adjusted in this PR since it was adjusted in the previous PR to increase the delay. I will remove this change though since it looks like it will not have any effect on test results.

@kevinskwang kevinskwang merged commit 51db6bb into sonic-net:master Jul 26, 2024
mssonicbld pushed a commit to mssonicbld/sonic-mgmt that referenced this pull request Jul 26, 2024
…c-net#13793)

* Increase post-PDU state change delay to allow all PSUs to react

* Revert delay increase for 'turn_all_outlets_on'
mssonicbld pushed a commit to mssonicbld/sonic-mgmt that referenced this pull request Jul 26, 2024
…c-net#13793)

* Increase post-PDU state change delay to allow all PSUs to react

* Revert delay increase for 'turn_all_outlets_on'
@mssonicbld
Copy link
Copy Markdown
Collaborator

Cherry-pick PR to 202405: #13831

@mssonicbld
Copy link
Copy Markdown
Collaborator

Cherry-pick PR to 202311: #13832

mssonicbld pushed a commit that referenced this pull request Jul 26, 2024
* Increase post-PDU state change delay to allow all PSUs to react

* Revert delay increase for 'turn_all_outlets_on'
mssonicbld pushed a commit that referenced this pull request Jul 26, 2024
* Increase post-PDU state change delay to allow all PSUs to react

* Revert delay increase for 'turn_all_outlets_on'
mssonicbld pushed a commit to mssonicbld/sonic-mgmt that referenced this pull request Jul 31, 2024
…c-net#13793)

* Increase post-PDU state change delay to allow all PSUs to react

* Revert delay increase for 'turn_all_outlets_on'
@mssonicbld
Copy link
Copy Markdown
Collaborator

Cherry-pick PR to 202305: #13892

mssonicbld pushed a commit that referenced this pull request Jul 31, 2024
* Increase post-PDU state change delay to allow all PSUs to react

* Revert delay increase for 'turn_all_outlets_on'
arista-hpandya pushed a commit to arista-hpandya/sonic-mgmt that referenced this pull request Oct 2, 2024
…c-net#13793)

* Increase post-PDU state change delay to allow all PSUs to react

* Revert delay increase for 'turn_all_outlets_on'
vikshaw-Nokia pushed a commit to vikshaw-Nokia/sonic-mgmt that referenced this pull request Oct 23, 2024
…c-net#13793)

* Increase post-PDU state change delay to allow all PSUs to react

* Revert delay increase for 'turn_all_outlets_on'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants