.github: Enable SSH to linux runners by seemethere · Pull Request #62280 · pytorch/pytorch

seemethere · 2021-07-27T18:19:08Z

Stack from ghstack:

-> .github: Enable SSH to linux runners #62280

Enables SSH to linux GHA runners for FB employees while on the FB VPN

SSH keys will be added to runners when the label "with-ssh" is applied to
your pull request.

Depends on https://github.com/fairinternal/pytorch-gha-infra/pull/8

Signed-off-by: Eli Uriegas eliuriegas@fb.com

Closes https://github.com/fairinternal/pytorch-gha-infra/issues/5

Differential Revision: D29941681

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> [ghstack-poisoned]

facebook-github-bot · 2021-07-27T18:19:13Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/62280
📄 Preview docs built from this PR

💊 CI failures summary and remediations

As of commit e70b4c1 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

seemethere · 2021-07-27T18:26:36Z

@seemethere has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Depnds on meta-pytorch/pytorch-gha-infra#8 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Differential Revision: [D29941681](https://our.internmc.facebook.com/intern/diff/D29941681) [ghstack-poisoned]

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> ghstack-source-id: 2ae8856 Pull Request resolved: #62280 Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Depnds on meta-pytorch/pytorch-gha-infra#8 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Differential Revision: [D29941681](https://our.internmc.facebook.com/intern/diff/D29941681) [ghstack-poisoned]

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> ghstack-source-id: f3c2f14 Pull Request resolved: #62280 Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Depnds on meta-pytorch/pytorch-gha-infra#8 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Differential Revision: [D29941681](https://our.internmc.facebook.com/intern/diff/D29941681) [ghstack-poisoned]

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> ghstack-source-id: 6cc87ae Pull Request resolved: #62280 Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

seemethere · 2021-08-03T17:00:19Z

@seemethere has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

janeyx99 · 2021-08-03T18:34:08Z

.github/templates/linux_ci_workflow.yml.j2

+        # Always hold for active ssh sessions
+        if: always()
+        run: |
+          echo "Holding runner for 2 hours until all ssh sessions have logged out"


This means that even if the ssh session is on, the runner will still shut down after 2 hours regardless, right?

Yes, this is consistent with CircleCI's current re-run with ssh functionality, see: https://circleci.com/docs/2.0/ssh-access-jobs/

samestep · 2021-08-03T18:48:44Z

.github/templates/linux_ci_workflow.yml.j2

+      - name: Hold runner for 2 hours or until ssh sessions have drained
+        # Always hold for active ssh sessions
+        if: always()
+        run: |
+          echo "Holding runner for 2 hours until all ssh sessions have logged out"
+          for _ in $(seq 1440); do
+            # Break if no ssh session exists anymore
+            if [[ "$(who)" = "" ]]; then
+              break
+            fi
+            echo "."
+            sleep 5
+          done


looks like this same run script is used in a few different places; would it make sense to put it into .github/scripts and just call it from here?

samestep

lgtm! I guess most of the logic is in https://github.com/seemethere/add-github-ssh-key?

seemethere · 2021-08-03T19:49:04Z

lgtm! I guess most of the logic is in seemethere/add-github-ssh-key?

Yup most of the logic for this particular feature is in the terraform for the security groups as well as the the github actions to add the ssh keys.

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Depends on meta-pytorch/pytorch-gha-infra#8 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Closes meta-pytorch/pytorch-gha-infra#5 Differential Revision: [D29941681](https://our.internmc.facebook.com/intern/diff/D29941681) [ghstack-poisoned]

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> ghstack-source-id: 203f3f6 Pull Request resolved: #62280 Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

seemethere · 2021-08-03T20:00:12Z

@seemethere has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Depends on meta-pytorch/pytorch-gha-infra#8 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Closes meta-pytorch/pytorch-gha-infra#5 Differential Revision: [D29941681](https://our.internmc.facebook.com/intern/diff/D29941681) [ghstack-poisoned]

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> ghstack-source-id: 6ee7575 Pull Request resolved: #62280 Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

seemethere · 2021-08-03T21:06:08Z

@seemethere has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2021-08-03T22:17:10Z

@seemethere merged this pull request in c48dfe0.

.github: Enable SSH to linux runners

5812a53

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> [ghstack-poisoned]

seemethere requested review from driazati, janeyx99 and zhouzhuojie as code owners July 27, 2021 18:19

facebook-github-bot added the cla signed label Jul 27, 2021

seemethere marked this pull request as draft July 27, 2021 18:27

seemethere added the module: ci Related to continuous integration label Jul 27, 2021

seemethere added the with-ssh label Jul 29, 2021

seemethere marked this pull request as ready for review August 3, 2021 16:59

seemethere requested a review from a team August 3, 2021 17:02

janeyx99 reviewed Aug 3, 2021

View reviewed changes

samestep reviewed Aug 3, 2021

View reviewed changes

samestep approved these changes Aug 3, 2021

View reviewed changes

janeyx99 approved these changes Aug 3, 2021

View reviewed changes

facebook-github-bot closed this in c48dfe0 Aug 3, 2021

facebook-github-bot added the Merged label Aug 3, 2021

facebook-github-bot deleted the gh/seemethere/121/head branch August 7, 2021 14:17

seemethere mentioned this pull request Aug 24, 2021

[RFC] Using with-ssh for Github Actions CI (Feedback / Meta issue) #63889

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

.github: Enable SSH to linux runners#62280

.github: Enable SSH to linux runners#62280
seemethere wants to merge 7 commits intogh/seemethere/121/basefrom
gh/seemethere/121/head

seemethere commented Jul 27, 2021 •

edited

Loading

Uh oh!

facebook-github-bot commented Jul 27, 2021 •

edited

Loading

Uh oh!

seemethere commented Jul 27, 2021

Uh oh!

seemethere commented Aug 3, 2021

Uh oh!

janeyx99 Aug 3, 2021

Uh oh!

seemethere Aug 3, 2021

Uh oh!

samestep Aug 3, 2021

Uh oh!

seemethere Aug 3, 2021

Uh oh!

samestep left a comment

Uh oh!

seemethere commented Aug 3, 2021

Uh oh!

seemethere commented Aug 3, 2021

Uh oh!

seemethere commented Aug 3, 2021

Uh oh!

facebook-github-bot commented Aug 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

seemethere commented Jul 27, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Jul 27, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

Uh oh!

seemethere commented Jul 27, 2021

Uh oh!

seemethere commented Aug 3, 2021

Uh oh!

janeyx99 Aug 3, 2021

Choose a reason for hiding this comment

Uh oh!

seemethere Aug 3, 2021

Choose a reason for hiding this comment

Uh oh!

samestep Aug 3, 2021

Choose a reason for hiding this comment

Uh oh!

seemethere Aug 3, 2021

Choose a reason for hiding this comment

Uh oh!

samestep left a comment

Choose a reason for hiding this comment

Uh oh!

seemethere commented Aug 3, 2021

Uh oh!

seemethere commented Aug 3, 2021

Uh oh!

seemethere commented Aug 3, 2021

Uh oh!

facebook-github-bot commented Aug 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

seemethere commented Jul 27, 2021 •

edited

Loading

facebook-github-bot commented Jul 27, 2021 •

edited

Loading