Skip to content

gh cs ssh cli integration with openssh config#4915

Merged
vilmibm merged 45 commits intocli:trunkfrom
znull:znull/stdio
Jan 11, 2022
Merged

gh cs ssh cli integration with openssh config#4915
vilmibm merged 45 commits intocli:trunkfrom
znull:znull/stdio

Conversation

@znull
Copy link
Contributor

@znull znull commented Dec 15, 2021

This PR allows the creation of virtual ssh "hosts" for codespaces that look and behave like real ssh hosts at the CLI. This enables codespaces to integrate seamlessly with the rich ecosystem of existing openssh features and add-ons, like:

  • sharing multiple ssh sessions over a single connection to the codespace with ControlMaster
  • tab completion for ssh/scp/sshfs (or anything else that supports ssh completion)
  • direct git push/pull to/from the codespace using standard ssh urls

Repeated ssh to a codespace is also much faster this way.

It does this by adding two features:

  1. A gh cs ssh --stdio option that starts the remote sshd and proxies a connection to it to gh's stdin/stdout
  2. A gh cs ssh config subcommand that generates an ssh configuration for use with #1

For example, a user would set this up by doing:

$ gh cs list
NAME                               REPOSITORY      BRANCH  STATE      CREATED AT
znull-cli-cli-4q6gr7qfjjp          cli/cli         trunk   Available  4m
znull-github-brubeck-6vwrpgvfrq6x  github/brubeck  master  Available  3m

$ gh cs ssh config | tee ~/.ssh/codespaces
Host cs.znull-cli-cli-4q6gr7qfjjp.trunk
        User codespace
        ProxyCommand /home/znull/src/cli/bin/gh cs ssh -c znull-cli-cli-4q6gr7qfjjp --stdio
        UserKnownHostsFile=/dev/null
        StrictHostKeyChecking no
        LogLevel quiet
        ControlMaster auto

Host cs.znull-github-brubeck-6vwrpgvfrq6x.master
        User codespace
        ProxyCommand /home/znull/src/cli/bin/gh cs ssh -c znull-github-brubeck-6vwrpgvfrq6x --stdio
        UserKnownHostsFile=/dev/null
        StrictHostKeyChecking no
        LogLevel quiet
        ControlMaster auto

$ echo 'include ~/.ssh/codespaces' >> ~/.ssh/config

With that in place, codespaces can now be treated like ordinary ssh remote hosts at the command line:

$ ssh cs.znull-cli-cli-4q6gr7qfjjp.trunk uname -a
Linux codespaces_c9f123 5.4.0-1063-azure #66~18.04.1-Ubuntu SMP Thu Oct 21 09:59:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ scp cs.znull-cli-cli-4q6gr7qfjjp.trunk:/workspaces/cli/README.md /dev/null
README.md                                      100% 4251    25.9KB/s   00:00

Note that hostname completion for the codespace works here, as well as remote pathname completion for scp.

sshfs works as well - both sshfs itself, and shell hostname and remote path completion:

$ sshfs cs.znull-cli-cli-4q6gr7qfjjp.trunk:/workspaces/cli ~/mnt

$ ls -l ~/mnt/README.md
-rw-rw-rw- 1 znull root 4251 Dec 17 10:49 /home/znull/mnt/README.md

The codespace can also be used as a git remote, again with hostname and remote path completion:

$ git clone cs.znull-cli-cli-4q6gr7qfjjp.trunk:/workspaces/cli
Cloning into 'cli'...
remote: Enumerating objects: 31234, done.
remote: Counting objects: 100% (31234/31234), done.
remote: Compressing objects: 100% (9772/9772), done.
remote: Total 31234 (delta 20908), reused 31109 (delta 20783), pack-reused 0
Receiving objects: 100% (31234/31234), 22.87 MiB | 12.65 MiB/s, done.
Resolving deltas: 100% (20908/20908), done.

$ cd cli
$ git remote -v
origin  cs.znull-cli-cli-4q6gr7qfjjp.trunk:/workspaces/cli (fetch)
origin  cs.znull-cli-cli-4q6gr7qfjjp.trunk:/workspaces/cli (push)

Notes / Enhancements

  • The generated ssh config needs the remote codespace ssh username, and it's not possible (as far as I know) to get this without connecting to the codespace and starting sshd. So unavailable codespaces are skipped.
  • Users must manually re-run gh cs ssh config when codespaces are added/removed; this could potentially be automated.
  • I chose to generate virtual hostnames that begin with cs. to group them together for tab completion purposes, but we could use other schemes or make this configurable.

fixes #4713

@znull znull requested review from a team as code owners December 15, 2021 22:05
@znull znull requested review from vilmibm and removed request for a team December 15, 2021 22:05
@cliAutomation cliAutomation added the external pull request originating outside of the CLI core team label Dec 15, 2021
Copy link
Contributor

@mislav mislav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! The direction generally looks good, but I'm wondering about whether this overloads the gh cs ssh command too much.

@znull
Copy link
Contributor Author

znull commented Dec 16, 2021

Thanks for this! The direction generally looks good, but I'm wondering about whether this overloads the gh cs ssh command too much.

I agree. What makes sense as an alternative? gh cs ssh config perhaps? (can cobra do that, or would it have to be gh cs ssh-config? I pushed an implementation as a config subcommand, but I'm open to suggestions.

@znull znull requested a review from mislav December 16, 2021 22:32
When running `gh cs ssh config` without a `-c` option, we skip
codespaces that aren't available. This change suppresses that behavior
when a single codespace is explicitly requested, starting the codespace
if it's not running.
Copy link
Contributor

@mislav mislav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking pretty good! My final notes are mostly about error/context handling.

sshCmd.Flags().BoolVar(&opts.stdio, "stdio", false, "Proxy sshd connection to stdio")
sshCmd.Flags().MarkHidden("stdio")

sshCmd.AddCommand(newConfigCmd(app))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point about disambiguation. Let's decide on final naming at the start of January.

Copy link
Contributor

@adonovan adonovan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but @mislav should have the final say.

znull added 10 commits December 20, 2021 13:57
cmdutil.Factory.Executable() accounts for things like package managers
and symlinks to the actual executable.

An alternative to passing the *cmdutil.Factory down the stack would be
stashing the executable string in the codespace.App, which works (and
the diff is smaller), but it produced some odd non-local test failures.
This way seems less mysterious and more like other uses of Factory in
the codebase.
Also, other than that, restore the original ordering of this function
This way, factory can satisfy an interface that requires `Executable()`.
This is to avoid having to explicitly pass it to each subcommand that
needs it. Each codespaces command runs in the context of App, so that's
a point of shared context that we can store state in.
Copy link
Contributor

@mislav mislav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. Thank you for all the hard work! I have pushed the changes to iron out how Executable() is being passed around and consumed. Basically, this information is now stored on codespace.App level rather than being passed around to subcommands.

Copy link
Contributor

@vilmibm vilmibm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<3

@vilmibm vilmibm merged commit 2deb0ec into cli:trunk Jan 11, 2022
@znull znull deleted the znull/stdio branch January 11, 2022 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external pull request originating outside of the CLI core team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gh cs: add ssh --config flag

6 participants