repl: Add WSL and SSH remote kernel support#47891
repl: Add WSL and SSH remote kernel support#47891Veykril merged 45 commits intozed-industries:mainfrom
Conversation
- Implement WSL and SSH remote kernels (crates/repl/src/kernels/*) and wire up spawn/kill kernel proto messages and client requests. - fix: notebook_ui, use buffer so that notebooks open in remote/WSL settings. - fix: add musl in nix for cross-compilation, without this remote server doesn't build inside NixOS
- remove unused worktree_store - change smol timer to cx.background_executor - on_action listeners using _: &ActionType instead of &ActionType which was being interpreted as variable binding - removed useless Entity::from() in sessions.rs
| }) | ||
| .await | ||
| .log_err()?; | ||
| .await; |
There was a problem hiding this comment.
This could all be
.await
.inspect_err(|e| {
log::warn!(
"RemoteToolchainStore::list_toolchains: RPC failed for language {}: {:?}",
language_name,
e
);
})
.ok()?;| Some(envelope.payload.working_directory) | ||
| }; | ||
|
|
||
| // Spawn kernel (Assuming python for now, or we'd need to parse kernelspec logic here or pass the command) |
There was a problem hiding this comment.
True, we're probably going to have to provide the listing based on what's available on the remote as well as be able to launch using those as well.
There was a problem hiding this comment.
I was hesitant to wire up full remote kernel discovery for now, felt like it might bloat this PR too much.
Do you think we should block this until we have proper spec listing, or is the python3 fallback okay for an initial merge?
There was a problem hiding this comment.
Note that python3 won't work on windows by default as its just python there
| .map(|toolchain| { | ||
| background_executor.spawn(async move { | ||
| // For remote projects, we assume python is available assuming toolchain is reported. | ||
| // We can skip the `ipykernel` check or run it remotely. |
There was a problem hiding this comment.
We probably do still need to do some kind of ipykernel check. I think in some cases people want the same flow as VS Code where they can install ipykernel. That doesn't always work in every env (oddly even in my uv sync created venvs it has trouble).
There was a problem hiding this comment.
Should we add a prompt to install ipykernel if the selected kernel is missing it? that sort of reflects the notification workflow from VSCode.. I would want to investigate why it's not working on uv venv..
There was a problem hiding this comment.
I'm sorry I meant that VS Code runs into a problem with a fresh uv sync'ed repo and is unable to install into it. Maybe VS Code has to detect that it's a uv backed repo that needs to use uv pip install ipykernel (or uv add if intended to be kept).
| } | ||
| } | ||
| }) | ||
| .detach(); |
There was a problem hiding this comment.
This again reminds me we need .split() on zmq.rs so we don't have to create so many background processes and can send while receiving on all the channels.
There was a problem hiding this comment.
Yeah, had to fight socket ownership here, once zmq adds split this would allow us to drop multiple spawners
There was a problem hiding this comment.
I started on that here: zeromq/zmq.rs#194
Probably easier to maintain in runtimed, zmq.rs, and even Zed to just use the Tokio setup that Zed now has inside (my work predates that).
There was a problem hiding this comment.
.split() is now supported in here as of #48823 so this should nicely clean up.
Veykril
left a comment
There was a problem hiding this comment.
This does not seem to support docker and also I don't quite understand why we have to special case between different remote types here. Ideally all the specifics of ssh and wsl should be hidden away behind our RemoteConnectionOptions and such. (I have only skimmed over the ssh/wsl modules here for now, so do tell the reasons for the split here)
I am also confused by the fact that we now have a Remote variant but also an ssh and wsl remote variant each
| .log_err()?; | ||
| .inspect_err(|e| { | ||
| log::warn!( | ||
| "RemoteToolchainStore::list_toolchains: RPC failed for language {}: {:?}", | ||
| language_name, | ||
| e | ||
| ); | ||
| }) | ||
| .ok()?; |
There was a problem hiding this comment.
This should remain a log_err imo
| Some(envelope.payload.working_directory) | ||
| }; | ||
|
|
||
| // Spawn kernel (Assuming python for now, or we'd need to parse kernelspec logic here or pass the command) |
There was a problem hiding this comment.
Note that python3 won't work on windows by default as its just python there
| let child = this.update(&mut cx, |this, _| this.kernels.remove(&kernel_id)); | ||
| if let Some(mut child) = child { | ||
| child.kill().log_err(); | ||
| let _ = child.status().await; // Perhaps kill should be enough, should we wait? |
There was a problem hiding this comment.
kill should suffice, no need to wait since we don't make any use of the return value here
There was a problem hiding this comment.
Alright, I will remove this
There was a problem hiding this comment.
Handled the python3 by having both python and python3 and removed this
| KernelSpecification::SshRemote(_) => (kernelspec.name(), "SSH Remote", None), | ||
| KernelSpecification::WslRemote(_) => (kernelspec.name(), "WSL Remote", None), |
There was a problem hiding this comment.
Why are these separate from Remote?
Yeah, the naming is a bit overloaded right now. The simpler mental model I have right now is:
While we can do this under the same RemoteConnectionOptions because the underlying implementation paths are distinct enough, ( RPC vs Command::new('wsl')), I thought the separation variation would be cleaner. For the docker support, I was planning for a follow=up PR to keep this one focused to the issues on getting WSL stable first. Implementation for docker should mirror the WSL approach. I can change the KernelSpecification::Remote to JupyterServer for a clearer distinction. And to answer the
This ties back to the above discussion where because SshRemote and WslRemote are managed processes, they don't have a URL to show the user. Essentially the Renaming Remote to JupyterServer would make this more clear.. |
|
Also one more question.. when you say docker you mean the new Dev Containers support right? |
|
Yea lets rename that variant then, and yes i the dev-containers. Absolutely fine to keep that out of this PR for now though |
|
Changed the original body to not reflect the NixOS dev build fix and Jupyter View additions as they are dev and behind feature flag respectively. |
Let's incorporate JupyterHTTPServer into the naming so it's clear what it's connecting to, since they can be doing it locally or in the cloud. That way remote refers to the Zed concept of it. |
Replace manual starts_with/slicing with strip_prefix when detecting \\wsl$ and \\wsl.localhost UNC prefixes
|
Not sure why the remote.workspace error was present, we do need it when using repl over ssh. |
- Instead of relying on python and python3, using toolchain path to get the proper envrionment - update with the new repl menu in remote as well
e0114fd to
022816b
Compare
022816b to
5da69e6
Compare
Veykril
left a comment
There was a problem hiding this comment.
one more thing but other lgtm
crates/remote/src/transport/ssh.rs
Outdated
| "-o".to_string(), | ||
| "BatchMode=yes".to_string(), | ||
| "-o".to_string(), | ||
| "ConnectTimeout=10".to_string(), |
There was a problem hiding this comment.
I worry about this change, we haven't needed this until now for anything else in the ssh layer so I fear this may cause unexpected issues. Do we need this?
There was a problem hiding this comment.
hmm, I kept this as my connection was unstable, not sure if we need it. Anyhow, if the kernel hangs, we can handle timeouts at the SshRunningKernel level later. I was able to make it work without it anyways
| .map(|workspace| workspace.read(cx).project().read(cx).is_local()) | ||
| .map(|workspace| { | ||
| let project = workspace.read(cx).project().read(cx); | ||
| project.is_local() || project.is_remote() |
There was a problem hiding this comment.
This is always true as is_remote is just !is_local, I think you want !project.is_via_collab() (the methods here need some better documentation ....)
There was a problem hiding this comment.
Yeah, thank you for this! Merged laxly when repl menu was added to upstream, didn't realize this was A or not A by making this.
There was a problem hiding this comment.
will make the same changes on repl session UI as well
previous check on menu logic was A or not A, allow changing for when not via collab revert back SSH args, if kernel suffers hangs, will handle this in SshRunningKernel
Head branch was pushed to by a user without write access
|
btw @Veykril, I was going through the remoting 1.0 notes and dev-containers notes before actually starting on Dev Containers implementation. A lot of what I'll have to do will need to answer some of the open questions in those notes on how to go about this, would you be free to discuss an approach? Port handling is the core piece here and from the notes it looks like it could be some effort just to break the DevContainer CLI dependency before we can even get to the forwarding itself. I'd rather have team consensus on that before implementing anything. Could also just break it down into smaller PRs targeting the noted issues you have first and later on bring the Python kernel once the ground is solid. |
Closes #15196, #46918
Release Notes: