Fix symlinks not being searchable#46344
Fix symlinks not being searchable#46344auwi-nordic wants to merge 3 commits intozed-industries:mainfrom
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
15eaaeb to
72fe473
Compare
|
|
||
| - Description: When to scan content of linked directories. | ||
| - Setting: `scan_symlinks` | ||
| - Default: `never` |
There was a problem hiding this comment.
The default should be expanded according to related comments in Zed's code. The fact that main branch never indexes it a bug I think.
There was a problem hiding this comment.
I guess it depends on how you interpret the comment. Whether "scanning" and "including in searches" mean the same or not.
There was a problem hiding this comment.
Looking at the code closer I guess they're not the same. But using default expanded does make sense I think. Could you weigh in @probably-neb
There was a problem hiding this comment.
One could argue the default should be "always".. it's the only one which doesn't cause confusing behavior that users might think is a bug. It has bad performance characteristics for some projects, but that may cause users to look for settings that change the behavior, and hopefully they'd find this setting.
It may be best to start with "never" to reduce the chance of introducing regressions with default settings in the initial release.
Then switching to expanded or always could be a later update
If the performance of the "always" case is improved, perhaps the setting could be removed entirely. Configuring file_scan_exclusions settings for the project can be used to ignore symlinked directories with undesired content.
AndreRoelofs
left a comment
There was a problem hiding this comment.
The PR looks good. Great job!
72fe473 to
bf8267e
Compare
|
FYI, there was a regression of the functionality when I rebased from main and resolved some conflicts. I fixed the issue and did a force push, but maybe it should have been a new commit.. didn't notice you'd reviewed it already 😬 This bit was added index f28d238d6e..5f1d9729ea 100644
--- i/crates/worktree/src/worktree.rs
+++ w/crates/worktree/src/worktree.rs
@@ -4331,6 +4331,28 @@ impl BackgroundScanner {
let mut state = self.state.lock().await;
let root_path = state.snapshot.abs_path.clone();
for path in paths {
+ // Check if the path itself is a directory that should be scanned
+ if let Some(entry) = state.snapshot.entry_for_path(path) {
+ if entry.is_dir() && entry.kind != EntryKind::UnloadedDir {
+ // This is an already-loaded directory being explicitly refreshed.
+ // Rescan it if it should be scanned according to the current settings.
+ if state.should_scan_directory(&entry, self.settings.scan_symlinks, None) {
+ let abs_path = root_path.join(path.as_std_path());
+ state
+ .enqueue_scan_dir(
+ abs_path.into(),
+ &entry,
+ &scan_job_tx,
+ self.fs.as_ref(),
+ )
+ .await;
+ state.paths_to_scan.insert(path.clone());
+ continue;
+ }
+ }
+ }
+
+ // Check ancestors for unloaded directories |
|
Ok np. I will compile the branch tomorrow morning locally and use it for a bit to see if any issues are present. |
|
The current PR crashed my pc. I know which line of code it was from looking at the PR yesterday. I will confirm and try to fix it locally but shouldn't be too hard. I will spend more time on this over the weekend. |
|
Ok I am stumped. When I compiled your pr for release and ran it, my computer crashed due to rust-analyzer-2026-01-05 memory skyrocketing almost instantly to 40GB+ when I opened it. That happened when I opened the project that I am working on which is Eurora The issue is that this behavior is not present in other repositories. It was also not present in projects that have a very similar structure to mine. In fact, when I pulled my own project again in a clean folder and got it up and running - the issues seemed to not be present either. The original project skyrocketed even after cargo clean and removing rust analyzer cache. The fix came after removing all of the node_modules from root and child folders. Which is weird because rust-analyzer shouldn't bother with those anyway. I guess we need more people to test it. I might've run into a rare rust-analyzer bug. |
|
By the way, what's the expected size of the rust build cache (or target directory) for the Zed project? I did notice that my project folder was taking up 100-200GB which seems insane. But I did read that Rust generates a lot of metadata when compiling, so wasn't sure if it's normal or not. I suppose I can try compiling from main and see what happens. I was thinking of going over the code again to improve quality when I find some more time. But something like a rare rust-analyzer bug could be beyond me for now. I'm using a build from this branch along with a couple of other fixes as my daily driver now, so at least I hope I'll discover any serious issues with the code from a user perspective. |
|
Yes unfortunately it's normal. My Zed takes up 170 GB. From what I experienced - Rust inflates target dir with time. So I had 300 GB target directories after 6 months of work in a relatively small Rust repo with under 100k lines of Rust code. It's partly accumulation and partly artifacts for incremental builds. Especially the way me and you are using it, causes Rust to compile separate debug, test, bench and release targets. I also experienced some other issues with general responsiveness of Zed but that might be due to the changes I made on top of yours. I will compile your PR without my changes and will use it for a bit, since I know how to solve the RAM bug now. |
|
Which scan_symlinks setting are you using there? I tried to check out Eurora, ran I'm only able to test on MacOS right now though. Although in the remote case I'm running the remote side on Linux. The branch I'm using to test right now has a few other fixes, but not sure if that's relevant |
|
I am using Never, I have not changed the settings yet myself so they should be default. For Eurora I think you need to run the following commands so that our environments are the same I am on Linux Ubuntu. I usually test release compilation of Zed, not just I also made a small refactor of the scan directory. It shouldn't affect the functionality, just made it so it tries to calculate the final boolean instead of returning early, here. |
|
The reason why I have stopped responding is because I experienced memory issues with even my fork which merely started indexing the external files. So I decided to switch to the official Zed distribution. However now I am experiencing similar issues even on the release version. I think it might be due to my project growing or some bug in rust-analyzer itself. With that we can summarize the issues with the latest changes:
The problem is that I don't know under which circumstances. I have not been able to reproduce it on command. But launching it in Eurora project where I work every day does cause insane memory spikes. But not on fresh installations of Eurora repo. I will spend the next weekend debugging the difference between the old and new repo of Eurora in an attempt to diagnose the actual issue. |
|
@auwi-nordic @AndreRoelofs Any updates on this since January? |
Currently there is a bug in Zed related to how it handles the symlinks indexing. I think somewhere in the code or in a git commit there are details regarding how it should work. Namely only the expanded symlinks are indexed. The PR I initially made did just that, and fixed the bug where the symlinks were never indexed, no matter what one did. That PR is a subset of this one and is stable, especially with recent Zed changes that removed some memory leaks. This PR is much more extensive and adds various strategies for indexing symlinks. I forked this and synced to official main branch here. However this PR is not stable. I can take a look if @auwi-nordic is fine with it and fix the issues, or we could work together on this. The current issues in this pr, compiled locally on Ubuntu 24.04:
The previously reported memory issues were not directly related to the changes in either my or this pr but were previously present in Zed. Causing me to restart my workspace every few hours as RAM usage kept steadily increasing even on the official Zed binary. I believe our changes supercharged the previous issues with rust-analyzer Zed integration. Luckily all the related issues seem to now be fixed and I do not see infinite memory spiking. What are your thoughts @auwi-nordic? |
|
@AndreRoelofs Yeah, I haven't found time to continue work on improving quality of the PR. I was fairly happy with the specification of the settings, and the test, but the core of the implementation may need some improvements. Would be great if you could take a look. I'll try to set aside some time to work on my Zed PRs in a week or two, but can't promise anything. |
|
@auwi-nordic I plan on looking through it shortly. Do you have any pointers for me as to causes of the issues mentioned? |
|
Given the recent discussions and
will close this for now. |


Closes #41887
Release Notes: