Skip to content

Fix symlinks not being searchable#46344

Closed
auwi-nordic wants to merge 3 commits intozed-industries:mainfrom
auwi-nordic:fix/not_searching_symlinks-auwi
Closed

Fix symlinks not being searchable#46344
auwi-nordic wants to merge 3 commits intozed-industries:mainfrom
auwi-nordic:fix/not_searching_symlinks-auwi

Conversation

@auwi-nordic
Copy link
Copy Markdown

@auwi-nordic auwi-nordic commented Jan 8, 2026

Closes #41887

Release Notes:

  • Fixed symlinks not being searchable

@cla-bot cla-bot bot added the cla-signed The user has signed the Contributor License Agreement label Jan 8, 2026
@SomeoneToIgnore

This comment was marked as outdated.


- Description: When to scan content of linked directories.
- Setting: `scan_symlinks`
- Default: `never`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default should be expanded according to related comments in Zed's code. The fact that main branch never indexes it a bug I think.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it depends on how you interpret the comment. Whether "scanning" and "including in searches" mean the same or not.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the code closer I guess they're not the same. But using default expanded does make sense I think. Could you weigh in @probably-neb

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One could argue the default should be "always".. it's the only one which doesn't cause confusing behavior that users might think is a bug. It has bad performance characteristics for some projects, but that may cause users to look for settings that change the behavior, and hopefully they'd find this setting.

It may be best to start with "never" to reduce the chance of introducing regressions with default settings in the initial release.
Then switching to expanded or always could be a later update

If the performance of the "always" case is improved, perhaps the setting could be removed entirely. Configuring file_scan_exclusions settings for the project can be used to ignore symlinked directories with undesired content.

@zed-industries-bot
Copy link
Copy Markdown
Contributor

Warnings
⚠️

This PR is missing release notes.

Please add a "Release Notes" section that describes the change:

Release Notes:

- Added/Fixed/Improved ...

If your change is not user-facing, you can use "N/A" for the entry:

Release Notes:

- N/A

Generated by 🚫 dangerJS against 72fe473

Copy link
Copy Markdown
Contributor

@AndreRoelofs AndreRoelofs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR looks good. Great job!

@auwi-nordic auwi-nordic force-pushed the fix/not_searching_symlinks-auwi branch from 72fe473 to bf8267e Compare January 8, 2026 12:11
@auwi-nordic
Copy link
Copy Markdown
Author

FYI, there was a regression of the functionality when I rebased from main and resolved some conflicts. I fixed the issue and did a force push, but maybe it should have been a new commit.. didn't notice you'd reviewed it already 😬

This bit was added

index f28d238d6e..5f1d9729ea 100644
--- i/crates/worktree/src/worktree.rs
+++ w/crates/worktree/src/worktree.rs
@@ -4331,6 +4331,28 @@ impl BackgroundScanner {
             let mut state = self.state.lock().await;
             let root_path = state.snapshot.abs_path.clone();
             for path in paths {
+                // Check if the path itself is a directory that should be scanned
+                if let Some(entry) = state.snapshot.entry_for_path(path) {
+                    if entry.is_dir() && entry.kind != EntryKind::UnloadedDir {
+                        // This is an already-loaded directory being explicitly refreshed.
+                        // Rescan it if it should be scanned according to the current settings.
+                        if state.should_scan_directory(&entry, self.settings.scan_symlinks, None) {
+                            let abs_path = root_path.join(path.as_std_path());
+                            state
+                                .enqueue_scan_dir(
+                                    abs_path.into(),
+                                    &entry,
+                                    &scan_job_tx,
+                                    self.fs.as_ref(),
+                                )
+                                .await;
+                            state.paths_to_scan.insert(path.clone());
+                            continue;
+                        }
+                    }
+                }
+
+                // Check ancestors for unloaded directories

@AndreRoelofs
Copy link
Copy Markdown
Contributor

Ok np. I will compile the branch tomorrow morning locally and use it for a bit to see if any issues are present.

@AndreRoelofs
Copy link
Copy Markdown
Contributor

The current PR crashed my pc. I know which line of code it was from looking at the PR yesterday. I will confirm and try to fix it locally but shouldn't be too hard. I will spend more time on this over the weekend.

@AndreRoelofs
Copy link
Copy Markdown
Contributor

AndreRoelofs commented Jan 12, 2026

Ok I am stumped. When I compiled your pr for release and ran it, my computer crashed due to rust-analyzer-2026-01-05 memory skyrocketing almost instantly to 40GB+ when I opened it. That happened when I opened the project that I am working on which is Eurora

The issue is that this behavior is not present in other repositories. It was also not present in projects that have a very similar structure to mine. In fact, when I pulled my own project again in a clean folder and got it up and running - the issues seemed to not be present either. The original project skyrocketed even after cargo clean and removing rust analyzer cache.

The fix came after removing all of the node_modules from root and child folders. Which is weird because rust-analyzer shouldn't bother with those anyway.

I guess we need more people to test it. I might've run into a rare rust-analyzer bug.

@auwi-nordic
Copy link
Copy Markdown
Author

By the way, what's the expected size of the rust build cache (or target directory) for the Zed project? I did notice that my project folder was taking up 100-200GB which seems insane. But I did read that Rust generates a lot of metadata when compiling, so wasn't sure if it's normal or not. I suppose I can try compiling from main and see what happens.

I was thinking of going over the code again to improve quality when I find some more time. But something like a rare rust-analyzer bug could be beyond me for now.

I'm using a build from this branch along with a couple of other fixes as my daily driver now, so at least I hope I'll discover any serious issues with the code from a user perspective.

@AndreRoelofs
Copy link
Copy Markdown
Contributor

Yes unfortunately it's normal. My Zed takes up 170 GB. From what I experienced - Rust inflates target dir with time. So I had 300 GB target directories after 6 months of work in a relatively small Rust repo with under 100k lines of Rust code. It's partly accumulation and partly artifacts for incremental builds. Especially the way me and you are using it, causes Rust to compile separate debug, test, bench and release targets.

I also experienced some other issues with general responsiveness of Zed but that might be due to the changes I made on top of yours. I will compile your PR without my changes and will use it for a bit, since I know how to solve the RAM bug now.

@AndreRoelofs
Copy link
Copy Markdown
Contributor

AndreRoelofs commented Jan 13, 2026

While I could manage the issues with memory by removing the cached folders, there are still issues. Zed is noticeably less responsive and takes much more CPU. The file explorer panel is also flashing between the actual project directory and only a handful of files from it like you can see here
image

The rust-analyzer bug I mentioned is very likely related to these changes as the code that interacts with it likely uses the affected functions.

@auwi-nordic
Copy link
Copy Markdown
Author

auwi-nordic commented Jan 13, 2026

Which scan_symlinks setting are you using there?

I tried to check out Eurora, ran pnpm install and pnpm build and tried to explore and search the workspace a bit. But didn't notice any issues. Hard to say if it uses more CPU or not, but certainly not significantly more.

I'm only able to test on MacOS right now though. Although in the remote case I'm running the remote side on Linux.

The branch I'm using to test right now has a few other fixes, but not sure if that's relevant

main...auwi-nordic:zed:auwi-combined

@AndreRoelofs
Copy link
Copy Markdown
Contributor

AndreRoelofs commented Jan 13, 2026

I am using Never, I have not changed the settings yet myself so they should be default.

For Eurora I think you need to run the following commands so that our environments are the same pnpm install pnpm proto:typescript pnpm build cargo build. You might also need a protoc binary if cargo build fails but I don't think you do.

I am on Linux Ubuntu. I usually test release compilation of Zed, not just cargo run from root. The editor currently freezes and flashes every 5-10 seconds between normal project files and a small subset, eventually freezing completely on the small subset. Are you perhaps testing on cargo run?

I also made a small refactor of the scan directory. It shouldn't affect the functionality, just made it so it tries to calculate the final boolean instead of returning early, here.

@AndreRoelofs
Copy link
Copy Markdown
Contributor

AndreRoelofs commented Jan 20, 2026

The reason why I have stopped responding is because I experienced memory issues with even my fork which merely started indexing the external files. So I decided to switch to the official Zed distribution. However now I am experiencing similar issues even on the release version. I think it might be due to my project growing or some bug in rust-analyzer itself.

With that we can summarize the issues with the latest changes:

  • Significantly increased rust-analyzer memory usage under certain circumstances.

The problem is that I don't know under which circumstances. I have not been able to reproduce it on command. But launching it in Eurora project where I work every day does cause insane memory spikes. But not on fresh installations of Eurora repo. I will spend the next weekend debugging the difference between the old and new repo of Eurora in an attempt to diagnose the actual issue.

@maxdeviant maxdeviant changed the title Fix/not searching symlinks auwi Fix symlinks not being searchable Feb 19, 2026
@gr1mpatr0n
Copy link
Copy Markdown

@auwi-nordic @AndreRoelofs Any updates on this since January?

@AndreRoelofs
Copy link
Copy Markdown
Contributor

AndreRoelofs commented Mar 12, 2026

@auwi-nordic @AndreRoelofs Any updates on this since January?

Currently there is a bug in Zed related to how it handles the symlinks indexing. I think somewhere in the code or in a git commit there are details regarding how it should work. Namely only the expanded symlinks are indexed.

The PR I initially made did just that, and fixed the bug where the symlinks were never indexed, no matter what one did. That PR is a subset of this one and is stable, especially with recent Zed changes that removed some memory leaks.

This PR is much more extensive and adds various strategies for indexing symlinks. I forked this and synced to official main branch here. However this PR is not stable. I can take a look if @auwi-nordic is fine with it and fix the issues, or we could work together on this.

The current issues in this pr, compiled locally on Ubuntu 24.04:

  1. Project explorer occasionally flickers between the project files and a small subset of them.
  2. Significant performance regression in tree exploration for default strategy, despite only 20 single small text files being symlinked.
  3. Selecting a file in project file explorer sometimes results in no-op, requiring mutliple actions.
  4. ctrl + p -> file name -> enter; results in occasional no-op
  5. Freezes during editing of files

The previously reported memory issues were not directly related to the changes in either my or this pr but were previously present in Zed. Causing me to restart my workspace every few hours as RAM usage kept steadily increasing even on the official Zed binary. I believe our changes supercharged the previous issues with rust-analyzer Zed integration. Luckily all the related issues seem to now be fixed and I do not see infinite memory spiking.

What are your thoughts @auwi-nordic?

@auwi-nordic
Copy link
Copy Markdown
Author

@AndreRoelofs Yeah, I haven't found time to continue work on improving quality of the PR. I was fairly happy with the specification of the settings, and the test, but the core of the implementation may need some improvements.

Would be great if you could take a look.

I'll try to set aside some time to work on my Zed PRs in a week or two, but can't promise anything.

@AndreRoelofs
Copy link
Copy Markdown
Contributor

@auwi-nordic I plan on looking through it shortly. Do you have any pointers for me as to causes of the issues mentioned?

@SomeoneToIgnore
Copy link
Copy Markdown
Contributor

Given the recent discussions and

image

will close this for now.

probably-neb pushed a commit that referenced this pull request Mar 31, 2026
Closes #41887

This is a stable subset of other efforts to make symlink flows more
intuitive like #46344

Release Notes:

- Fixed expanded symlinks not being searchable
@auwi-nordic auwi-nordic mentioned this pull request Apr 10, 2026
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed The user has signed the Contributor License Agreement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Project search ignores files inside symlinked directories

6 participants