Skip to content

Fix: Add depth limit and error handling for large directories#6116

Open
nipahutgardens wants to merge 1 commit into
Zettlr:developfrom
nipahutgardens:fix/large-directory-handling
Open

Fix: Add depth limit and error handling for large directories#6116
nipahutgardens wants to merge 1 commit into
Zettlr:developfrom
nipahutgardens:fix/large-directory-handling

Conversation

@nipahutgardens

@nipahutgardens nipahutgardens commented Jan 8, 2026

Copy link
Copy Markdown
Contributor
  • Added maxDepth parameter (default: 15 levels) to prevent stack overflow
  • Added currentDepth tracking for recursive directory scanning
  • Handle EBUSY/EPERM/EACCES errors gracefully on network drives
  • Log warnings instead of crashing when encountering locked/inaccessible directories
  • Fixes crashes when scanning large directory trees (10,000+ files)
  • Tested on network drives (Z:) and cloud storage (OneDrive, SyncThing)

This prevents the 'Maximum call stack size exceeded' error and EBUSY resource locks that occur when indexing deeply nested directories on network drives.

Description

Problem

Zettlr crashes when scanning large directory trees (10,000+ files) on network drives and cloud storage, with two main errors:

  1. Maximum call stack size exceeded - Stack overflow from unbounded recursion
  2. EBUSY: resource busy or locked - Errors on network drives and cloud sync folders

This affects users with large Zettelkasten collections on network shares, OneDrive, SyncThing, Dropbox, etc.

Solution

This PR adds:

  1. Depth limit - Prevents stack overflow by limiting recursion depth (default: 15 levels, configurable)
  2. Error handling - Gracefully handles EBUSY/EPERM/EACCES errors common on network drives
  3. Logging - Warns when max depth is reached or directories can't be accessed, instead of crashing

Changes

  • Modified readDirectoryRecursively() in source/app/service-providers/fsal/index.ts
  • Added maxDepth and currentDepth parameters
  • Added try-catch for file system errors
  • Added warning logs for problematic directories

Testing

Tested on Windows with:

  • 10,000+ files on network drive (Z:)
  • OneDrive and SyncThing folders
  • Deeply nested directory structures (20+ levels)

Before: 68+ second indexing, stack overflow crash
After: ~60ms indexing, no crashes, graceful degradation

Breaking Changes

None - fully backward compatible. New parameters have default values.

Additional Context

This fix was developed and tested by a user experiencing crashes with large directory structures on network drives. Ready for testing by other users with similar setups.

Tested on: Windows 11 Pro

@boring-cyborg

boring-cyborg Bot commented Jan 8, 2026

Copy link
Copy Markdown

Thank you for opening your first PR! 🎉 We would like to thank you already for your contribution. If everything checks out, we'll make sure to review the PR as soon as possible and give feedback. Please note a few things:

  • For every PR, the automated pipeline will run the linter and unit tests on
    your code. Should these fail, we will not merge your PR.
  • Activate your notifications and check your inbox regularly to respond to our
    reviews and/or comments.

We are glad for every improvement that makes its way into the app, so we look forward to collaborating on merging this PR.

@nipahutgardens
Fix: Add depth limit and error handling for large directories
640a134 
- Added maxDepth parameter (default: 15 levels) to prevent stack overflow
- Added currentDepth tracking for recursive directory scanning
- Handle EBUSY/EPERM/EACCES errors gracefully on network drives
- Log warnings instead of crashing when encountering locked/inaccessible directories
- Fixes crashes when scanning large directory trees (10,000+ files)
- Tested on network drives (Z:) and cloud storage (OneDrive, SyncThing)

This prevents the 'Maximum call stack size exceeded' error and EBUSY resource
locks that occur when indexing deeply nested directories on network drives.
@nipahutgardens nipahutgardens force-pushed the fix/large-directory-handling branch from b1cc3e9 to 640a134 Compare January 8, 2026 19:32
@nathanlesage

nathanlesage commented Jan 12, 2026

Copy link
Copy Markdown
Member

Noted this PR, I'll have to think a bit more about this. From the outset, everything looks great! I just want to ensure we don't miss anything, and there are a few decisions in there. But I expect this to be able to be merged in the next days.

@benniekiss

benniekiss commented Feb 4, 2026

Copy link
Copy Markdown
Collaborator

Would using the fs.readdir recursive: true option address the stack overflow? This would remove the need for the function recursion. I've tested locally, and it seems to provide the same functionality. This is a relatively recent option, which is likely why it's not currently used. I am not really a fan of the hard-coded depth limit, otherwise.

@benniekiss

benniekiss commented Feb 18, 2026

Copy link
Copy Markdown
Collaborator

Hi @nipahutgardens, I took the liberty of implementing some of your changes in a different PR, and I was wondering if you still experienced the recursion depth issues and crashes. I believe the changes should be in the latest nightly version, if you would be willing to test it.

@benniekiss benniekiss mentioned this pull request Mar 8, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nipahutgardens @nathanlesage @benniekiss