perf: consolidate paths to parent path when threshold exceeded for fsevents watcher#60
Merged
sapphi-red merged 1 commit intoDec 19, 2025
Conversation
Merged
This was referenced Dec 19, 2025
graphite-app Bot
pushed a commit
to rolldown/rolldown
that referenced
this pull request
Dec 22, 2025
Now that I implemented some improvements (rolldown/notify#59, rolldown/notify#60), it doesn't error on large projects. kqueue doesn't have a way to watch recursively, so it's faster to use fsevent which supports recursive watches.
sapphi-red
added a commit
that referenced
this pull request
May 19, 2026
Close #62. Mirror the fsevents PR #60 pattern: when 10+ sibling paths are registered under the same parent, collapse them into a single watch on the parent instead of opening one `ReadDirectoryChangesW` handle per child. Implementation: * Replace the `WatchMode` value of `self.watches` with a new `UserWatch` struct that carries the user's intent plus the resolved OS-level primary dir (and optional tracked-parent for `TargetMode::TrackPath`). * Add `rebuild_watch_handles`: build a `ConsolidatingPathTrie` over the primary dirs from every user watch, compute the recursive flag per consolidated target by checking for strict descendants in the user watches, layer in tracked-parent watches that aren't already covered, and diff against the open `watch_handles` to converge the OS state. * `add_watch` / `remove_watch` / `apply_staged` now update `self.watches` first and then call `rebuild_watch_handles` instead of poking `add_watch_raw` directly. * The event filter at `handle_event` was a direct hash lookup on the OS dir name; consolidation can leave the OS dir different from any user request, so it now walks ancestors via `is_event_covered`. * `pre_open_tracked_parent` works around a Windows quirk where calling `path.metadata()` on a directory *before* its parent dir has a `ReadDirectoryChangesW` handle silently drops the parent's later `FILE_ACTION_MODIFIED` event for that directory. The pre-call opens the tracked parent before any metadata is read, matching the order the previous per-call code used. Adds five new tests covering: 10-sibling consolidation, event delivery through a consolidated parent, mixed recursive/non-recursive collapse, de-consolidation when the count drops back below the threshold, and the batched `paths_mut().commit()` path.
sapphi-red
added a commit
that referenced
this pull request
May 19, 2026
Close #62. Mirror the fsevents PR #60 pattern: when 10+ sibling paths are registered under the same parent, collapse them into a single watch on the parent instead of opening one `ReadDirectoryChangesW` handle per child. Implementation: * Replace the `WatchMode` value of `self.watches` with a new `UserWatch` struct that carries the user's intent plus the resolved OS-level primary dir (and optional tracked-parent for `TargetMode::TrackPath`). * Add `rebuild_watch_handles`: build a `ConsolidatingPathTrie` over the primary dirs from every user watch, compute the recursive flag per consolidated target by checking for strict descendants in the user watches, layer in tracked-parent watches that aren't already covered, and diff against the open `watch_handles` to converge the OS state. * `add_watch` / `remove_watch` / `apply_staged` now update `self.watches` first and then call `rebuild_watch_handles` instead of poking `add_watch_raw` directly. * The event filter at `handle_event` was a direct hash lookup on the OS dir name; consolidation can leave the OS dir different from any user request, so it now walks ancestors via `is_event_covered`. * `pre_open_tracked_parent` works around a Windows quirk where calling `path.metadata()` on a directory *before* its parent dir has a `ReadDirectoryChangesW` handle silently drops the parent's later `FILE_ACTION_MODIFIED` event for that directory. The pre-call opens the tracked parent before any metadata is read, matching the order the previous per-call code used. Adds five new tests covering: 10-sibling consolidation, event delivery through a consolidated parent, mixed recursive/non-recursive collapse, de-consolidation when the count drops back below the threshold, and the batched `paths_mut().commit()` path.
sapphi-red
added a commit
that referenced
this pull request
May 19, 2026
Close #62. Mirror the fsevents PR #60 pattern: when 10+ sibling paths are registered under the same parent, collapse them into a single watch on the parent instead of opening one `ReadDirectoryChangesW` handle per child. Implementation: * Replace the `WatchMode` value of `self.watches` with a new `UserWatch` struct that carries the user's intent plus the resolved OS-level primary dir (and optional tracked-parent for `TargetMode::TrackPath`). * Add `rebuild_watch_handles`: build a `ConsolidatingPathTrie` over the primary dirs from every user watch, compute the recursive flag per consolidated target by checking for strict descendants in the user watches, layer in tracked-parent watches that aren't already covered, and diff against the open `watch_handles` to converge the OS state. * `add_watch` / `remove_watch` / `apply_staged` now update `self.watches` first and then call `rebuild_watch_handles` instead of poking `add_watch_raw` directly. * The event filter at `handle_event` was a direct hash lookup on the OS dir name; consolidation can leave the OS dir different from any user request, so it now walks ancestors via `is_event_covered`. * `pre_open_tracked_parent` works around a Windows quirk where calling `path.metadata()` on a directory *before* its parent dir has a `ReadDirectoryChangesW` handle silently drops the parent's later `FILE_ACTION_MODIFIED` event for that directory. The pre-call opens the tracked parent before any metadata is read, matching the order the previous per-call code used. Adds five new tests covering: 10-sibling consolidation, event delivery through a consolidated parent, mixed recursive/non-recursive collapse, de-consolidation when the count drops back below the threshold, and the batched `paths_mut().commit()` path.
sapphi-red
added a commit
that referenced
this pull request
May 20, 2026
…93) ### Summary On Windows, registering many sibling paths under the same parent previously opened one `ReadDirectoryChangesW` handle per child. This PR mirrors the fsevents consolidation pattern from #60: when 10 or more sibling paths are registered under the same parent, they collapse into a single watch on the parent instead. closes #62 ### How it works * `UserWatch` struct. `self.watches` now stores the user's intent plus the resolved OS-level primary dir (and optional tracked-parent for `TargetMode::TrackPath`), instead of a bare `WatchMode`. * `rebuild_watch_handles`. Builds a `ConsolidatingPathTrie` over the primary dirs of all user watches, computes the recursive flag per consolidated target by checking for strict descendants, layers in tracked-parent watches, and diffs against the open handles to converge OS state. `add_watch`, `remove_watch`, and `apply_staged` now update `self.watches` and call this, rather than poking `add_watch_raw` directly. * `is_event_covered`. Since consolidation can leave the OS dir different from any user request, event filtering now walks ancestors instead of doing a direct hash lookup. * Consolidated pre-open. `apply_staged` no longer does a per-path `pre_open_tracked_parent`, which caused N opens that consolidation usually threw away. A single `pre_open_consolidated` pass runs the same consolidation trie over staged parents and opens the consolidated set directly, so the later `rebuild_watch_handles` usually finds handles already in place, with no close-and-reopen. This preserves the Windows metadata quirk fix: a directory's `metadata()` must run after its parent is watched, or the parent's later `FILE_ACTION_MODIFIED` is silently dropped. ### Benchmarks A new `windows_sibling_subdirs` workload watches N sibling directories under a shared parent. This is the shape issue #62 describes, for example per-package or per-dependency directories in a monorepo. The existing `windows_paths_mut` bench hid consolidation's benefit because 40% of its files sit directly in the tempdir root, collapsing everything into one recursive watch. The `sibling_subdirs` workload was also added for all watcher backends (inotify, fsevents, kqueue, `ReadDirectoryChangesW`, poll). Existing `paths_mut` file counts were lowered from `[100, 500, 1000]` to `[100, 200, 400]` to keep runtimes reasonable. ### Results * `windows_sibling_subdirs` runs 27 to 80% faster with consolidation, scaling with N. * The consolidated pre-open pass brings the `windows_paths_mut` workload back to par with the no-consolidation baseline (`nonrecursive/100`: 2.60 ms to 2.09 ms, 20% faster), while `windows_sibling_subdirs/nonrecursive/500` improved from 11.92 ms to 9.25 ms (22% faster).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
If more than 10 paths are watched, watch the parent path instead.
This behavior was inspired by chokidar:
https://github.com/paulmillr/chokidar/blob/7c50e25d10a497ce4409f6e52eb630f0d7647b97/lib/fsevents-handler.js#L113-L119