perf: consolidate watch paths via ConsolidatingPathTrie on Windows#93
Merged
Conversation
ConsolidatingPathTrie
236b6c8 to
d91c372
Compare
Close #62. Mirror the fsevents PR #60 pattern: when 10+ sibling paths are registered under the same parent, collapse them into a single watch on the parent instead of opening one `ReadDirectoryChangesW` handle per child. Implementation: * Replace the `WatchMode` value of `self.watches` with a new `UserWatch` struct that carries the user's intent plus the resolved OS-level primary dir (and optional tracked-parent for `TargetMode::TrackPath`). * Add `rebuild_watch_handles`: build a `ConsolidatingPathTrie` over the primary dirs from every user watch, compute the recursive flag per consolidated target by checking for strict descendants in the user watches, layer in tracked-parent watches that aren't already covered, and diff against the open `watch_handles` to converge the OS state. * `add_watch` / `remove_watch` / `apply_staged` now update `self.watches` first and then call `rebuild_watch_handles` instead of poking `add_watch_raw` directly. * The event filter at `handle_event` was a direct hash lookup on the OS dir name; consolidation can leave the OS dir different from any user request, so it now walks ancestors via `is_event_covered`. * `pre_open_tracked_parent` works around a Windows quirk where calling `path.metadata()` on a directory *before* its parent dir has a `ReadDirectoryChangesW` handle silently drops the parent's later `FILE_ACTION_MODIFIED` event for that directory. The pre-call opens the tracked parent before any metadata is read, matching the order the previous per-call code used. Adds five new tests covering: 10-sibling consolidation, event delivery through a consolidated parent, mixed recursive/non-recursive collapse, de-consolidation when the count drops back below the threshold, and the batched `paths_mut().commit()` path.
d91c372 to
22d450b
Compare
The existing `windows_paths_mut` bench puts 40% of its files directly in the tempdir root, so every watched parent dir consolidates back into that root and the result is a single recursive watch on the entire tempdir subtree. That hides consolidation's actual benefit — on this workload consolidation is slightly slower because pre-opened handles are immediately replaced. The new `windows_sibling_subdirs` bench watches N sibling directories under a shared parent, which is the shape issue #62 is about (e.g., watching each package directory in a monorepo, or each dependency directory). With consolidation enabled this workload is 27–80% faster than with consolidation disabled, with the win scaling as N grows.
8d4ab77 to
bc2a0b2
Compare
…ed commit `apply_staged` previously did a per-path `pre_open_tracked_parent` — N opens just to satisfy the Windows metadata-quirk that requires the direct parent to be watched before `path.metadata()` runs. Whenever consolidation later collapsed those N parents into a higher ancestor, all the pre-opens were thrown away (N closes + 1 reopen). Replace the per-path pre-open with a single `pre_open_consolidated` step that runs the same consolidation trie lexically over each staged path's direct parent, opens the *consolidated* set, and picks the recursive flag based on whether any staged parent is a strict descendant of the consolidated target. The subsequent `rebuild_watch_handles` against the actual primaries usually finds the target already in place, so no close-and-reopen is needed. On the existing `windows_paths_mut` bench, which collapses everything into a single recursive watch on the test root because 40% of files sit directly in it, this drops setup from ~13 OS ops to ~1. Measured delta on `windows_paths_mut/nonrecursive/1000`: 21.37 ms → 20.78 ms (-3%) and `windows_paths_mut/nonrecursive/100`: 2.60 ms → 2.09 ms (-20%), bringing the consolidated path back to par with the no-consolidation baseline while keeping the consolidation benefit. The sibling-subdir workload — where consolidation actually saves handles — also speeds up at the high end (`windows_sibling_subdirs/ nonrecursive/500`: 11.92 ms → 9.25 ms, -22%). All 87 existing tests pass unchanged; the conservative open uses exactly the same direct-parent invariant that `pre_open_tracked_parent` relied on, so the metadata-quirk fix is preserved.
Add a `sibling_subdirs` workload to the paths_mut benchmark for every watcher backend (inotify, fsevents, kqueue, ReadDirectoryChangesW, and the cross-platform poll watcher). It watches many sibling subdirectories under a shared parent -- representative of monorepo or dependency-directory monitoring -- to measure the benefit of path consolidation. Also lower the existing paths_mut file counts from [100, 500, 1000] to [100, 200, 400] to keep benchmark runtimes reasonable.
`watches` previously stored a `UserWatch` struct that bundled the user's raw `WatchMode` request together with the metadata-resolved coverage (`primary` dir, `tracked_parent`). This conflated the source of truth with derived state. Split it into two maps keyed by user path: * `watches: HashMap<PathBuf, WatchMode>` keeps only the raw user request. * `resolved_watches: HashMap<PathBuf, ResolvedWatch>` holds the resolved coverage. It is server-only and not shared with the event thread, which needs only the raw mode. `rebuild_watch_handles` prunes `resolved_watches` against `watches` at the start, since the event thread can drop a `NoTrack` entry from `watches` directly without touching `resolved_watches`. This keeps consolidation from ever using a stale resolution. All 87 lib tests pass.
ConsolidatingPathTrieConsolidatingPathTrie on Windows
Merged
sapphi-red
pushed a commit
that referenced
this pull request
May 20, 2026
## 🤖 New release * `rolldown-notify`: 10.3.1 -> 10.4.0 (✓ API compatible changes) * `rolldown-notify-debouncer-mini`: 0.8.7 -> 0.8.8 * `rolldown-notify-debouncer-full`: 0.7.7 -> 0.7.8 <details><summary><i><b>Changelog</b></i></summary><p> ## `rolldown-notify` <blockquote> ## [10.4.0](rolldown-notify-v10.3.1...rolldown-notify-v10.4.0) - 2026-05-20 ### Added - consolidate paths further more for fsevents backend if needed ([#91](#91)) ### Other - consolidate watch paths via `ConsolidatingPathTrie` on Windows ([#93](#93)) - batch path changes via `PathsMut` on Windows ([#94](#94)) </blockquote> ## `rolldown-notify-debouncer-mini` <blockquote> ## [0.8.8](rolldown-notify-debouncer-mini-v0.8.7...rolldown-notify-debouncer-mini-v0.8.8) - 2026-05-20 ### Other - updated the following local packages: rolldown-notify </blockquote> ## `rolldown-notify-debouncer-full` <blockquote> ## [0.7.8](rolldown-notify-debouncer-full-v0.7.7...rolldown-notify-debouncer-full-v0.7.8) - 2026-05-20 ### Other - updated the following local packages: rolldown-notify </blockquote> </p></details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/). Co-authored-by: rolldown-guard[bot] <278280044+rolldown-guard[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On Windows, registering many sibling paths under the same parent previously opened one
ReadDirectoryChangesWhandle per child. This PR mirrors the fsevents consolidation pattern from #60: when 10 or more sibling paths are registered under the same parent, they collapse into a single watch on the parent instead.closes #62
How it works
UserWatchstruct.self.watchesnow stores the user's intent plus the resolved OS-level primary dir (and optional tracked-parent forTargetMode::TrackPath), instead of a bareWatchMode.rebuild_watch_handles. Builds aConsolidatingPathTrieover the primary dirs of all user watches, computes the recursive flag per consolidated target by checking for strict descendants, layers in tracked-parent watches, and diffs against the open handles to converge OS state.add_watch,remove_watch, andapply_stagednow updateself.watchesand call this, rather than pokingadd_watch_rawdirectly.is_event_covered. Since consolidation can leave the OS dir different from any user request, event filtering now walks ancestors instead of doing a direct hash lookup.apply_stagedno longer does a per-pathpre_open_tracked_parent, which caused N opens that consolidation usually threw away. A singlepre_open_consolidatedpass runs the same consolidation trie over staged parents and opens the consolidated set directly, so the laterrebuild_watch_handlesusually finds handles already in place, with no close-and-reopen. This preserves the Windows metadata quirk fix: a directory'smetadata()must run after its parent is watched, or the parent's laterFILE_ACTION_MODIFIEDis silently dropped.Benchmarks
A new
windows_sibling_subdirsworkload watches N sibling directories under a shared parent. This is the shape issue #62 describes, for example per-package or per-dependency directories in a monorepo. The existingwindows_paths_mutbench hid consolidation's benefit because 40% of its files sit directly in the tempdir root, collapsing everything into one recursive watch.The
sibling_subdirsworkload was also added for all watcher backends (inotify, fsevents, kqueue,ReadDirectoryChangesW, poll). Existingpaths_mutfile counts were lowered from[100, 500, 1000]to[100, 200, 400]to keep runtimes reasonable.Results
windows_sibling_subdirsruns 27 to 80% faster with consolidation, scaling with N.windows_paths_mutworkload back to par with the no-consolidation baseline (nonrecursive/100: 2.60 ms to 2.09 ms, 20% faster), whilewindows_sibling_subdirs/nonrecursive/500improved from 11.92 ms to 9.25 ms (22% faster).