fix: narrow purge for watched directories in NodeWatchFileSystem#21020
Merged
Conversation
Watchpack reports a watched directory (a context dependency) in `changes` whenever its contents change, alongside the individual file events. `inputFileSystem.purge(dir)` uses `key.startsWith(dir)` internally, so calling it on the directory wiped the stat cache of every file inside, even though only file-level events actually invalidate file stats. `ContextModuleFactory.resolveDependencies` then re-stat-ed every file in the tree on every incremental rebuild. For directories we explicitly watch, invalidate only the directory's own `readdir` entry; file-level changes in the same aggregated event continue to purge file stats and the parent `readdir` (via `purgeParent`) as before, so adds, removes, modifies and nested changes are still detected. Single-file rebuilds on a local 4000-file context drop from ~370 ms to ~225 ms (~40%). Closes #13636
…actory `ContextModuleFactory.resolveDependencies` previously fired the `alternativeRequests` hook once per file with a single-item array, paying per-call overhead (closure allocation, `compiler.resolverFactory.get(...)` lookup, intermediate array allocations inside `RequireContextPlugin`'s tap) for every file in the context. Collect all matched files per directory walk and invoke the hook once with the full batch; the existing tap already iterates the items array so the output is unchanged. Trims a further ~15 ms from steady-state rebuilds on a 4000-file `require.context` workload.
…TED_ANY Replace the `/** @type {EXPECTED_ANY} */ (fs)._readdirBackend` cast in `NodeWatchFileSystem`'s narrow-purge path with a local `CacheBackend` typedef and an `InputFileSystemWithBackends` augmented file-system type, so the private backend slot we read off enhanced-resolve's `CachedInputFileSystem` is described precisely instead of widened to `any`. No behavior change.
…o _readdirBackend
Replace the private `_readdirBackend.purge(item)` access with the
public unified API added in enhanced-resolve@5.22.0:
fs.purge(item, { exact: true })
This invalidates only the exact key across every backend (stat,
lstat, readdir, readlink, realpath, readFile, readJson) without
removing cached entries for descendants — the same semantics the
previous _readdirBackend probe was approximating, but expressed as
the consumer-facing API rather than a private slot.
Drops the CacheBackend / InputFileSystemWithBackends JSDoc typedefs
that only existed to type the private access, and bumps the
enhanced-resolve peer to ^5.22.0.
Performance characteristic is unchanged: median single-file rebuild
on a 4000-file watched context is ~650 ms vs ~1260 ms without
narrowing (≈49% local reproduction).
Lockfile update following the package.json bump to ^5.22.0, which
introduced the { exact: true } option on CachedInputFileSystem#purge
consumed by NodeWatchFileSystem.
🦋 Changeset detectedLatest commit: e8cb5a2 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
This PR is packaged and the instant preview is available (a086147). Install it locally:
npm i -D webpack@https://pkg.pr.new/webpack@a086147
yarn add -D webpack@https://pkg.pr.new/webpack@a086147
pnpm add -D webpack@https://pkg.pr.new/webpack@a086147 |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #21020 +/- ##
=======================================
Coverage 91.57% 91.57%
=======================================
Files 573 573
Lines 59460 59465 +5
Branches 16054 16058 +4
=======================================
+ Hits 54449 54458 +9
+ Misses 5011 5007 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Improves watch-mode incremental rebuild performance for large context dependencies by avoiding overly broad filesystem cache invalidation and reducing per-file hook-call overhead in context dependency resolution.
Changes:
- Narrow
inputFileSystem.purge()invalidation for explicitly watched context directories by using{ exact: true }(enhanced-resolve ≥ 5.22.0) inNodeWatchFileSystem. - Batch
ContextModuleFactory’salternativeRequestshook invocation per directory instead of per file to reduce overhead. - Bump
enhanced-resolveto^5.22.0and update filesystem purge typings/documentation accordingly; add changesets for release notes.
Reviewed changes
Copilot reviewed 6 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
lib/node/NodeWatchFileSystem.js |
Purges watched directories with { exact: true } to avoid wiping descendant stat cache on directory change events. |
lib/ContextModuleFactory.js |
Batches alternativeRequests hook calls per directory to cut per-file overhead during context resolution. |
lib/util/fs.js |
Extends Purge typedef docs/signature to include optional { exact?: boolean }. |
package.json |
Updates enhanced-resolve dependency to ^5.22.0. |
yarn.lock |
Locks enhanced-resolve to 5.22.0. |
types.d.ts |
Updates generated InputFileSystem.purge type signature to accept optional { exact?: boolean }. |
.changeset/watch-narrow-purge-context-dir.md |
Adds changeset entry for the watch-mode purge narrowing performance fix. |
.changeset/batch-alternative-requests.md |
Adds changeset entry for batching alternativeRequests performance improvement. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
enhanced-resolve 5.22.0 introduced a CompiledAliasOptions typedef (in AliasUtils.js) referenced transitively via TsconfigPathsData.alias. yarn fix:special picks it up; CI's lint:special job requires the regenerated file to be committed.
Contributor
Types CoverageCoverage after merging claude/fix-issue-13636-ts24Z into main will be
Coverage Report
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Watchpack reports a watched directory (a context dependency) in
changeswhenever its contents change, alongside the individual file events.
inputFileSystem.purge(dir)useskey.startsWith(dir)internally, socalling it on the directory wiped the stat cache of every file inside,
even though only file-level events actually invalidate file stats.
ContextModuleFactory.resolveDependenciesthen re-stat-ed every file inthe tree on every incremental rebuild.
For directories we explicitly watch, invalidate only the directory's own
readdirentry; file-level changes in the same aggregated event continueto purge file stats and the parent
readdir(viapurgeParent) asbefore, so adds, removes, modifies and nested changes are still detected.
Single-file rebuilds on a local 4000-file context drop from ~370 ms to
~225 ms (~40%).
Closes #13636