fix: invalidate provided-exports cache with lazy barrel#21326
Conversation
🦋 Changeset detectedLatest commit: 0f6670e The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
This PR is packaged and the instant preview is available (7639066). Install it locally:
npm i -D webpack@https://pkg.pr.new/webpack@7639066
yarn add -D webpack@https://pkg.pr.new/webpack@7639066
pnpm add -D webpack@https://pkg.pr.new/webpack@7639066 |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #21326 +/- ##
==========================================
- Coverage 92.86% 92.47% -0.40%
==========================================
Files 594 594
Lines 65210 65335 +125
Branches 18143 18189 +46
==========================================
- Hits 60559 60417 -142
- Misses 4651 4918 +267
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will improve performance by 40.13%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Memory | benchmark "many-modules-esm", scenario '{"name":"mode-development","mode":"development"}' |
1.1 MB | 1.9 MB | -39.59% |
| ❌ | Memory | benchmark "wasm-modules-async", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' |
189.6 KB | 246.9 KB | -23.23% |
| ⚡ | Memory | benchmark "asset-modules-inline", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' |
1,194.8 KB | 382.6 KB | ×3.1 |
| ⚡ | Memory | benchmark "wasm-modules-sync", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' |
354 KB | 132.9 KB | ×2.7 |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing fix-lazy-barrel-flag-dep (0f6670e) with main (c2628cf)
|
@hai-x looks good let’s merge and make the patch release |
c1240b0 to
e1774fa
Compare
|
Thanks for the review! I've reworked the approach in two ways:
Could you take another look? |
|
tried with the current patch and seems this doesn't fix the issue. i delete webpack cache and start a fresh build, few lazy compilations later it throws |
|
now that i can reproduce the issue consistently locally, this PR behaves the same as 5.108.3, while #21322 prevents the errors. |
|
though i applied only this PR on top of 5.108.3, in case this depends on something else merged after 5.108.3... |
@Knagis Hi, have you try the version, or just edit the |
|
@Knagis can you try to remove the old cache and run code with this patch |
|
The PR which I closed just disable cache, yes it is the fix, but the solution is not good, ideally we should find the root of the problem and keep cache |
|
@Knagis Could you share some reproduction steps? I can repro it with
|
|
i'm taking https://patch-diff.githubusercontent.com/raw/webpack/webpack/pull/21326.patch and adding it as yarn patch on top of 5.108.3 (we have all kinds of rules in place that force npm packages from our own repository etc. so patch is the easiest) in my scenario it now fails with just memory cache, not even restarting the build (initial build, then one lazy chunk builds and then the next one fails). |
Make sense, let me try it with memory cache and lazy compilation. Thanks for sharing. |
eda84cc to
56d1fc0
Compare
|
@Knagis Yeah, this issue is about unsafe memory cache with |
56d1fc0 to
c4e91c9
Compare
c4e91c9 to
0f6670e
Compare
Types CoverageCoverage after merging fix-lazy-barrel-flag-dep into main will be
Coverage Report |
|
@Knagis friendly ping, want to make the patch release |
|
Right now with the latest version of this PR i can no longer reproduce the error. Not that i could reproduce all the cases before :) |
|
@Knagis lets try, if you faced with the issue agin after the patch release feel free to open a new issue |
Summary
What kind of change does this PR introduce?
Fixes #21318.
A lazy barrel cached while its re-export targets were deferred could serve stale provided exports after a later build un-defers a target, producing link error
"export 'x' was not found". Two independent cache layers had this staleness; each gets its own fix.FlagDependencyExportsPlugin)lib/FlagDependencyExportsPlugin.js: the cache etag wasbuildInfo.hashalone; it's now a plain string etag that appends the barrel's sorted still-lazy request keys:${hash}|${[...lazyKeys].sort().join("|")}. No hashing (etags are only string-compared downstream), no buildInfo state.lib/LazyBarrel.js: the still-lazy keys are derived from state the controller already maintains: newLazyBarrelInfo.getLazyRequestKeys()(the_requestToDepGroupmap keys) andLazyBarrelController.getLazyRequestKeys(module). Nothing is mirrored, mutated in sync, or serialized into the persistent cache.experiments.cacheUnaffected)lib/Compilation.js:compareReferencesassumed a non-rebuilt module's connection set is stable, an invariant lazy-barrel deferral breaks: un-deferring adds a connection without rebuilding the barrel. If the target module already existed unchanged, no other invalidation caught it and the stalememCacheshort-circuited in front of Fix 1. The snapshot is now{ map, count }, both sides apply the same dep filter so a map miss unambiguously means an added connection (immediate invalidation), and the count catches removals. The Add+remove pairs can't mask each other since additions are detected per-entry.Tests
test/PersistentCaching.test.js: cross-process stale-barrel case for Fix 1 (plus an mkdir fix for nested fixture paths).test/watchCases/lazy-barrel/stale-star-exports: new watch case: deferred star target imported on rebuild, plus a named re-export (sub2) variant.test/watchCases/lazy-barrel/stale-star-exports-mem-cache: new watch case reproducing Fix 2's scenario: the star target pre-exists via a direct import (so it's "unchanged"), then gets imported through the barrel under cache.cacheUnaffected. Failed before Fix 2, passes now.test/watchCases/lazy-barrel/unsafe-cache/, build-on-new-import: enableexportsPresence: "error"so stale provided exports fail loudly; unsafe-cache step 0 now starts from a fully-deferred state (the "requesting a builds its target" invariant moved to step 1'sexpect(a).toBe("a")).Did you add tests for your changes?
Yes
Does this PR introduce a breaking change?
No
If relevant, what needs to be documented once your changes are merged or what have you already documented?
n/a
Use of AI
Partial