Skip to content

test: fail on missing per-kind snapshot instead of auto write#21027

Merged
alexander-akait merged 1 commit into
mainfrom
error-test-snap
May 25, 2026
Merged

test: fail on missing per-kind snapshot instead of auto write#21027
alexander-akait merged 1 commit into
mainfrom
error-test-snap

Conversation

@hai-x

@hai-x hai-x commented May 23, 2026

Copy link
Copy Markdown
Member

Summary

Improve local development DX. Previously a local compile error silently wrote errors.snap and passed; now a missing snapshot fails (like CI) unless you regenerate it with -u.

What kind of change does this PR introduce?

matchKindSnapshot built the per-kind SnapshotState with updateSnapshot: parentState._updateSnapshot, which is "new" in a normal local run. Under "new", a missing errors.snap/warnings.snap is silently written and the test passes on its first run, so a newly introduced compile error/warning is only caught in CI (where the mode is "none").

This forces the mode to "none" unless snapshots are explicitly being updated (-u"all"). A missing or changed per-kind snapshot now fails in local runs too; recording or refreshing one requires yarn jest -u.

Before: running a test locally that hit a compilation error passed, silently auto-generating errors.snap.

5c4d1d2c-bcd3-4756-a117-9e12c6e43905

After: such a test now fails when the snapshot is missing (matching CI behavior), unless you run with -u to generate it.

81bd8fed-8095-4ae4-b45a-718b34b65dbc

cc @xiaoxiaojx

Did you add tests for your changes?

Existing

Does this PR introduce a breaking change?

Nothing

If relevant, what needs to be documented once your changes are merged or what have you already documented?

n/a

Use of AI

Partial

@changeset-bot

changeset-bot Bot commented May 23, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 9201ed4

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions

github-actions Bot commented May 23, 2026

Copy link
Copy Markdown
Contributor

This PR is packaged and the instant preview is available (c61c649).

Install it locally:

  • npm
npm i -D webpack@https://pkg.pr.new/webpack@c61c649
  • yarn
yarn add -D webpack@https://pkg.pr.new/webpack@c61c649
  • pnpm
pnpm add -D webpack@https://pkg.pr.new/webpack@c61c649

@github-actions

Copy link
Copy Markdown
Contributor

Types Coverage

Coverage after merging error-test-snap into main will be
98.96%
Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
bin
   webpack.js98.77%100%100%98.77%91
examples
   build-common.js100%100%100%100%
   buildAll.js100%100%100%100%
   examples.js100%100%100%100%
   template-common.js98.21%100%100%98.21%72
examples/custom-javascript-parser
   test.filter.js100%100%100%100%
examples/custom-javascript-parser/internals
   acorn-parse.js100%100%100%100%
   meriyah-parse.js100%100%100%100%
   oxc-parse.js91.30%100%100%91.30%140, 142–143, 145, 147, 153–154, 161, 168, 90
examples/markdown
   webpack.config.mjs100%100%100%100%
examples/typescript
   test.filter.js100%100%100%100%
examples/typescript-non-erasable
   test.filter.js50%100%100%50%5
examples/virtual-modules
   test.filter.js100%100%100%100%
examples/wasm-bindgen-esm
   test.filter.js100%100%100%100%
examples/wasm-complex
   test.filter.js100%100%100%100%
examples/wasm-simple
   test.filter.js100%100%100%100%
examples/wasm-simple-source-phase
   test.filter.js100%100%100%100%
lib
   APIPlugin.js100%100%100%100%
   AsyncDependenciesBlock.js100%100%100%100%
   AutomaticPrefetchPlugin.js100%100%100%100%
   BannerPlugin.js100%100%100%100%
   Cache.js98.21%100%100%98.21%101
   CacheFacade.js100%100%100%100%
   Chunk.js99.72%100%100%99.72%39
   ChunkGraph.js100%100%100%100%
   ChunkGroup.js100%100%100%100%
   ChunkTemplate.js100%100%100%100%
   CleanPlugin.js99.15%100%100%99.15%206, 226
   CodeGenerationResults.js100%100%100%100%
   CompatibilityPlugin.js100%100%100%100%
   Compilation.js98.45%100%100%98.45%1572, 1868, 1875, 1883, 1905, 2801, 3226, 3888, 3917, 3970–3971, 3975, 3980, 3996–3997, 4011–4012, 4017–4018, 4495, 4521, 511, 516, 5229, 5261, 5278, 5294, 5310, 5325, 5350–5351, 5353, 5681, 5686, 5692, 5695, 5707, 5709, 5713, 5729, 5744, 5776, 5830, 5854, 5968, 730–731
   Compiler.js99.55%100%100%99.55%1116–1117, 1125
   ConcatenationScope.js98.59%100%100%98.59%189
   ConditionalInitFragment.js100%100%100%100%
   ConstPlugin.js100%100%100%100%
   ContextExclusionPlugin.js100%100%100%100%
   ContextModule.js100%100%100%100%
   ContextModuleFactory.js97.40%100%100%97.40%258, 395, 418, 420, 424, 433–434
   ContextReplacementPlugin.js100%100%100%100%
   DefinePlugin.js98.92%100%100%98.92%158–159, 175, 194, 268
   DependenciesBlock.js100%100%100%100%
   Dependency.js98.20%100%100%98.20%379, 425
   DependencyTemplate.js100%100%100%100%
   DependencyTemplates.js100%100%100%100%
   DotenvPlugin.js98.41%100%100%98.41%378, 391–392
   DynamicEntryPlugin.js100%100%100%100%
   EntryOptionPlugin.js100%100%100%100%
   EntryPlugin.js100%100%100%100%
   Entrypoint.js100%100%100%100%
   EnvironmentPlugin.js97.14%100%100%97.14%49
   ErrorHelpers.js100%100%100%100%
   EvalDevToolModulePlugin.js100%100%100%100%
   EvalSourceMapDevToolPlugin.js100%100%100%100%
   ExportsInfo.js100%100%100%100%
   ExportsInfoApiPlugin.js100%100%100%100%
   ExternalModule.js98.97%100%100%98.97%425–429, 577
   ExternalModuleFactoryPlugin.js100%100%100%100%
   ExternalsPlugin.js100%100%100%100%
   FileSystemInfo.js99.50%100%100%99.50%182, 2252–2253, 2256, 2267, 2278, 2289, 278, 3694, 3709, 3733
   FlagAllModulesAsUsedPlugin.js100%100%100%100%
   FlagDependencyExportsPlugin.js98.74%100%100%98.74%399, 401, 405
   FlagDependencyUsagePlugin.js100%100%100%100%
   FlagEntryExportAsUsedPlugin.js100%100%100%100%
   Generator.js100%100%100%100%
   HotModuleReplacementPlugin.js100%100%100%100%
   HotUpdateChunk.js100%100%100%100%
   IgnorePlugin.js100%100%100%100%
   IgnoreWarningsPlugin.js100%100%100%100%
   InitFragment.js100%100%100%100%
   JavascriptMetaInfoPlugin.js100%100%100%100%
   LibraryTemplatePlugin.js100%100%100%100%
   LoaderOptionsPlugin.js100%100%100%100%
   LoaderTargetPlugin.js100%100%100%100%
   MainTemplate.js100%100%100%100%
   ManifestPlugin.js100%100%100%100%
   Module.js98.50%100%100%98.50%1305, 1310, 1371, 1385, 1447, 1456
   ModuleFactory.js100%100%100%100%
   ModuleFilenameHelpers.js98.85%100%100%98.85%106, 108
   ModuleGraph.js99.73%100%100%99.73%1004
   ModuleGraphConnection.js100%100%100%100%
   ModuleInfoHeaderPlugin.js100%100%100%100%
   ModuleNotFoundError.js100%100%100%100%
   ModuleProfile.js100%100%100%100%
   ModuleSourceTypeConstants.js100%100%100%100%
   ModuleTemplate.js100%100%100%100%
   ModuleTypeConstants.js100%100%100%100%
   MultiCompiler.js99.69%100%100%99.69%645
   MultiStats.js100%100%100%100%
   MultiWatching.js100%100%100%100%
   NoEmitOnErrorsPlugin.js100%100%100%100%
   NodeStuffPlugin.js100%100%100%100%
   NormalModule.js98.11%100%100%98.11%1208, 1211, 1228, 1245, 1492, 1526, 1542, 1629, 2252, 2257–2267, 569
   NormalModuleFactory.js99.47%100%100%99.47%1083, 1392, 486, 498
   NormalModuleReplacementPlugin.js100%100%100%100%
   NullFactory.js100%100%100%100%
   OptimizationStages.js100%100%100%100%
   OptionsApply.js100%100%100%100%
   Parser.js100%100%100%100%
   PlatformPlugin.js100%100%100%100%
   PrefetchPlugin.js100%100%100%100%
   ProgressPlugin.js98.85%100%100%98.85%519–520, 525, 527, 591
   ProvidePlugin.js100%100%100%100%
   RawModule.js100%100%100%100%
   RecordIdsPlugin.js100%100%100%100%
   RequestShortener.js100%100%100%100%
   ResolverFactory.js100%100%100%100%
   RuntimeGlobals.js100%100%100%100%
   RuntimeModule.js100%100%100%100%
   RuntimePlugin.js100%100%100%100%
   RuntimeTemplate.js100%100%100%100%
   SelfModuleFactory.js100%100%100%100%
   SingleEntryPlugin.js100%100%100%100%
   SourceMapDevToolModuleOptionsPlugin.js100%100%100%100%
   SourceMapDevToolPlugin.js98.62%100%100%98.62%220, 224, 226, 419, 430, 891
   Stats.js100%100%100%100%
   Template.js100%100%100%100%
   TemplatedPathPlugin.js98.86%100%100%98.86%136–137
   UseStrictPlugin.js100%100%100%100%
   WarnCaseSensitiveModulesPlugin.js100%100%100%100%
   WarnDeprecatedOptionPlugin.js100%100%100%100%
   WarnNoModeSetPlugin.js100%100%100%100%
   WatchIgnorePlugin.js100%100%100%100%
   Watching.js100%100%100%100%
   WebpackError.js100%100%100%100%
   WebpackIsIncludedPlugin.js100%100%100%100%
   WebpackOptionsApply.js100%100%100%100%
   WebpackOptionsDefaulter.js100%100%100%100%
   buildChunkGraph.js99.87%100%100%99.87%325
   cli.js98.46%100%100%98.46%10, 119, 471, 503, 545, 815
   index.js99.72%100%100%99.72%165
   validateSchema.js94.67%100%100%94.67%100, 87, 89, 98
   webpack.js96.33%100%100%96.33%10, 198, 220, 222
lib/asset
   AssetBytesGenerator.js100%100%100%100%
   AssetBytesParser.js100%100%100%100%
   AssetGenerator.js100%100%100%100%
   AssetModulesPlugin.js97.32%100%100%97.32%283, 307, 310, 36, 362, 41
   AssetParser.js100%100%100%100%
   AssetSourceGenerator.js100%100%100%100%
   AssetSourceParser.js100%100%100%100%
   RawDataUrlModule.js100%100%100%100%
lib/async-modules
   AsyncModuleHelpers.js100%100%100%100%
   AwaitDependenciesInitFragment.js100%100%100%100%
   InferAsyncModulesPlugin.js100%100%100%100%
lib/cache
   AddBuildDependenciesPlugin.js100%100%100%100%
   AddManagedPathsPlugin.js100%100%100%100%
   IdleFileCachePlugin.js97.92%100%100%97.92%71, 83, 91
   MemoryCachePlugin.js95.83%100%100%95.83%33
   MemoryWithGcCachePlugin.js93.15%100%100%93.15%106, 113–114, 122, 89
   PackFileCacheStrategy.js96.40%100%100%96.40%1250, 1350, 1354, 1416, 628, 647, 657–659, 661, 677–678, 683, 686, 688, 693, 698, 722, 728, 762, 768, 774, 779, 790, 799, 804–805, 807, 824, 830–831, 833
   ResolverCachePlugin.js100%100%100%100%
   getLazyHashedEtag.js100%100%100%100%
   mergeEtags.js100%100%100%100%
lib/config
   browserslistTargetHandler.js100%100%100%100%
   defaults.js99.29%100%100%99.29%1411–1413, 1421, 271, 274, 279, 283
   normalization.js99%100%100%99%191–192, 258, 273
   target.js100%100%100%100%
lib/container
   ContainerEntryDependency.js100%100%100%100%
   ContainerEntryModule.js100%100%100%100%
   ContainerEntryModuleFactory.js100%100%100%100%
   ContainerExposedDependency.js100%100%100%100%
   ContainerPlugin.js100%100%100%100%
   ContainerReferencePlugin.js100%100%100%100%
   FallbackDependency.js100%100%100%100%
   

@codecov

codecov Bot commented May 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.58%. Comparing base (75f60f6) to head (9201ed4).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #21027      +/-   ##
==========================================
- Coverage   91.59%   91.58%   -0.01%     
==========================================
  Files         573      573              
  Lines       59541    59541              
  Branches    16077    16077              
==========================================
- Hits        54534    54530       -4     
- Misses       5007     5011       +4     
Flag Coverage Δ
integration 89.50% <ø> (-0.01%) ⬇️
test262 45.31% <ø> (ø)
unit 37.95% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented May 23, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 36.1%

⚡ 6 improved benchmarks
❌ 5 regressed benchmarks
✅ 133 untouched benchmarks
⏩ 72 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Memory benchmark "css-modules", scenario '{"name":"mode-development","mode":"development"}' 1,077 KB 540.1 KB +99.39%
Memory benchmark "asset-modules-source", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 3,716.7 KB 173 KB ×21
Memory benchmark "context-esm", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 659.6 KB 146.2 KB ×4.5
Memory benchmark "side-effects-reexport", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 1,021.2 KB 1,305.5 KB -21.78%
Memory benchmark "cache-filesystem", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 1,027.2 KB 842.4 KB +21.94%
Memory benchmark "concatenate-modules", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 159.6 KB 131.5 KB +21.36%
Memory benchmark "wasm-modules-sync", scenario '{"name":"mode-production","mode":"production"}' 6 MB 8 MB -24.3%
Memory benchmark "concatenate-modules", scenario '{"name":"mode-development","mode":"development"}' 786.6 KB 1,116.3 KB -29.54%
Memory benchmark "many-modules-esm", scenario '{"name":"mode-development","mode":"development"}' 1,099.7 KB 844.3 KB +30.25%
Memory benchmark "wasm-modules-async", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 399.5 KB 1,550.6 KB -74.24%
Memory benchmark "context-esm", scenario '{"name":"mode-production","mode":"production"}' 7.4 MB 9.9 MB -25.82%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing error-test-snap (9201ed4) with main (12cb825)

Open in CodSpeed

Footnotes

  1. 72 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@alexander-akait alexander-akait merged commit c61c649 into main May 25, 2026
60 of 62 checks passed
@alexander-akait alexander-akait deleted the error-test-snap branch May 25, 2026 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants