Skip to content

Reduce skipped tests across Node, Deno and Bun and prune dead WebAssembly filters#21257

Merged
alexander-akait merged 15 commits into
mainfrom
fix/color-support-and-runtime-test-skips
Jun 23, 2026
Merged

Reduce skipped tests across Node, Deno and Bun and prune dead WebAssembly filters#21257
alexander-akait merged 15 commits into
mainfrom
fix/color-support-and-runtime-test-skips

Conversation

@alexander-akait

Copy link
Copy Markdown
Member

Summary

Reduces skipped/filtered tests and aligns behavior across Node, Deno and Bun:

  • Removes 32 always-true supportsWebAssembly() / supportsOptionalCatchBinding() test filters (true on Node >= 10.13, Deno and Bun) and simplifies 9 combined ones; re-enables cases/wasm/v128, whose .wasm fixture was invalid (a v128.load with no memory section).
  • Fixes the externals/import-attributes node-version gate /^v2[2-9]/, which silently skipped on Node 30+, to major >= 22 (matching json-import-attributes).
  • Fixes isColorSupported() to detect color on Deno and Bun: it probed process.env with the in operator, which their process.env shims do not honor; reading the values is identical on Node and works on all three runtimes.
  • Unskips the Bun-only parseJson, MultiItemCache and AbstractMethodError tests by asserting the engine-agnostic contract on Bun (JSC error text/stack differ from V8) while keeping the exact assertions on Node and Deno.

What kind of change does this PR introduce?

fix (CLI color detection) plus test (filter pruning and cross-runtime test fixes).

Did you add tests for your changes?

Yes — re-enabled test/cases/wasm/v128; the color fix is covered by Cli.basictest (isColorSupported and the colors blocks); the remaining changes are themselves test changes.

Does this PR introduce a breaking change?

No.

If relevant, what needs to be documented once your changes are merged or what have you already documented?

n/a

Use of AI

Yes — used Claude Code to audit the test filters and runtime skips, identify stale/over-restrictive ones, implement the changes and verify them on Node. The Deno and Bun paths are verified by construction (engine-agnostic assertions) and confirmed by the runtimes CI job.


Generated by Claude Code

@changeset-bot

changeset-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1f7b402

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
webpack Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

This PR is packaged and the instant preview is available (efeb5ce).

Install it locally:

  • npm
npm i -D webpack@https://pkg.pr.new/webpack@efeb5ce
  • yarn
yarn add -D webpack@https://pkg.pr.new/webpack@efeb5ce
  • pnpm
pnpm add -D webpack@https://pkg.pr.new/webpack@efeb5ce

@codecov

codecov Bot commented Jun 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.81%. Comparing base (9c64a63) to head (1f7b402).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #21257   +/-   ##
=======================================
  Coverage   92.80%   92.81%           
=======================================
  Files         591      591           
  Lines       64658    64729   +71     
  Branches    18012    18042   +30     
=======================================
+ Hits        60007    60075   +68     
- Misses       4651     4654    +3     
Flag Coverage Δ
css-parsing 28.71% <11.11%> (+0.01%) ⬆️
html5lib 31.18% <11.11%> (+0.01%) ⬆️
integration 88.94% <100.00%> (-0.01%) ⬇️
test262 45.40% <33.33%> (-0.04%) ⬇️
unit 41.50% <33.33%> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented Jun 23, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
❌ 2 regressed benchmarks
✅ 139 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Memory benchmark "asset-modules-inline", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 389.5 KB 1,230.9 KB -68.36%
Memory benchmark "wasm-modules-sync", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 123.6 KB 360.1 KB -65.69%
Memory benchmark "side-effects-reexport", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 859.1 KB 136.7 KB ×6.3
Memory benchmark "many-modules-esm", scenario '{"name":"mode-development","mode":"development"}' 1.9 MB 1.2 MB +65.89%
Memory benchmark "many-modules-commonjs", scenario '{"name":"mode-production","mode":"production"}' 9.7 MB 7 MB +38.84%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fix/color-support-and-runtime-test-skips (1f7b402) with main (ecc00c0)

Open in CodSpeed

The `supportsWebAssembly()` guard is true on every supported runtime
(Node >= 10.13, Deno, Bun), so those `test.filter.js` files never skip
anything. Remove the 31 pure guards plus the always-true
`supportsOptionalCatchBinding()` guard, and drop the dead
`supportsWebAssembly()` conjunct from 9 combined filters.

Fix the long-disabled `cases/wasm/v128` fixture: its `.wasm` encoded a
`v128.load` with no memory section (CompileError: memory instruction
with no memory), so it was hard-disabled with `() => false`. Replace it
with a valid v128 module and gate the case on SIMD support like the
sibling cases.
The filter used `/^v2[2-9]/.test(process.version)`, which only matches
Node v22–v29 and silently skips the case on Node 30 and later. Replace
it with a numeric `major >= 22` check, matching the sibling
`json-import-attributes` filter for the same `with` syntax.
`isColorSupported()` probed the environment with the `in` operator
(`"FORCE_COLOR" in env`, `"CI" in env`, …). Deno's and Bun's
`process.env` shims honor property access but not the `in`/`has` trap,
so those signals were missed and the CLI emitted no colors. Read the
values directly instead — identical on Node, and works on Deno and Bun.

With detection fixed, the "colors by default" test no longer needs a
runtime skip on Deno and Bun.
The malformed-JSON tests asserted V8's exact SyntaxError wording and
positions, so they were skipped on Bun (JavaScriptCore emits different
text). Assert the exact wording only on V8 (Node, Deno) and check the
engine-agnostic JSONParseError contract (name, systemError, numeric
position, non-empty message) on every runtime, and drop the Bun skip.
Both were skipped on Bun for engine-specific error text: MultiItemCache
asserted V8's exact "not a function" TypeError wording, and
AbstractMethodError asserted a caller name parsed out of V8's
Error.stack format. Assert the engine-agnostic contract instead — the
error type for MultiItemCache, and the base message on Bun with the
exact caller-info message kept on V8 (Node, Deno) — and drop the skips.
Add cases for `--color` and `--no-color` in process.argv, the two
branches of the color detection that no test exercised.
The parser handler ran `require("../../package.json")` lazily, so the
async resolver-cache tail could trigger it after a test tore down the
module registry (the "require after environment torn down" crash that
forced HotTestCases to be skipped on Deno). Load the version once at
module scope, like DefaultStatsFactoryPlugin already does.
With the import.meta version require hoisted out of the parser handler,
the async resolver-cache tail no longer requires a module after Jest
teardown, so the HotTestCases suites can run on Deno again. Drop them
from the Deno job's ignore list.
GitHub Actions always sets `GITHUB_ACTIONS`, which short-circuits the
`isCI` OR before the GITLAB_CI/CIRCLECI checks, so those branches were
never executed (or covered) in CI. Clear the CI provider vars in
beforeEach so each provider test exercises its own branch.
Drop the engine-dependent stack field from Errors.test.js snapshots (JSC
renders Error.prototype.stack non-deterministically) and canonicalize the
JSON.parse SyntaxError wording, so the suite runs on Bun without skips.

Remove the now-dead Bun branch from the issue-11673 filter (the minimize+eval
combo never occurs in the test matrix) and the lazy-compilation/https filter
(the case passes on Bun).
The case-insensitive-filesystem branch is skipped on Linux, so the -u that
regenerated the other snapshots left its stack line in place; remove it to
match cleanError dropping stack on macOS/Windows.
@alexander-akait alexander-akait force-pushed the fix/color-support-and-runtime-test-skips branch from a24bdbb to a763cc1 Compare June 23, 2026 14:58
…ation/https

Both removals were wrong. issue-11673 throws "require is not defined" under
the minimized-source-map runner (minimize + eval-cheap-module-source-map),
which I missed when concluding no runner combines minimize with eval. The
lazy-compilation/https case is flaky on Bun ("No update available"): local
runs passed but CI hit the failure.
…f skipping Bun

Under load the dynamic import's HTTPS activation request can reach the
lazy-compilation backend after the first manual recompile, leaving the module
a proxy so the rebuild reports "No update available" (seen on Bun async-node).
The fixed 1000ms NEXT_DEFERRED delay only narrows the race. Add a NEXT_RETRY
helper that re-runs the same fake-update version, and retry the activation
update until it lands. Verified by injecting a backend activation delay: the
old test fails, the retrying one passes. Drops the Bun skip.
Deno/Bun: skip the loader-load-module-progress watch-cache case; their
persistent-cache snapshot timing is nondeterministic so the cached
ProgressPlugin counts read back stale (3 instead of 4).

Bun: run with --smol so the shared worker-threads heap is GC'd aggressively,
avoiding the ~2.85GB peak that crashed the runtime (segfault/OOM).
With Bun running the full suite (via --smol), this case still reports "No
update available" even after retrying the recompile: Bun never processes the
activation request to the lazy-compilation HTTPS backend, so retries can't
recover it. Restore the Bun-only skip; the retry still hardens Node/Deno.
@github-actions

Copy link
Copy Markdown
Contributor

Types Coverage

Coverage after merging fix/color-support-and-runtime-test-skips into main will be
99.35%
Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
bin
   webpack.js98.77%100%100%98.77%91
examples
   build-common.js100%100%100%100%
   buildAll.js100%100%100%100%
   examples.js100%100%100%100%
   template-common.js98.21%100%100%98.21%72
examples/custom-javascript-parser
   test.filter.js100%100%100%100%
examples/custom-javascript-parser/internals
   acorn-parse.js100%100%100%100%
   meriyah-parse.js100%100%100%100%
   oxc-parse.js91.30%100%100%91.30%140, 142–143, 145, 147, 153–154, 161, 168, 90
examples/markdown
   webpack.config.mjs100%100%100%100%
examples/module-federation
   test.filter.js100%100%100%100%
examples/reexport-components
   test.filter.js100%100%100%100%
examples/typescript
   test.filter.js100%100%100%100%
examples/typescript-non-erasable
   test.filter.js50%100%100%50%5
examples/virtual-modules
   test.filter.js100%100%100%100%
examples/wasm-bindgen-esm
   test.filter.js100%100%100%100%
examples/wasm-complex
   test.filter.js100%100%100%100%
examples/wasm-emscripten
   test.filter.js100%100%100%100%
examples/wasm-simple
   test.filter.js100%100%100%100%
examples/wasm-simple-source-phase
   test.filter.js100%100%100%100%
lib
   APIPlugin.js100%100%100%100%
   AsyncDependenciesBlock.js100%100%100%100%
   AutomaticPrefetchPlugin.js100%100%100%100%
   BannerPlugin.js100%100%100%100%
   Cache.js98.21%100%100%98.21%101
   CacheFacade.js100%100%100%100%
   Chunk.js99.72%100%100%99.72%39
   ChunkGraph.js100%100%100%100%
   ChunkGroup.js100%100%100%100%
   ChunkTemplate.js100%100%100%100%
   CircularModulesPlugin.js98.81%100%100%98.81%136
   CleanPlugin.js99.15%100%100%99.15%207, 227
   CodeGenerationResults.js100%100%100%100%
   CompatibilityPlugin.js100%100%100%100%
   Compilation.js98.43%100%100%98.43%1623, 1942, 1949, 1957, 1979, 1982, 2922, 3401–3402, 3434, 4100, 4130, 4183–4184, 4188, 4193, 4209–4210, 4224–4225, 4230–4231, 4708, 4734, 519, 524, 5542, 5574, 5591, 5607, 5623, 5638, 5663–5664, 5666, 5994, 5999, 6005, 6008, 6020, 6022, 6026, 6042, 6057, 6089, 6143, 6167, 6281, 769–770
   Compiler.js99.56%100%100%99.56%1142–1143, 1151
   ConcatenationScope.js98.59%100%100%98.59%189
   ConditionalInitFragment.js100%100%100%100%
   ConstPlugin.js100%100%100%100%
   ContextExclusionPlugin.js100%100%100%100%
   ContextModule.js100%100%100%100%
   ContextModuleFactory.js97.40%100%100%97.40%258, 395, 418, 420, 424, 433–434
   ContextReplacementPlugin.js100%100%100%100%
   DefinePlugin.js99%100%100%99%171–172, 188, 207, 281
   DependenciesBlock.js100%100%100%100%
   Dependency.js98.50%100%100%98.50%470, 516
   DependencyTemplate.js100%100%100%100%
   DependencyTemplates.js100%100%100%100%
   DotenvPlugin.js98.41%100%100%98.41%378, 391–392
   DynamicEntryPlugin.js100%100%100%100%
   EntryOptionPlugin.js100%100%100%100%
   EntryPlugin.js100%100%100%100%
   Entrypoint.js100%100%100%100%
   EnvironmentPlugin.js97.14%100%100%97.14%49
   ErrorHelpers.js100%100%100%100%
   EvalDevToolModulePlugin.js100%100%100%100%
   EvalSourceMapDevToolPlugin.js100%100%100%100%
   ExportsInfo.js100%100%100%100%
   ExportsInfoApiPlugin.js100%100%100%100%
   ExternalModule.js98.50%100%100%98.50%1057, 1060, 445–449, 451, 597
   ExternalModuleFactoryPlugin.js100%100%100%100%
   ExternalsPlugin.js100%100%100%100%
   FileSystemInfo.js99.52%100%100%99.52%182, 2354–2355, 2358, 2369, 2380, 2391, 278, 3795, 3810, 3834
   FlagAllModulesAsUsedPlugin.js100%100%100%100%
   FlagDependencyExportsPlugin.js98.42%100%100%98.42%413, 422, 424, 428
   FlagDependencyUsagePlugin.js100%100%100%100%
   FlagEntryExportAsUsedPlugin.js100%100%100%100%
   Generator.js100%100%100%100%
   HotModuleReplacementPlugin.js100%100%100%100%
   HotUpdateChunk.js100%100%100%100%
   IgnorePlugin.js100%100%100%100%
   IgnoreWarningsPlugin.js100%100%100%100%
   InitFragment.js100%100%100%100%
   JavascriptMetaInfoPlugin.js100%100%100%100%
   LazyBarrel.js100%100%100%100%
   LibraryTemplatePlugin.js100%100%100%100%
   LoaderOptionsPlugin.js100%100%100%100%
   LoaderTargetPlugin.js100%100%100%100%
   MainTemplate.js100%100%100%100%
   ManifestPlugin.js100%100%100%100%
   Module.js98.50%100%100%98.50%1285, 1290, 1350, 1364, 1426, 1435
   ModuleFactory.js100%100%100%100%
   ModuleFilenameHelpers.js98.85%100%100%98.85%106, 108
   ModuleGraph.js99.73%100%100%99.73%1005
   ModuleGraphConnection.js100%100%100%100%
   ModuleInfoHeaderPlugin.js100%100%100%100%
   ModuleNotFoundError.js100%100%100%100%
   ModuleProfile.js100%100%100%100%
   ModuleSourceTypeConstants.js100%100%100%100%
   ModuleTemplate.js100%100%100%100%
   ModuleTypeConstants.js100%100%100%100%
   MultiCompiler.js99.69%100%100%99.69%659
   MultiStats.js100%100%100%100%
   MultiWatching.js100%100%100%100%
   NoEmitOnErrorsPlugin.js100%100%100%100%
   NodeStuffPlugin.js100%100%100%100%
   NormalModule.js97.90%100%100%97.90%1237, 1240, 1257, 1274, 1521, 1555, 1571, 1658, 2014, 2313, 2318–2328, 418, 422, 576
   NormalModuleFactory.js99.47%100%100%99.47%1083, 1392, 486, 498
   NormalModuleReplacementPlugin.js100%100%100%100%
   NullFactory.js100%100%100%100%
   OptimizationStages.js100%100%100%100%
   OptionsApply.js100%100%100%100%
   Parser.js100%100%100%100%
   PlatformPlugin.js100%100%100%100%
   PrefetchPlugin.js100%100%100%100%
   ProgressPlugin.js98.85%100%100%98.85%519–520, 525, 527, 591
   ProvidePlugin.js100%100%100%100%
   RawModule.js100%100%100%100%
   RecordIdsPlugin.js100%100%100%100%
   RequestShortener.js100%100%100%100%
   ResolverFactory.js100%100%100%100%
   RuntimeGlobals.js100%100%100%100%
   RuntimeModule.js100%100%100%100%
   RuntimePlugin.js100%100%100%100%
   RuntimeTemplate.js100%100%100%100%
   SelfModuleFactory.js100%100%100%100%
   SingleEntryPlugin.js100%100%100%100%
   SourceMapDevToolModuleOptionsPlugin.js100%100%100%100%
   SourceMapDevToolPlugin.js98.62%100%100%98.62%220, 224, 226, 419, 430, 889
   Stats.js100%100%100%100%
   Template.js100%100%100%100%
   TemplatedPathPlugin.js99.38%100%100%99.38%295–296
   UseStrictPlugin.js100%100%100%100%
   WarnCaseSensitiveModulesPlugin.js100%100%100%100%
   WarnDeprecatedOptionPlugin.js100%100%100%100%
   WarnNoModeSetPlugin.js100%100%100%100%
   WatchIgnorePlugin.js100%100%100%100%
   Watching.js100%100%100%100%
   WebpackError.js100%100%100%100%
   WebpackIsIncludedPlugin.js100%100%100%100%
   WebpackOptionsApply.js100%100%100%100%
   WebpackOptionsDefaulter.js100%100%100%100%
   buildChunkGraph.js99.87%100%100%99.87%371
   cli.js98.63%100%100%98.63%10, 119, 549, 581, 631, 905
   index.js99.72%100%100%99.72%184
   validateSchema.js94.67%100%100%94.67%100, 87, 89, 98
   webpack.js96.33%100%100%96.33%10, 198, 220, 222
lib/asset
   AssetBytesGenerator.js100%100%100%100%
   AssetBytesParser.js100%100%100%100%
   AssetGenerator.js100%100%100%100%
   AssetModule.js100%100%100%100%
   AssetModulesPlugin.js97.32%100%100%97.32%281, 305, 308, 36, 360, 41
   AssetParser.js100%100%100%100%
   AssetSourceGenerator.js100%100%100%100%
   AssetSourceParser.js100%100%100%100%
   RawDataUrlModule.js100%100%100%100%
lib/async-modules
   AsyncModuleHelpers.js100%100%100%100%
   AwaitDependenciesInitFragment.js100%100%100%100%
   InferAsyncModulesPlugin.js100%100%100%100%
lib/bun
   BunTargetPlugin.js100%100%100%100%
lib/cache
   AddBuildDependenciesPlugin.js100%100%100%100%
   AddManagedPathsPlugin.js100%100%100%100%
   IdleFileCachePlugin.js97.92%100%100%97.92%71, 83, 91
   MemoryCachePlugin.js95.83%100%100%95.83%33
   MemoryWithGcCachePlugin.js93.15%100%100%93.15%106, 113–114, 122, 89
   PackFileCacheStrategy.js96.40%100%100%96.40%1250, 1350, 1354, 1416, 628, 647, 657–659, 661, 677–678, 683, 686, 688, 693, 698, 722, 728, 762, 768, 774, 779, 790, 799, 804–805, 807, 824, 830–831, 833
   ResolverCachePlugin.js100%100%100%100%
   getLazyHashedEtag.js100%100%100%100%
   mergeEtags.js100%100%100%100%
lib/config
   browserslistTargetHandler.js100%100%100%100%
   defaults.js99.33%100%100%99.33%1456–1458, 1466, 274,

@alexander-akait alexander-akait merged commit efeb5ce into main Jun 23, 2026
68 checks passed
@alexander-akait alexander-akait deleted the fix/color-support-and-runtime-test-skips branch June 23, 2026 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant