Reduce skipped tests across Node, Deno and Bun and prune dead WebAssembly filters#21257
Conversation
🦋 Changeset detectedLatest commit: 1f7b402 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
This PR is packaged and the instant preview is available (efeb5ce). Install it locally:
npm i -D webpack@https://pkg.pr.new/webpack@efeb5ce
yarn add -D webpack@https://pkg.pr.new/webpack@efeb5ce
pnpm add -D webpack@https://pkg.pr.new/webpack@efeb5ce |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #21257 +/- ##
=======================================
Coverage 92.80% 92.81%
=======================================
Files 591 591
Lines 64658 64729 +71
Branches 18012 18042 +30
=======================================
+ Hits 60007 60075 +68
- Misses 4651 4654 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will not alter performance
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Memory | benchmark "asset-modules-inline", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' |
389.5 KB | 1,230.9 KB | -68.36% |
| ❌ | Memory | benchmark "wasm-modules-sync", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' |
123.6 KB | 360.1 KB | -65.69% |
| ⚡ | Memory | benchmark "side-effects-reexport", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' |
859.1 KB | 136.7 KB | ×6.3 |
| ⚡ | Memory | benchmark "many-modules-esm", scenario '{"name":"mode-development","mode":"development"}' |
1.9 MB | 1.2 MB | +65.89% |
| ⚡ | Memory | benchmark "many-modules-commonjs", scenario '{"name":"mode-production","mode":"production"}' |
9.7 MB | 7 MB | +38.84% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing fix/color-support-and-runtime-test-skips (1f7b402) with main (ecc00c0)
The `supportsWebAssembly()` guard is true on every supported runtime (Node >= 10.13, Deno, Bun), so those `test.filter.js` files never skip anything. Remove the 31 pure guards plus the always-true `supportsOptionalCatchBinding()` guard, and drop the dead `supportsWebAssembly()` conjunct from 9 combined filters. Fix the long-disabled `cases/wasm/v128` fixture: its `.wasm` encoded a `v128.load` with no memory section (CompileError: memory instruction with no memory), so it was hard-disabled with `() => false`. Replace it with a valid v128 module and gate the case on SIMD support like the sibling cases.
The filter used `/^v2[2-9]/.test(process.version)`, which only matches Node v22–v29 and silently skips the case on Node 30 and later. Replace it with a numeric `major >= 22` check, matching the sibling `json-import-attributes` filter for the same `with` syntax.
`isColorSupported()` probed the environment with the `in` operator (`"FORCE_COLOR" in env`, `"CI" in env`, …). Deno's and Bun's `process.env` shims honor property access but not the `in`/`has` trap, so those signals were missed and the CLI emitted no colors. Read the values directly instead — identical on Node, and works on Deno and Bun. With detection fixed, the "colors by default" test no longer needs a runtime skip on Deno and Bun.
The malformed-JSON tests asserted V8's exact SyntaxError wording and positions, so they were skipped on Bun (JavaScriptCore emits different text). Assert the exact wording only on V8 (Node, Deno) and check the engine-agnostic JSONParseError contract (name, systemError, numeric position, non-empty message) on every runtime, and drop the Bun skip.
Both were skipped on Bun for engine-specific error text: MultiItemCache asserted V8's exact "not a function" TypeError wording, and AbstractMethodError asserted a caller name parsed out of V8's Error.stack format. Assert the engine-agnostic contract instead — the error type for MultiItemCache, and the base message on Bun with the exact caller-info message kept on V8 (Node, Deno) — and drop the skips.
Add cases for `--color` and `--no-color` in process.argv, the two branches of the color detection that no test exercised.
The parser handler ran `require("../../package.json")` lazily, so the
async resolver-cache tail could trigger it after a test tore down the
module registry (the "require after environment torn down" crash that
forced HotTestCases to be skipped on Deno). Load the version once at
module scope, like DefaultStatsFactoryPlugin already does.
With the import.meta version require hoisted out of the parser handler, the async resolver-cache tail no longer requires a module after Jest teardown, so the HotTestCases suites can run on Deno again. Drop them from the Deno job's ignore list.
GitHub Actions always sets `GITHUB_ACTIONS`, which short-circuits the `isCI` OR before the GITLAB_CI/CIRCLECI checks, so those branches were never executed (or covered) in CI. Clear the CI provider vars in beforeEach so each provider test exercises its own branch.
Drop the engine-dependent stack field from Errors.test.js snapshots (JSC renders Error.prototype.stack non-deterministically) and canonicalize the JSON.parse SyntaxError wording, so the suite runs on Bun without skips. Remove the now-dead Bun branch from the issue-11673 filter (the minimize+eval combo never occurs in the test matrix) and the lazy-compilation/https filter (the case passes on Bun).
The case-insensitive-filesystem branch is skipped on Linux, so the -u that regenerated the other snapshots left its stack line in place; remove it to match cleanError dropping stack on macOS/Windows.
a24bdbb to
a763cc1
Compare
…ation/https
Both removals were wrong. issue-11673 throws "require is not defined" under
the minimized-source-map runner (minimize + eval-cheap-module-source-map),
which I missed when concluding no runner combines minimize with eval. The
lazy-compilation/https case is flaky on Bun ("No update available"): local
runs passed but CI hit the failure.
…f skipping Bun Under load the dynamic import's HTTPS activation request can reach the lazy-compilation backend after the first manual recompile, leaving the module a proxy so the rebuild reports "No update available" (seen on Bun async-node). The fixed 1000ms NEXT_DEFERRED delay only narrows the race. Add a NEXT_RETRY helper that re-runs the same fake-update version, and retry the activation update until it lands. Verified by injecting a backend activation delay: the old test fails, the retrying one passes. Drops the Bun skip.
Deno/Bun: skip the loader-load-module-progress watch-cache case; their persistent-cache snapshot timing is nondeterministic so the cached ProgressPlugin counts read back stale (3 instead of 4). Bun: run with --smol so the shared worker-threads heap is GC'd aggressively, avoiding the ~2.85GB peak that crashed the runtime (segfault/OOM).
With Bun running the full suite (via --smol), this case still reports "No update available" even after retrying the recompile: Bun never processes the activation request to the lazy-compilation HTTPS backend, so retries can't recover it. Restore the Bun-only skip; the retry still hardens Node/Deno.
Types CoverageCoverage after merging fix/color-support-and-runtime-test-skips into main will be
Coverage Report |
Summary
Reduces skipped/filtered tests and aligns behavior across Node, Deno and Bun:
supportsWebAssembly()/supportsOptionalCatchBinding()test filters (true on Node >= 10.13, Deno and Bun) and simplifies 9 combined ones; re-enablescases/wasm/v128, whose.wasmfixture was invalid (av128.loadwith no memory section).externals/import-attributesnode-version gate/^v2[2-9]/, which silently skipped on Node 30+, tomajor >= 22(matchingjson-import-attributes).isColorSupported()to detect color on Deno and Bun: it probedprocess.envwith theinoperator, which theirprocess.envshims do not honor; reading the values is identical on Node and works on all three runtimes.parseJson,MultiItemCacheandAbstractMethodErrortests by asserting the engine-agnostic contract on Bun (JSC error text/stack differ from V8) while keeping the exact assertions on Node and Deno.What kind of change does this PR introduce?
fix (CLI color detection) plus test (filter pruning and cross-runtime test fixes).
Did you add tests for your changes?
Yes — re-enabled
test/cases/wasm/v128; the color fix is covered byCli.basictest(isColorSupportedand the colors blocks); the remaining changes are themselves test changes.Does this PR introduce a breaking change?
No.
If relevant, what needs to be documented once your changes are merged or what have you already documented?
n/a
Use of AI
Yes — used Claude Code to audit the test filters and runtime skips, identify stale/over-restrictive ones, implement the changes and verify them on Node. The Deno and Bun paths are verified by construction (engine-agnostic assertions) and confirmed by the runtimes CI job.
Generated by Claude Code