perf: reduce CPU and memory overhead of the HTML and CSS pipelines#21332
Merged
Conversation
Parsers: batch-scan HTML comment bodies, single-slice named-entity lookup, offset-based (formerly quadratic) srcset tokenizing, one lazy attribute-map closure per parse; CSS visitors drop per-node lowercase copies and per-ident slices when no ICSS definitions exist, and the streaming parser releases its retained source between parses. Generators/plugins: build one dependency-template context per module instead of per dependency, memoize the css inheritance chain for a reference-equal render cache check, skip [webpack/auto] substitution and re-hashing when HTML content has no placeholder, and avoid the throwaway exports object + JSON string in CssGenerator.getSize. Claude-Session: https://claude.ai/code/session_01FSWqLu3mk6zXFtLURntBGJ
Flatten CssIcssExportDependency entry locations to four numbers (the nested loc objects were the parser's hottest allocation and are retained on the dependency for the compilation lifetime), read source positions through the LocConverter cursor instead of allocating A.loc objects, drop the substring allocation from LocConverter's forward scan, memoize composes self-reference resolution per parse, avoid lowercased copies of already-lowercase keywords and at-rule names, and stop scanning incoming connections in CssGenerator#getTypes once the answer can't change. Measured on a 210-module CSS + HTML fixture build: retained heap -3MB, peak heap -5%, sampled parser allocations at the targeted sites -40..80%.
Answer 'is this a known property' through an ASCII-case-folded hash index over the known-properties table and compare composes/from/global keywords directly against source ranges, so the common declaration never slices its property name; the ICSS name slice happens only when @value definitions exist. Behavior is unchanged, including vendor-prefixed and custom-property paths, which keep the string-based route.
…location Line-level profiling of the streaming CSS parser showed skipped component values still paying a full SoA slot write before being dropped, and every leaf clearing flag/list slots that only containers read. Skipped leaf tokens now short-circuit in the two skip-checked consume loops without building a node, leaf allocation is down to three array writes (containers clear their own slots), and value/function-arg whitespace joins the non-CSS-Modules skip set — consumers already tolerate its absence via nextNonWhitespace-style checks. Non-modules parse of a 3MB stylesheet is ~9% faster; full-AST parses ~5%.
…rsers CSS: cache the peeked token as a boolean flag instead of an object slot (saves a GC write barrier per token — the hottest line in the stream), table-drive the whitespace-run loop, and gate the skip checks behind a _skipActive flag so no-skip parses pay one boolean test per value. HTML: replace the name-intern Map with an open-addressed table (one or two array reads per tag/attribute name), hoist the entity-decode replace callbacks so decoding allocates no closure, and classify the skip.text scan through a lookup table instead of per-char comparison chains. HTML AST build 4-7% faster, entity-heavy documents ~5%, CSS parse ~2%.
🦋 Changeset detectedLatest commit: 9dc99c0 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
This PR is packaged and the instant preview is available (ae28c54). Install it locally:
npm i -D webpack@https://pkg.pr.new/webpack@ae28c54
yarn add -D webpack@https://pkg.pr.new/webpack@ae28c54
pnpm add -D webpack@https://pkg.pr.new/webpack@ae28c54 |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #21332 +/- ##
========================================
Coverage 92.64% 92.64%
========================================
Files 594 594
Lines 65758 65923 +165
Branches 18269 18321 +52
========================================
+ Hits 60921 61075 +154
- Misses 4837 4848 +11
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Addresses the code-quality bot: the do/while body always assigns before the first read.
Merging this PR will degrade performance by 27.09%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Memory | benchmark "asset-modules-source", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' |
191 KB | 513.7 KB | -62.81% |
| ❌ | Memory | benchmark "many-modules-esm", scenario '{"name":"mode-development","mode":"development"}' |
1.2 MB | 1.9 MB | -38.45% |
| ❌ | Memory | benchmark "wasm-modules-async", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' |
193.1 KB | 244.9 KB | -21.15% |
| ⚡ | Memory | benchmark "css-modules", scenario '{"name":"mode-development","mode":"development"}' |
1,332.5 KB | 850.7 KB | +56.64% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing perf/html-css-memory (9dc99c0) with main (e6fb547)
Contributor
There was a problem hiding this comment.
Pull request overview
This PR applies profile-driven performance optimizations to webpack’s experimental HTML and CSS pipelines (tokenizers/parsers, generators, and related plugins), primarily targeting reduced allocations, reduced retained compilation state, and faster hot-path scanning.
Changes:
- Refactors HTML/CSS generators to reuse a single
DependencyTemplateContextper module and avoid repeatedly creating per-dependency context state. - Optimizes HTML tokenization/entity decoding, srcset parsing, and HTML module post-processing to reduce repeated slicing, closure allocations, and unnecessary hashing.
- Optimizes CSS parsing/tokenization and CSS Modules rendering paths to reduce per-node/per-token overhead and avoid deep equality checks via memoized structures.
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| types.d.ts | Updates generated typings to reflect adjusted generator method signatures. |
| lib/util/LocConverter.js | Reworks forward scanning to avoid substring allocations on location computations. |
| lib/html/syntax.js | Adds multiple tokenizer/entity/srcset/name-interning fast paths to reduce allocations and repeated slicing. |
| lib/html/HtmlParser.js | Makes attribute-map decoding lazy and parse-scoped to avoid per-element closure allocation. |
| lib/html/HtmlModulesPlugin.js | Avoids unnecessary placeholder splitting and reuses hashes when content is unchanged. |
| lib/html/HtmlGenerator.js | Lazily loads CSS plugin and reuses a single template context across dependency applications. |
| lib/dependencies/CssIcssExportDependency.js | Flattens per-export location storage (numbers) and materializes loc objects only when needed. |
| lib/css/syntax.js | Optimizes whitespace scanning, token caching, skip handling, and clears retained SoA state after parses. |
| lib/css/CssParser.js | Avoids per-visit slicing/lowercasing where possible; adds memoization for repeated checks. |
| lib/css/CssModulesPlugin.js | Memoizes module inheritance chains to avoid per-render materialization and deep comparisons. |
| lib/css/CssGenerator.js | Reuses a single template context per module and reduces repeated work in type detection. |
| .changeset/html-css-perf-memory.md | Adds a patch changeset entry for the performance improvements. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
34
to
38
| const input = this._input; | ||
| let i = input.indexOf("\n", this.pos); | ||
| if (i === -1 || i >= pos) { | ||
| this.column += pos - this.pos; | ||
| } else { |
Member
Author
There was a problem hiding this comment.
Good catch — fixed in c261a8e. Went with a charCodeAt loop over exactly [this.pos, pos) rather than the suggested backwards lastIndexOf, since a backwards scan from pos is equally unbounded on newline-free input (it would walk to the string start). The loop is strictly delta-bounded, allocation-free, and verified with a 40k-position fuzz against the original algorithm plus a 50k-small-advance run over a newline-free 4 MB source (22 ms, linear).
Generated by Claude Code
Member
Author
|
Regarding the CodSpeed-flagged Generated by Claude Code |
…ypings Scan exactly [pos, target) with charCodeAt instead of indexOf — a forward indexOf could run to end-of-input on newline-free minified sources and turn repeated small advances quadratic (Copilot review). Also update the CssIcssExportDependency unit test entries to the flattened location fields (lint job's test typecheck).
| * @param {InitFragment<GenerateContext>[]} initFragments mutable list of init fragments | ||
| * @param {ReplaceSource} source the current replace source which can be modified | ||
| * @param {GenerateContext & { cssData: CssData }} generateContext the render context | ||
| * @param {DependencyTemplateContext & { cssData: CssData }} templateContext the template context (shared across all dependencies of the module) |
Member
Author
There was a problem hiding this comment.
Right that the contract was missing type — fixed in 32d34bc by widening the inline intersection to DependencyTemplateContext & { cssData: CssData, type: string } (same shape as CssDependencyTemplateContext). Kept the inline form rather than referencing the typedef so the emitted types.d.ts surface stays a local, predictable diff.
Generated by Claude Code
Comment on lines
+32
to
+35
| // Advance: scan exactly [this.pos, pos) — no substring allocation, | ||
| // and strictly bounded by the delta (an `indexOf`-based scan could | ||
| // run to end-of-input on newline-free minified sources and turn | ||
| // repeated small advances quadratic). |
Comment on lines
+1060
to
+1062
| // Lazy decoded-attribute map for the element currently being entered — | ||
| // parse-scoped so the hot Element visitor doesn't allocate a closure | ||
| // (and memo slot) per element. |
Comment on lines
+1304
to
+1306
| // Fast-forward over the run of ordinary comment text without | ||
| // re-entering the per-state switch; stop on the significant | ||
| // code points handled above. |
Comment on lines
+3713
to
+3715
| // The two `replace` callbacks are hoisted (one per `isAttribute` mode) so a | ||
| // decode call doesn't allocate a fresh closure and the callback stays | ||
| // monomorphic for the regex engine. |
|
|
||
| /** | ||
| * 2-token lookahead: is the next non-whitespace pair `<ident> <colon>` (the prerequisite for consume-a-declaration steps 1 + 3)? Used by `consumeABlocksContents` to skip a declaration attempt that would otherwise call consume-the-remnants-of-a-bad-declaration and be undone by `restoreMark`. A webpack fast-path, not a spec algorithm — so the implementation is free to peek cheaply. It scans raw code points after the cached ident for the next significant one (skipping whitespace + comments) rather than tokenizing them, and never advances the stream: the ident stays cached in `_next` for `consumeADeclaration` to reuse instead of re-tokenizing the property name. Comments skipped here fire `onComment` later, when the chosen consume algorithm tokenizes past them (once, in source order, as before). | ||
| * 2-token lookahead: is the next non-whitespace pair `<ident> <colon>` (the prerequisite for consume-a-declaration steps 1 + 3)? Used by `consumeABlocksContents` to skip a declaration attempt that would otherwise call consume-the-remnants-of-a-bad-declaration and be undone by `restoreMark`. A webpack fast-path, not a spec algorithm — so the implementation is free to peek cheaply. It scans raw code points after the cached ident for the next significant one (skipping whitespace + comments) rather than tokenizing them, and never advances the stream: the ident stays cached (`_hasNext`) for `consumeADeclaration` to reuse instead of re-tokenizing the property name. Comments skipped here fire `onComment` later, when the chosen consume algorithm tokenizes past them (once, in source order, as before). |
Comment on lines
+32
to
+35
| // Advance: scan exactly [this.pos, pos) — no substring allocation, | ||
| // and strictly bounded by the delta (an `indexOf`-based scan could | ||
| // run to end-of-input on newline-free minified sources and turn | ||
| // repeated small advances quadratic). |
Comment on lines
+1060
to
+1062
| // Lazy decoded-attribute map for the element currently being entered — | ||
| // parse-scoped so the hot Element visitor doesn't allocate a closure | ||
| // (and memo slot) per element. |
Comment on lines
+136
to
+140
| /** | ||
| * The effective inheritance chain (own layer/supports/media plus inherited | ||
| * entries) derives only from module fields fixed at build time, so build the | ||
| * array once per module; `renderModule`'s cache check can then compare by | ||
| * reference instead of re-materializing and deep-comparing it per render. |
Comment on lines
+7985
to
+7987
| // (Tracked as a start offset into `input` — `-1` = empty — and sliced | ||
| // once per descriptor instead of appended to char-by-char.) | ||
| descriptorStart = -1; |
|
|
||
| /** | ||
| * 2-token lookahead: is the next non-whitespace pair `<ident> <colon>` (the prerequisite for consume-a-declaration steps 1 + 3)? Used by `consumeABlocksContents` to skip a declaration attempt that would otherwise call consume-the-remnants-of-a-bad-declaration and be undone by `restoreMark`. A webpack fast-path, not a spec algorithm — so the implementation is free to peek cheaply. It scans raw code points after the cached ident for the next significant one (skipping whitespace + comments) rather than tokenizing them, and never advances the stream: the ident stays cached in `_next` for `consumeADeclaration` to reuse instead of re-tokenizing the property name. Comments skipped here fire `onComment` later, when the chosen consume algorithm tokenizes past them (once, in source order, as before). | ||
| * 2-token lookahead: is the next non-whitespace pair `<ident> <colon>` (the prerequisite for consume-a-declaration steps 1 + 3)? Used by `consumeABlocksContents` to skip a declaration attempt that would otherwise call consume-the-remnants-of-a-bad-declaration and be undone by `restoreMark`. A webpack fast-path, not a spec algorithm — so the implementation is free to peek cheaply. It scans raw code points after the cached ident for the next significant one (skipping whitespace + comments) rather than tokenizing them, and never advances the stream: the ident stays cached (`_hasNext`) for `consumeADeclaration` to reuse instead of re-tokenizing the property name. Comments skipped here fire `onComment` later, when the chosen consume algorithm tokenizes past them (once, in source order, as before). |
Comment on lines
500
to
+504
| exportType: /** @type {ExportType} */ (read()), | ||
| loc: /** @type {DependencyLocation=} */ (read()) | ||
| locStartLine: /** @type {number} */ (read()), | ||
| locStartColumn: /** @type {number} */ (read()), | ||
| locEndLine: /** @type {number} */ (read()), | ||
| locEndColumn: /** @type {number} */ (read()) |
Member
Author
There was a problem hiding this comment.
This can't occur with webpack's persistent cache: cache.buildDependencies.defaultWebpack defaults to webpack's own lib/ directory (lib/config/defaults.js), so any webpack upgrade fails the build-dependency snapshot and the whole pack is discarded before deserialization — old-format entries are never fed to the new deserialize(). Serialization shapes change between releases under this invariant routinely, so no legacy decode path is needed.
Generated by Claude Code
Comment on lines
+2553
to
+2556
| * 2-token lookahead: is the next non-whitespace pair `<ident> <colon>`? | ||
| * Fast-path (not a spec algorithm) peeking raw code points without advancing | ||
| * the stream — the ident stays cached for `consumeADeclaration`, and skipped | ||
| * comments still fire `onComment` later in source order. |
Comment on lines
+2553
to
+2556
| * 2-token lookahead: is the next non-whitespace pair `<ident> <colon>`? | ||
| * Fast-path (not a spec algorithm) peeking raw code points without advancing | ||
| * the stream — the ident stays cached for `consumeADeclaration`, and skipped | ||
| * comments still fire `onComment` later in source order. |
Comment on lines
+614
to
+616
| // `hasCssText` only affects the exports-only / non-link result sets, so | ||
| // the common link case can stop scanning connections once js is seen. | ||
| // This runs uncached per call via `getReferencedSourceTypes` (#20800). |
Comment on lines
+3201
to
+3203
| // Slice the candidate run from `input` once; prefixes are taken | ||
| // from this short string instead of re-slicing the (potentially | ||
| // huge) input per length. |
Comment on lines
+3591
to
+3593
| // Property names are analyzed by range — the common declaration | ||
| // (unknown property, no composes anchor, no `@value`s) never | ||
| // slices its name out of the source. |
Comment on lines
+1811
to
+1813
| // Leaves never read the flag / list slots, so those are cleared by | ||
| // `_soaAllocContainer` only — leaves dominate, and this keeps their | ||
| // allocation at three array writes. |
Comment on lines
+141
to
+156
| const moduleInheritanceCache = new WeakMap(); | ||
|
|
||
| /** | ||
| * Gets the memoized inheritance chain of a css module. | ||
| * @param {CssModule} module css module | ||
| * @returns {Inheritance} inheritance chain including the module's own entry | ||
| */ | ||
| const getModuleInheritance = (module) => { | ||
| let inheritance = moduleInheritanceCache.get(module); | ||
| if (inheritance === undefined) { | ||
| inheritance = [[module.cssLayer, module.supports, module.media]]; | ||
| if (module.inheritance) inheritance.push(...module.inheritance); | ||
| moduleInheritanceCache.set(module, inheritance); | ||
| } | ||
| return inheritance; | ||
| }; |
Member
Author
There was a problem hiding this comment.
Good catch — fixed in 9dc99c0. getModuleInheritance now stores the source fields alongside the chain and revalidates all four by reference on each lookup, so a updateCacheModule reassignment invalidates the memo (and the renderModule fast path correctly misses on the new chain object). Verified with the css integration and watch suites.
Generated by Claude Code
updateCacheModule reassigns cssLayer/supports/media/inheritance on the cached CssModule instance, so an instance-keyed memo could serve a stale chain on watch rebuilds.
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Profile-driven optimization of the experimental HTML and CSS pipelines (parsers, generators, plugins), in six behavior-preserving commits: fewer allocations on per-token/per-node/per-export hot paths, less retained per-compilation state, and faster tokenizer cores. Measured: HTML AST build −10–15% CPU and ~−50% GC pause, srcset parsing −15%, CSS Modules export-path allocations −40–80%, ~3 MB less heap retained per compilation on a 210-module fixture build.
What kind of change does this PR introduce?
perf
Did you add tests for your changes?
No new tests — the changes are behavior-preserving and covered by the existing css/html unit, integration, and snapshot suites (all green at every commit).
Does this PR introduce a breaking change?
No.
CssIcssExportDependencyentries serialize per-export locations as four numbers instead of a nested object, which only affects the version-keyed persistent cache layout.If relevant, what needs to be documented once your changes are merged or what have you already documented?
n/a
Use of AI
Developed with AI assistance (Claude Code): CPU/heap profiling, benchmarking, implementing the optimizations, and running the verification suites; every change was reviewed against profiler evidence and validated by the existing tests.
Generated by Claude Code