perf: reduce HTML parser memory and CPU with parser-level skip options#21323
Conversation
Add an opt-in `rawText` mode to `buildHtmlAst` used by `HtmlParser`: text nodes keep only their source offsets instead of a decoded/coalesced string, which the walk never reads (bodies are re-sliced from offsets). Tree- construction decisions still use the decoded value and merging mirrors the default path, so node structure and offsets are unchanged. Cuts HTML parse heap ~6% and time ~8% on text-heavy documents.
Align `HtmlParser` with `CssParser`: rather than calling `buildHtmlAst` and
passing a pre-built `ast`, it now calls `.process(source, { skipTextData: true })`
and the AST is built inside the processor's grammar. Rename the `rawText`
option to `skipTextData` and pass it through the process options object, like
the CSS `as` option. Node structure and offsets are unchanged.
Convert the HtmlParser unit tests to drive the real parser on real HTML
instead of mocking `buildHtmlAst`.
`options.ast` only existed so the old `HtmlParser` could pass a pre-built tree; with AST building moved inside the processor's grammar nothing supplies it, and the CSS `SourceProcessor` has no equivalent. `fragmentContext` stays: it is the HTML analog of the CSS `as` parse-mode option and is exercised by the html5lib fragment conformance tests.
Generalize the text-data optimization into a `skip` set on `buildHtmlAst` /
the HTML processor: `skip.text` (drop prose text, keep only raw-text element
bodies, offset-only), `skip.comments` (drop comment nodes) and `skip.doctype`
(drop the doctype node; quirks detection is unaffected). Skips are pure output
reductions — tree construction runs unchanged, so element structure and offsets
are identical with any combination.
`HtmlParser` enables `{ text, doctype }` (it reads script/style bodies by offset
and never reads prose text or the doctype); comments stay for `webpackIgnore`.
On a text-heavy 554 KiB document this cuts parser heap ~28% and time ~44%.
Add a buildHtmlAst test asserting every skip combination yields the same element
tree across construction edge cases (foster parenting, adoption agency, foreign
content, raw-text elements, quirks).
`text` was misleading — the flag drops only prose (flow-content) text and
keeps raw-text element bodies (`<script>`/`<style>`/`<textarea>`/`<title>`/…),
so `proseText` names what it actually does. `HtmlParser` now passes
`skip: { proseText: true, doctype: true }`.
Expand the skip guard tests: foreign-content CDATA, quirks (no-doctype),
entity/whitespace routing, all four raw-text body types, button scope, and
fragment-context parsing — asserting the element tree is identical under every
skip combination, and that all raw-text bodies survive offset-only.
…Text node Rename the misleading `proseText` to `text` (the DOM/WHATWG node type) and make it drop *every* Text node rather than keeping raw-text element bodies. A raw-text element's body span is instead recorded as its `contentEnd` offset, so `HtmlParser` reads `<script>`/`<style>` content as [`tagEnd`, `contentEnd`] without a Text node. Also rename `TEXT_CONTENT_ELEMENTS` to `RAW_TEXT_ELEMENTS` (the spec's raw text / escapable raw text elements). Memory is unchanged (~28%); element structure and offsets stay identical under any skip combination.
Under `skip.text` the text node is dropped, so construction only needs the run's whitespace-ness. For plain text — no entity, NUL, CR, or pending leading-newline swallow — the decoded value equals the raw range, so scan it once and dispatch a canonical whitespace marker instead of slicing and entity- decoding a string that is immediately discarded. Anything trickier falls through to the exact path unchanged. Cuts ~4-8% off parse time on text-heavy documents (more on markup with fewer entities). Element structure and offsets stay byte-identical; the skip guard tests add whitespace-entity / CR / NUL cases to lock in the fallbacks.
Void elements are inserted and immediately popped, so they never receive children — mkEl can hand them a shared frozen empty array instead of allocating one per element (~1200 arrays on a large document). Frozen so any accidental append throws rather than corrupting the shared instance. ~1% less parser heap; AST unchanged.
🦋 Changeset detectedLatest commit: a68f231 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
This PR is packaged and the instant preview is available (f7a3f6d). Install it locally:
npm i -D webpack@https://pkg.pr.new/webpack@f7a3f6d
yarn add -D webpack@https://pkg.pr.new/webpack@f7a3f6d
pnpm add -D webpack@https://pkg.pr.new/webpack@f7a3f6d |
Merging this PR will degrade performance by 35.6%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Memory | benchmark "asset-modules-inline", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' |
384.4 KB | 1,195.8 KB | -67.86% |
| ⚡ | Memory | benchmark "many-modules-esm", scenario '{"name":"mode-production","mode":"production"}' |
9.7 MB | 7.5 MB | +29.01% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing claude/html-parser-perf-compare-hcqefg (a68f231) with main (e848598)
There was a problem hiding this comment.
Pull request overview
This PR optimizes webpack’s experimental HTML parsing pipeline by allowing the HTML AST builder / SourceProcessor to skip emitting AST node kinds that the HtmlParser consumer doesn’t read, reducing allocations and parse overhead while keeping tree construction behavior the same.
Changes:
- Add
skipoptions tobuildHtmlAst/ HTMLSourceProcessorto omittext,comments, and/ordoctypenodes, and record raw-text body spans viacontentEnd. - Update
HtmlParserto parse viaSourceProcessor.process(...)with parser-levelskipoptions and to extract inline<script>/<style>bodies by offset ranges. - Update/extend unit tests and add a changeset entry for the optimization.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
lib/html/syntax.js |
Adds AST-skip options, contentEnd, shared empty children array, and changes grammar to build AST internally. |
lib/html/HtmlParser.js |
Switches to parser-level skip options and reads inline bodies via [tagEnd, contentEnd]. |
test/HtmlParser.unittest.js |
Converts tests to drive real HTML parsing end-to-end (no AST mocking). |
test/buildHtmlAst.unittest.js |
Adds coverage ensuring skip options preserve element structure/offsets and validates contentEnd behavior. |
.changeset/html-parser-drop-text-data.md |
Documents the performance optimization as a patch changeset. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| }; | ||
|
|
||
| walk(options.ast || buildHtmlAst(input, options.fragmentContext), null); | ||
| walk(buildHtmlAst(input, options.fragmentContext, options.skip), null); |
There was a problem hiding this comment.
Dropping ast was intentional (it was unused). The stale test that passed { ast } has been removed, so nothing relies on it anymore.
Generated by Claude Code
| /** | ||
| * @typedef {object} HtmlProcessOptions | ||
| * @property {string=} fragmentContext context element tag name for fragment parsing (see `buildHtmlAst`) | ||
| * @property {(HtmlDocument | HtmlDocumentFragment)=} ast walk this pre-built AST instead of parsing the input | ||
| * @property {string=} fragmentContext context element tag name for fragment parsing (see `buildHtmlAst`); the HTML analog of the CSS parser's `as` parse-mode option | ||
| * @property {HtmlAstSkip=} skip node kinds to omit from the AST for speed/memory (see `HtmlAstSkip`) | ||
| */ |
There was a problem hiding this comment.
ast is deliberately removed rather than kept for compat, and the only test that used it is gone, so the typedef stays free of it.
Generated by Claude Code
| // `skip.text` drops the body `Text` node; the raw content | ||
| // span is [`tagEnd`, `contentEnd`] on the element. | ||
| const bodyStart = node.tagEnd; | ||
| const bodyEnd = node.contentEnd; | ||
| if (bodyEnd <= bodyStart) continue; | ||
| value = source.slice(bodyStart, bodyEnd); |
There was a problem hiding this comment.
Good catch — the root cause was cloneSubtree not setting contentEnd. Fixed there so every element (cloned included) carries a numeric contentEnd, so bodyEnd can no longer be undefined here.
Generated by Claude Code
| // End of a raw-text element's body under `skip.text` (defaults to the | ||
| // body start, i.e. empty); lets consumers read `<script>`/`<style>` | ||
| // content as [`tagEnd`, `contentEnd`] without a `Text` node. | ||
| contentEnd: pos ? pos.tagEnd : 0, | ||
| // Present from creation (only `<template>` fills it) so every element | ||
| // keeps one monomorphic hidden class for the open-stack/scope loops. | ||
| templateContent: undefined |
There was a problem hiding this comment.
Done — cloneSubtree now sets contentEnd and clones templateContent (recursively) so cloned elements match mkEl's monomorphic shape.
Generated by Claude Code
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #21323 +/- ##
==========================================
- Coverage 92.86% 92.48% -0.39%
==========================================
Files 594 594
Lines 65210 65240 +30
Branches 18143 18154 +11
==========================================
- Hits 60559 60337 -222
- Misses 4651 4903 +252
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
cloneSubtree now sets contentEnd and clones templateContent so cloned elements match mkEl's shape and never expose an undefined contentEnd.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
Comments suppressed due to low confidence (1)
test/buildHtmlAst.unittest.js:837
- This multi-line comment is longer than the repo's guidance for comments in
test/(AGENTS.md:110 suggests 1 line, max 2 short lines). Consider shortening it to a compact statement of intent.
);
expect(child(child(t.children, "colgroup").children, "col")).toBeDefined();
expect(find("<table><colgroup>", "colgroup")).toBeDefined();
expect(
| // These tests drive the real tree builder end-to-end (like the CSS parser | ||
| // tests): `HtmlParser` builds the AST inside `SourceProcessor.process`, so | ||
| // there is nothing to mock — each case parses real HTML and asserts the | ||
| // extracted dependencies. |
There was a problem hiding this comment.
| // Skip the AST output this walk never reads: all `text` nodes (it reads | ||
| // `<script>`/`<style>` bodies by offset, via each element's `contentEnd`) | ||
| // and the `doctype` node. `comments` are NOT skipped — magic comments | ||
| // (`webpackIgnore`) need them. The processor builds the AST internally | ||
| // (like the CSS parser). |
There was a problem hiding this comment.
| // Shared frozen children array for void elements — they are inserted and | ||
| // immediately popped, so they never receive children. Frozen so any (buggy) | ||
| // append throws instead of corrupting the shared instance. |
There was a problem hiding this comment.
| // `skip.text` fast path: the node is dropped, so construction only needs | ||
| // the run's whitespace-ness (`isAllWs` / framesetOk). For plain text — | ||
| // no `&` (entities), `\0`, `\r`, or a pending leading-newline swallow — | ||
| // the decoded value equals the raw range, so scan it and dispatch a | ||
| // canonical marker (`" "` all-whitespace, `"x"` otherwise), skipping the | ||
| // per-run slice + entity decode. Anything trickier falls through. |
There was a problem hiding this comment.
Under skip.text, foreign-content <script>/<style> (e.g. SVG <style>) lost their body span because contentEnd was HTML-namespace only, while HtmlParser extracts those bodies regardless of namespace. Record it for every raw-text tag name. Also broaden the skip-combination guard, add coverage, and tighten comments.
Types CoverageCoverage after merging claude/html-parser-perf-compare-hcqefg into main will be
Coverage Report |
Summary
The experimental HTML parser (
experiments.html) built a full object-per-node DOM for every module and retained data the consumer never reads. This PR letsbuildHtmlAst/ the HTMLSourceProcessoromit AST node kinds a consumer doesn't need, via askipoption ({ text, comments, doctype }), and wiresHtmlParserto use it: it drops allTextnodes (reading<script>/<style>bodies by offset via a newcontentEnd) and the doctype node, while keeping comments for magic comments. It also aligns the HTML parser entry with the CSS parser (the AST is built inside.process(...), so the consumer no longer callsbuildHtmlAstor passes a pre-builtast— that unused process option is removed), adds a plain-text fast path underskip.textthat avoids a slice + entity-decode of a string that is immediately discarded, and shares one frozenchildrenarray for void elements. Net on text-heavy documents: ~28% less parser heap and ~10%+ less CPU, with parse results unchanged.What kind of change does this PR introduce?
perf (with the supporting refactor of the parser's process API).
Did you add tests for your changes?
Yes.
test/buildHtmlAst.unittest.jsgains a "skip options preserve element structure" block asserting the element tree + offsets are identical under everyskipcombination across construction edge cases (foster parenting, adoption agency, foreign content, quirks, and theskip.textwhitespace fast-path fallbacks — character references, CR, NUL).test/HtmlParser.unittest.jswas converted to drive the real parser on real HTML instead of mockingbuildHtmlAst.Does this PR introduce a breaking change?
No. The changes are internal to the experimental HTML support; the
skipoptions are opt-in and defaultbuildHtmlAstoutput is byte-identical (verified against the previous behavior).If relevant, what needs to be documented once your changes are merged or what have you already documented?
n/a — internal parser change, no public config surface.
Use of AI
Yes. AI (Claude) was used to profile the parser, prototype and measure each optimization, and draft the implementation and tests. Every change was verified with byte-identical-AST comparisons against the prior behavior and the added guard tests; the full jest/html5lib/tsc/lint suites should be confirmed green by CI on this PR (they could not be run in the authoring environment).
Generated by Claude Code