chore(bench): add browser benchmark harness via Vitest + Playwright#18261
Open
MathiasWP wants to merge 2 commits into
Open
chore(bench): add browser benchmark harness via Vitest + Playwright#18261MathiasWP wants to merge 2 commits into
MathiasWP wants to merge 2 commits into
Conversation
Adds a real-browser performance benchmark harness for code paths where JSDOM
diverges from production engines (DOM operations, attribute writes, layout-
triggering work, transitions). Uses Vitest's bench() API in browser mode with
Playwright-driven headless Chromium; tinybench handles warmup, adaptive
iteration, and sample collection.
Bench-author API is intentionally minimal — describe what to measure, with
optional setup/teardown:
bench('label', () => { ...hot loop... }, { setup, teardown });
Each bench reports hz, mean, p99, rme, and sample count. The existing JSDOM
benchmark suite in benchmarking/benchmarks/ is unchanged and remains the right
home for reactivity-graph perf (kairo, sbench, derived/effects), where JSDOM
gives correct relative ordering.
New files:
- benchmarking/browser/vitest.config.js
- benchmarking/browser/benches/{html_swap,create_element}.bench.js
- benchmarking/browser/README.md (when to use, how to write benches)
New dev dep: @vitest/browser ~2.1.9. playwright was already installed.
Run:
pnpm bench:browser # all
pnpm bench:browser swap # filter
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
4 tasks
MathiasWP
added a commit
to MathiasWP/svelte
that referenced
this pull request
May 21, 2026
Mirrors what the HTML branch does. The trailing `undefined` argument to `createElementNS` hits a measurably slower path in V8/Blink than the bare 2-arg form (~10% slower for SVG creation in the benches in sveltejs#18261). SVG/MathML creation is much colder than HTML, but the fix is one line and makes the wrapper symmetric across both namespaces. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
MathiasWP
added a commit
to MathiasWP/svelte
that referenced
this pull request
May 21, 2026
The prior pass diverged on first mismatch and then ran LIS over the entire suffix `[i_start, length)` — fine for big reorders (reverse, shuffle), but expensive on small ones where most of the tail was actually stable. In a real-browser benchmark a single far swap regressed ~27% vs the legacy heuristic because the LIS bookkeeping (Map allocations, the LIS sequence itself, chain rebuild) dominated for an O(2) move workload. Walk forward AND backward to identify the stable prefix and suffix in lockstep, then run LIS only over the unstable middle. Falls back to the legacy heuristic when the walks expose an add/remove (length mismatch in the middle), an INERT/offscreen entry, or a non-branch sibling effect — same eligibility rules as before. Measured in real Chromium (Chromium 145) via the browser bench harness in sveltejs#18261: | | baseline | prior LIS | new LIS | vs heuristic | | ----------- | -------: | --------: | -------: | -------------------: | | swap (far) | ~115k | ~84k | ~129k | **+12%** (was -27%) | | reverse | ~4,800 | ~30,900 | ~33,400 | **+595%** (was +544%) | | shuffle | ~20,000 | ~25,000 | ~28,400 | **+42%** (was +25%) | A small refactor pulls the flag eligibility check into `is_eligible_branch` since it now appears in three places (prefix walk, suffix walk, and the old-chain walk in the middle pass). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
MathiasWP
added a commit
to MathiasWP/svelte
that referenced
this pull request
May 21, 2026
These benchmarks belong with the dedicated browser benchmark infrastructure in sveltejs#18261, not bundled into this PR. The real-browser numbers in the PR description come from that harness. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rich-Harris
pushed a commit
that referenced
this pull request
May 28, 2026
…18262) ## Summary The current wrapper always calls `document.createElementNS(namespace ?? NAMESPACE_HTML, tag, options)` — even for HTML elements (the >99% case), and even when `options` would be `undefined`. Two effects compound: 1. **Route HTML elements through `createElement`** — Blink has a fast path that skips the namespace lookup `createElementNS` always performs. 2. **Omit the trailing `undefined` argument** — V8/Blink take a slower path for `createElementNS(ns, tag, undefined)` (and `createElement(tag, undefined)`) than for the bare 2-arg form. This applies symmetrically to the SVG/MathML branch, where the wrapper now also avoids the `undefined` 3rd arg. The wrapper dispatches to the fastest call shape for every input — `{HTML, non-HTML}` × `{with is, without is}` × no `undefined` ever. ## Affects Every place Svelte constructs a DOM element internally: `<svelte:element>`, `run_scripts`, the per-component `<style>` injector, the `{@html}` wrapper, and the `<template>` element used to clone string templates. ## Numbers Measured in headless Chromium (Chromium 145) using the browser bench harness from #18261, 2–3 runs, median. Lower per-call is better; higher hz is better. ### Raw call shapes (what each shape costs in the browser) | | hz | per-call | | ------------------------------------------- | -------: | -------: | | `createElement(tag)` | ~1,210k | ~0.83 µs | | `createElement(tag, undefined)` | ~901k | ~1.11 µs | | `createElementNS(NS_HTML, tag)` | ~913k | ~1.10 µs | | `createElementNS(NS_HTML, tag, undefined)` | ~630k | ~1.59 µs | | `createElement(tag, { is })` | ~484k | ~2.07 µs | | `createElementNS(SVG_NS, tag)` | ~333k | ~3.01 µs | | `createElementNS(SVG_NS, tag, undefined)` | ~303k | ~3.30 µs | | `createElementNS(SVG_NS, tag, { is })` | ~245k | ~4.08 µs | Two stable effects fall out: - **Trailing `undefined` is consistently slower** than the bare form — ~26% on `createElement`, ~31% on `createElementNS` (HTML), ~10% on `createElementNS` (SVG). - **`createElement` skips a namespace lookup** that `createElementNS` always performs — ~32% delta for equal-shape calls (`createElement(tag)` vs `createElementNS(NS_HTML, tag)`). ### Per-case impact of this PR | Case (namespace, `is`) | Old wrapper call | Old hz | New wrapper call | New hz | Speedup | | ----------------------------- | --------------------------------------------- | --------: | ------------------------------ | --------: | ------: | | HTML, no `is` (dominant path) | `createElementNS(NS_HTML, tag, undefined)` | ~630k | `createElement(tag)` | ~1,210k | **~92%** (1.92×) | | HTML, with `is` | `createElementNS(NS_HTML, tag, { is })` | (slow) | `createElement(tag, { is })` | ~484k | similar to above (just the NS-skip) | | SVG/MathML, no `is` | `createElementNS(ns, tag, undefined)` | ~303k | `createElementNS(ns, tag)` | ~333k | **~10%** | | SVG/MathML, with `is` | `createElementNS(ns, tag, { is })` | ~245k | same | ~245k | no change | No measurable change in JSDOM (both shapes route through the same JS implementation there). The headline gain — **~92% on the dominant HTML-no-`is` path** — combines both effects roughly equally: dropping the `undefined` (~46%) and switching to `createElement` (~32%). Wrapper-vs-raw was also verified: the new wrapper measured to within ±2% of raw `document.createElement(tag)` for the HTML-no-`is` case, so the function-call indirection adds no measurable overhead. See #18261 for the benchmark and methodology. ## Test plan - [x] All 5881 runtime tests still pass (runtime-runes + runtime-legacy) - [x] `<svelte:element xmlns={null}>` correctly falls back to HTML (covered by existing `dynamic-element-dynamic-namespace` test) - [x] SVG/MathML namespaces still go through `createElementNS` - [x] Custom-element `is` option still honoured on both branches 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Vitest 4 (now on main) changed the browser-mode config API: - `provider: 'playwright'` string → `playwright()` factory from the new `@vitest/browser-playwright` package - single `browser.name` field → `browser.instances` array (multi-browser ready) Also: - bump `@vitest/browser` 2.1.9 → 4.1.8 and align the whole vitest family on 4.1.8 (browser packages must match vitest core exactly) - drop the `optimizeDeps.include` workaround — it failed to resolve esm-env/clsx as bare specifiers and is unnecessary under Vitest 4's browser dep optimizer (cold runs no longer trigger a reload) - document running benches across multiple engines via `instances` Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a real-browser performance benchmark harness for code paths where JSDOM disagrees with production engines. Uses Vitest's
bench()API in browser mode (headless Chromium via Playwright); tinybench handles warmup, adaptive iteration, and sample collection. The existing JSDOM benchmark suite is unchanged — it stays the right home for reactivity-graph perf (kairo, sbench, derived/effects) where JSDOM is faithful.Why
JSDOM is a pure-JS DOM implementation. For DOM-heavy operations (creation, removal, attribute writes, layout-triggering work, transitions) it can disagree with real engines by an order of magnitude or more — sometimes inverting the verdict on a perf change. One concrete example that motivated this:
createElement(tag)vscreateElementNS(NAMESPACE_HTML, tag). JSDOM showed ≈0% difference (both delegate to the same internal path). Real Chromium showscreateElementis ~89% faster — there's a documented fast path in Blink that skips namespace lookup. Measured here, 3 runs, median:createElementNS('http://www.w3.org/1999/xhtml', tag)(current)createElement(tag)(proposed fast path)JSDOM hid this win entirely. Without the harness we'd miss real wins like this.
Authoring API
Intentionally minimal — describe what to measure, with optional setup/teardown. No manual warmup, trial collection, or statistics:
Output:
Running
First run downloads headless Chromium (~250 MB, cached in
~/Library/Caches/ms-playwright/). If Playwright complains the executable is missing:What's in scope vs. what isn't
In: the harness itself + two seed benchmarks that demonstrate the workload shape (
html_swap,create_element). Both serve as living examples for contributors. The README documents when to add a browser bench vs. when to stick with JSDOM.Out: any actual perf change to
packages/svelte/src/. The harness is generic infrastructure; perf changes are separate PRs.What's next
bench:browser:comparescript (run twice across branches, diff hz). For now: run on main, switch branch, run again, compare.instancesarray makes this trivial.Test plan
pnpm bench:browserruns both seed benchmarks and produces stable output (rme < 5%)🤖 Generated with Claude Code