Skip to content

feat: fork static export#415

Merged
lazarv merged 1 commit intomainfrom
feat/fork-static-export
Apr 30, 2026
Merged

feat: fork static export#415
lazarv merged 1 commit intomainfrom
feat/fork-static-export

Conversation

@lazarv
Copy link
Copy Markdown
Owner

@lazarv lazarv commented Apr 30, 2026

The static exporter has been rebuilt around a streaming path source and an opt-in multi-process render coordinator. Before this change, exporting a large site materialised the full path list into an array, then ran every render in Promise.all on a single thread. At ~24k pages of Shiki-rendered HTML the resident set climbed into the multi-gigabyte range even though only a handful of pages were live at any instant, and runs would OOM long before they finished. Sites with non-trivial server-component work were also bottlenecked on a single RSC thread regardless of available CPU.

The new pipeline splits that work in two. A shared streaming layer (buildPathStream, pMapStream, fanout) consumes path sources lazily — options.exportPaths and configRoot.export can now be async generators or any AsyncIterable, and the renderer pulls one path per free worker, so peak memory is O(concurrency × chunkSize) regardless of total path count. Layered on top of that, a forked coordinator can spread the render across N child processes, each running its own RSC main thread and SSR worker thread, dispatching paths over IPC. Every artifact (HTML, gz/br sidecars, .postponed.json, .prerender-cache.json) is written to disk inside the child, so output bytes never cross the IPC boundary; the coordinator only ferries small log entries back. The two modes produce an identical artifact set — postpone and prerender-cache work in multi-process mode because each child uses the same setupStaticRender + emitAllArtifacts path the single-process exporter does.

The number of render workers is exposed as a new --export-concurrency flag (and an exportConcurrency field in react-server.config.mjs), defaulting to a CPU-scaled value between 2 and 4. Setting it to 1 collapses to the single-process in-line exporter — cheapest for small sites and useful for debugging. Documentation has been added in both English and Japanese covering the flag itself and the documented "level up" pattern for config.export as an async function* for cases where the path list is too large to materialise (CMS pagination, large database queries, tens of thousands of file-router slugs).

A handful of related bugs surfaced and were fixed alongside the rewrite. The backpressure PR had moved prerenderInit after the static handlers in lib/start/create-server.mjs, which broke the PPR resume flow because lib/handlers/static.mjs mutates PrerenderStorage when it serves a .postponed.json sidecar; the middleware is moved back ahead of the static handlers. The static-worker child's fatal() was racing process.exit(1) against an unflushed IPC send, so worker crashes appeared to the parent as bare exit codes with no underlying error — fixed by waiting on the send callback before exiting. The renderer worker URL in static-runtime.mjs was incorrect (./render-stream.mjs resolved into lib/build/); it now resolves correctly into lib/start/. The Postponed control signal that React's prerender machinery throws at dynamic boundaries was being printed as a red error stack by the static exporter's logger proxy; it's now filtered by the REACT_SERVER_POSTPONED digest. Per-path render failures are once again printed in red on stderr above the summary line — the streaming refactor had silently dropped the error log while keeping the count. The compression default is restored to the documented false (a previous attempt to fix "compressed sizes not shown" in multi-process had inadvertently flipped the default on); the real fix lives in the IPC plumbing instead. And the prerender-disabled check in emitHtml now gates the entire postpone/prerender-cache machinery (the callback wiring, the cache Set allocation, the sidecar emission) on a single computed prerenderEnabled, derived from both the per-path and config-level prerender state — previously a per-path prerender: false flowed into render but still installed the onPostponed callback.

Tests cover the new pipeline at three layers. Unit tests exercise each streaming primitive in isolation — pMapStream for bounded concurrency and lazy pulls, toPathStream for input-shape normalisation, buildPathStream for the generator/array branch split (including a passthrough-then-append pattern that mirrors the docs and a fail-fast error-propagation case), validatedPathStream for descriptor validation, and fanout for backpressured one-chunk-in-flight delivery. A second describe asserts the load-bearing structural invariant — pulled - completed <= concurrency — at a modest N where the assertion is N-independent but cheap. The end-to-end at-scale coverage lives in a new fixture spec under test-build-start: test/fixtures/static-export-many/ is a two-file fixture (a single-component entry plus a react-server.config.mjs whose export is an async function* yielding STATIC_EXPORT_MANY_COUNT paths, defaulting to 100k), and the spec runs the full build + serve round-trip and spot-checks four sample paths. The harness's vitestSetup.mjs now derives options.export from a serialisable initialConfig.export flag so a spec can opt into static export without touching internal build options; the actual generator function still lives on disk because functions can't cross the JSON boundary into the build worker.

@cloudflare-workers-and-pages
Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
react-server-docs c713a0f Apr 30 2026, 10:51 PM

@github-actions
Copy link
Copy Markdown

⚡ Flight Protocol Benchmark

Commit: 5ece51b

Serialization (renderToReadableStream)

Scenario @lazarv/rsc webpack vs webpack
react: minimal element 228.6K 29.2K 🟢 +682.2%
react: shallow wide (1000) 2.1K 373 🟢 +471.5%
react: deep nested (100) 17.2K 6.0K 🟢 +188.3%
react: product list (50) 6.1K 1.9K 🟢 +219.1%
react: large table (500x10) 280 91 🟢 +205.9%
data: primitives 180.4K 33.6K 🟢 +437.5%
data: large string (100KB) 7.0K 7.0K ⚪ -0.8%
data: nested objects (20) 56.5K 26.3K 🟢 +114.9%
data: large array (10K) 118 105 🟢 +11.7%
data: Map & Set 10.8K 5.9K 🟢 +81.9%
data: Date/BigInt/Symbol 166.5K 35.9K 🟢 +363.7%
data: typed arrays 33.2K 12.5K 🟢 +165.8%
data: mixed payload 8.0K 4.1K 🟢 +95.1%

Prerender (prerender)

Scenario @lazarv/rsc ops/s mean
react: minimal element 258.8K 3.9 µs
react: shallow wide (1000) 2.0K 494.1 µs
react: deep nested (100) 16.2K 61.9 µs
react: product list (50) 5.6K 177.7 µs
react: large table (500x10) 272 3.68 ms
data: primitives 191.5K 5.2 µs
data: large string (100KB) 690 1.45 ms
data: nested objects (20) 56.6K 17.7 µs
data: large array (10K) 116 8.65 ms
data: Map & Set 11.3K 88.9 µs
data: Date/BigInt/Symbol 185.8K 5.4 µs
data: typed arrays 665 1.50 ms
data: mixed payload 7.6K 131.6 µs

Deserialization (createFromReadableStream)

Scenario @lazarv/rsc webpack vs webpack
react: minimal element 167.3K 137.1K 🟢 +22.0%
react: shallow wide (1000) 19.9K 2.0K 🟢 +916.6%
react: deep nested (100) 101.3K 18.4K 🟢 +451.4%
react: product list (50) 52.2K 14.5K 🟢 +258.7%
react: large table (500x10) 4.2K 2.0K 🟢 +111.3%
data: primitives 138.6K 129.9K 🟢 +6.8%
data: large string (100KB) 40.2K 30.5K 🟢 +31.6%
data: nested objects (20) 84.3K 70.2K 🟢 +20.0%
data: large array (10K) 279 246 🟢 +13.4%
data: Map & Set 16.8K 14.4K 🟢 +16.6%
data: Date/BigInt/Symbol 138.1K 110.0K 🟢 +25.6%
data: typed arrays 55.5K 42.8K 🟢 +29.6%
data: mixed payload 25.4K 14.9K 🟢 +70.5%

Roundtrip (serialize + deserialize)

Scenario @lazarv/rsc webpack vs webpack
react: minimal element 105.5K 21.8K 🟢 +385.0%
react: shallow wide (1000) 1.8K 271 🟢 +562.4%
react: deep nested (100) 14.7K 4.2K 🟢 +246.9%
react: product list (50) 5.4K 1.7K 🟢 +225.4%
react: large table (500x10) 266 84 🟢 +218.3%
data: primitives 81.9K 25.4K 🟢 +222.3%
data: large string (100KB) 5.9K 7.0K 🔴 -16.9%
data: nested objects (20) 34.0K 18.2K 🟢 +86.8%
data: large array (10K) 78 72 🟢 +8.4%
data: Map & Set 6.1K 4.0K 🟢 +54.1%
data: Date/BigInt/Symbol 71.1K 22.6K 🟢 +214.4%
data: typed arrays 25.1K 11.2K 🟢 +123.8%
data: mixed payload 5.9K 3.0K 🟢 +96.3%
Legend & methodology

Indicators: 🟢 > 1% faster | 🔴 > 1% slower | ⚪ within noise margin

vs webpack: compares @lazarv/rsc against react-server-dom-webpack within the same run.
vs baseline: compares @lazarv/rsc against the previous main branch run.

Values shown are operations/second (higher is better). Each scenario runs for at least 100 iterations with warmup.

Benchmarks run on GitHub Actions runners (shared infrastructure) — expect ~5% variance between runs. Consistent directional changes across multiple scenarios are more meaningful than any single number.

@codecov-commenter
Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
841 1 840 19
View the top 1 failed test(s) by shortest run time
__test__/scroll-restoration.spec.mjs > scroll restoration: back navigation restores scroll position
Stack Traces | 24.4s run time
AssertionError: expected 0 to be greater than 700
 ❯ __test__/scroll-restoration.spec.mjs:126:21

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@github-actions
Copy link
Copy Markdown

⚡ Benchmark Results

PR c713a0f main 32e9d20
Config 50 connections, 10s/test 50 connections, 10s/test
Benchmark Req/s vs main Avg Latency vs main P99 Latency Throughput
minimal 1251 🔴 -12.4% 39.26 ms 🔴 +14.1% 75 ms 0.9 MB/s
small 1309 🔴 -6.4% 37.6 ms 🔴 +7.0% 66 ms 1.3 MB/s
medium 389 🟢 +3.7% 126.37 ms 🟢 -4.0% 176 ms 5.8 MB/s
large 46 🟢 +7.0% 1051.46 ms 🟢 -3.8% 2004 ms 4.6 MB/s
deep 871 ⚪ +0.4% 56.72 ms ⚪ -0.4% 89 ms 3.0 MB/s
wide 72 🟢 +11.2% 667.82 ms 🟢 -8.0% 1296 ms 3.9 MB/s
cached 3533 🟢 +11.2% 13.54 ms 🟢 -10.9% 28 ms 52.1 MB/s
client-min 456 🔴 -10.9% 108.67 ms 🔴 +12.5% 166 ms 2.0 MB/s
client-small 461 🔴 -15.0% 107.58 ms 🔴 +18.2% 152 ms 2.2 MB/s
client-med 366 🔴 -5.8% 134.94 ms 🔴 +6.0% 225 ms 6.8 MB/s
client-large 87 🟢 +18.3% 567.41 ms 🟢 -12.9% 1006 ms 9.1 MB/s
client-deep 442 🔴 -9.9% 111.89 ms 🔴 +11.2% 165 ms 3.2 MB/s
client-wide 140 🟢 +13.3% 349.59 ms 🟢 -10.6% 633 ms 8.2 MB/s
rsc-client-large 1039 🔴 -15.4% 47.41 ms 🔴 +18.4% 68 ms 2.7 MB/s
rsc-client-wide 1064 🔴 -16.0% 46.29 ms 🔴 +19.1% 64 ms 2.8 MB/s
static-json 7629 🔴 -31.6% 6.17 ms 🔴 +63.7% 16 ms 3.2 MB/s
static-js 7255 🔴 -33.5% 6.4 ms 🔴 +66.2% 16 ms 9.1 MB/s
404-miss 4817 🔴 -23.1% 9.83 ms 🔴 +30.9% 22 ms 0.6 MB/s
hybrid-min 440 🔴 -16.0% 112.42 ms 🔴 +18.9% 168 ms 2.1 MB/s
hybrid-small 438 🔴 -12.5% 112.99 ms 🔴 +14.3% 171 ms 2.6 MB/s
hybrid-medium 235 🔴 -5.5% 210.55 ms 🔴 +5.9% 305 ms 10.0 MB/s
hybrid-large 41 🟢 +6.1% 1110.8 ms 🟢 -3.6% 1912 ms 13.3 MB/s
hybrid-deep 354 🔴 -8.3% 138.59 ms 🔴 +9.0% 201 ms 4.9 MB/s
hybrid-wide 63 🟢 +15.5% 763.58 ms 🟢 -10.0% 1418 ms 12.5 MB/s
hybrid-cached 2993 🔴 -1.8% 16.17 ms 🔴 +1.8% 31 ms 127.4 MB/s
hybrid-client-min 472 🔴 -13.8% 104.51 ms 🔴 +15.7% 154 ms 2.1 MB/s
hybrid-client-small 477 🔴 -15.1% 103.88 ms 🔴 +17.9% 157 ms 2.3 MB/s
hybrid-client-medium 357 🔴 -5.9% 137.7 ms 🔴 +5.4% 207 ms 6.7 MB/s
hybrid-client-large 85 🟢 +8.3% 566.43 ms 🟢 -8.0% 968 ms 8.9 MB/s
hybrid-client-deep 437 🔴 -11.1% 113.25 ms 🔴 +12.7% 174 ms 3.2 MB/s
hybrid-client-wide 144 🟢 +20.0% 342.18 ms 🟢 -15.6% 623 ms 8.4 MB/s
Legend

🟢 > 1% improvement | 🔴 > 1% regression | ⚪ within noise margin

Benchmarks run on GitHub Actions runners (shared infrastructure) — expect ~5% variance between runs. Consistent directional changes across multiple routes are more meaningful than any single number.

@lazarv lazarv merged commit 9af9b44 into main Apr 30, 2026
102 of 103 checks passed
@lazarv lazarv deleted the feat/fork-static-export branch April 30, 2026 23:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants