feat(node-streams): wire primitives into render pipeline (7/8)#89860
feat(node-streams): wire primitives into render pipeline (7/8)#89860feedthejim wants to merge 1 commit intofeedthejim/node-stream-03-primitivesfrom
Conversation
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
635593b to
ba0aa8d
Compare
bf6e861 to
5d447ac
Compare
ba0aa8d to
4f8168c
Compare
5d447ac to
99fdd5a
Compare
4f8168c to
c4c989d
Compare
99fdd5a to
99bd282
Compare
c4c989d to
9efcb95
Compare
c7f7e91 to
f7db683
Compare
8813630 to
4199831
Compare
f7db683 to
0665da4
Compare
Stats from current PR🔴 3 regressions
📊 All Metrics📖 Metrics GlossaryDev Server Metrics:
Build Metrics:
Change Thresholds:
⚡ Dev Server
📦 Dev Server (Webpack) (Legacy)📦 Dev Server (Webpack)
⚡ Production Builds
📦 Production Builds (Webpack) (Legacy)📦 Production Builds (Webpack)
📦 Bundle SizesBundle Sizes⚡ TurbopackClient Main Bundles: **397 kB** → **397 kB**
|
| Canary | PR | Change | |
|---|---|---|---|
| middleware-b..fest.js gzip | 757 B | 763 B | ✓ |
| Total | 757 B | 763 B |
Build Details
Build Manifests
| Canary | PR | Change | |
|---|---|---|---|
| _buildManifest.js gzip | 452 B | 449 B | ✓ |
| Total | 452 B | 449 B | ✅ -3 B |
📦 Webpack
Client
Main Bundles
| Canary | PR | Change | |
|---|---|---|---|
| 5528-HASH.js gzip | 5.47 kB | N/A | - |
| 6280-HASH.js gzip | 57.1 kB | N/A | - |
| 6335.HASH.js gzip | 169 B | N/A | - |
| 912-HASH.js gzip | 4.53 kB | N/A | - |
| e8aec2e4-HASH.js gzip | 62.6 kB | N/A | - |
| framework-HASH.js gzip | 59.7 kB | 59.7 kB | ✓ |
| main-app-HASH.js gzip | 254 B | 254 B | ✓ |
| main-HASH.js gzip | 39.1 kB | 39.1 kB | ✓ |
| webpack-HASH.js gzip | 1.68 kB | 1.68 kB | ✓ |
| 262-HASH.js gzip | N/A | 4.53 kB | - |
| 2889.HASH.js gzip | N/A | 169 B | - |
| 5602-HASH.js gzip | N/A | 5.48 kB | - |
| 6948ada0-HASH.js gzip | N/A | 62.6 kB | - |
| 9544-HASH.js gzip | N/A | 57.7 kB | - |
| Total | 231 kB | 231 kB |
Polyfills
| Canary | PR | Change | |
|---|---|---|---|
| polyfills-HASH.js gzip | 39.4 kB | 39.4 kB | ✓ |
| Total | 39.4 kB | 39.4 kB | ✓ |
Pages
| Canary | PR | Change | |
|---|---|---|---|
| _app-HASH.js gzip | 194 B | 194 B | ✓ |
| _error-HASH.js gzip | 183 B | 180 B | 🟢 3 B (-2%) |
| css-HASH.js gzip | 331 B | 330 B | ✓ |
| dynamic-HASH.js gzip | 1.81 kB | 1.81 kB | ✓ |
| edge-ssr-HASH.js gzip | 256 B | 256 B | ✓ |
| head-HASH.js gzip | 351 B | 352 B | ✓ |
| hooks-HASH.js gzip | 384 B | 383 B | ✓ |
| image-HASH.js gzip | 580 B | 581 B | ✓ |
| index-HASH.js gzip | 260 B | 260 B | ✓ |
| link-HASH.js gzip | 2.5 kB | 2.5 kB | ✓ |
| routerDirect..HASH.js gzip | 320 B | 319 B | ✓ |
| script-HASH.js gzip | 386 B | 386 B | ✓ |
| withRouter-HASH.js gzip | 315 B | 315 B | ✓ |
| 1afbb74e6ecf..834.css gzip | 106 B | 106 B | ✓ |
| Total | 7.97 kB | 7.97 kB | ✅ -2 B |
Server
Edge SSR
| Canary | PR | Change | |
|---|---|---|---|
| edge-ssr.js gzip | 126 kB | 126 kB | ✓ |
| page.js gzip | 249 kB | 251 kB | 🔴 +2.41 kB (+1%) |
| Total | 374 kB | 377 kB |
Middleware
| Canary | PR | Change | |
|---|---|---|---|
| middleware-b..fest.js gzip | 617 B | 614 B | ✓ |
| middleware-r..fest.js gzip | 156 B | 155 B | ✓ |
| middleware.js gzip | 43.4 kB | 44 kB | 🔴 +573 B (+1%) |
| edge-runtime..pack.js gzip | 842 B | 842 B | ✓ |
| Total | 45 kB | 45.6 kB |
Build Details
Build Manifests
| Canary | PR | Change | |
|---|---|---|---|
| _buildManifest.js gzip | 715 B | 718 B | ✓ |
| Total | 715 B | 718 B |
Build Cache
| Canary | PR | Change | |
|---|---|---|---|
| 0.pack gzip | 3.87 MB | 3.92 MB | 🔴 +54.7 kB (+1%) |
| index.pack gzip | 103 kB | 105 kB | 🔴 +1.52 kB (+1%) |
| index.pack.old gzip | 103 kB | 103 kB | ✓ |
| Total | 4.07 MB | 4.13 MB |
🔄 Shared (bundler-independent)
Runtimes
| Canary | PR | Change | |
|---|---|---|---|
| app-page-exp...dev.js gzip | 316 kB | 322 kB | 🔴 +6.02 kB (+2%) |
| app-page-exp..prod.js gzip | 167 kB | 171 kB | 🔴 +3.41 kB (+2%) |
| app-page-tur...dev.js gzip | 315 kB | 321 kB | 🔴 +6 kB (+2%) |
| app-page-tur..prod.js gzip | 167 kB | 170 kB | 🔴 +3.42 kB (+2%) |
| app-page-tur...dev.js gzip | 312 kB | 318 kB | 🔴 +5.89 kB (+2%) |
| app-page-tur..prod.js gzip | 165 kB | 169 kB | 🔴 +3.45 kB (+2%) |
| app-page.run...dev.js gzip | 312 kB | 318 kB | 🔴 +5.93 kB (+2%) |
| app-page.run..prod.js gzip | 165 kB | 169 kB | 🔴 +3.44 kB (+2%) |
| app-route-ex...dev.js gzip | 70.5 kB | 70.6 kB | ✓ |
| app-route-ex..prod.js gzip | 49 kB | 49.1 kB | ✓ |
| app-route-tu...dev.js gzip | 70.5 kB | 70.6 kB | ✓ |
| app-route-tu..prod.js gzip | 49 kB | 49.1 kB | ✓ |
| app-route-tu...dev.js gzip | 70.1 kB | 70.2 kB | ✓ |
| app-route-tu..prod.js gzip | 48.8 kB | 48.9 kB | ✓ |
| app-route.ru...dev.js gzip | 70.1 kB | 70.2 kB | ✓ |
| app-route.ru..prod.js gzip | 48.7 kB | 48.9 kB | ✓ |
| dist_client_...dev.js gzip | 324 B | 324 B | ✓ |
| dist_client_...dev.js gzip | 326 B | 326 B | ✓ |
| dist_client_...dev.js gzip | 318 B | 318 B | ✓ |
| dist_client_...dev.js gzip | 317 B | 317 B | ✓ |
| pages-api-tu...dev.js gzip | 43.2 kB | 50.6 kB | 🔴 +7.38 kB (+17%) |
| pages-api-tu..prod.js gzip | 32.9 kB | 37.6 kB | 🔴 +4.73 kB (+14%) |
| pages-api.ru...dev.js gzip | 43.2 kB | 50.5 kB | 🔴 +7.38 kB (+17%) |
| pages-api.ru..prod.js gzip | 32.8 kB | 37.6 kB | 🔴 +4.73 kB (+14%) |
| pages-turbo....dev.js gzip | 52.5 kB | 59.9 kB | 🔴 +7.43 kB (+14%) |
| pages-turbo...prod.js gzip | 38.4 kB | 43.2 kB | 🔴 +4.78 kB (+12%) |
| pages.runtim...dev.js gzip | 52.5 kB | 59.9 kB | 🔴 +7.43 kB (+14%) |
| pages.runtim..prod.js gzip | 38.4 kB | 43.2 kB | 🔴 +4.78 kB (+12%) |
| server.runti..prod.js gzip | 63.5 kB | 69.2 kB | 🔴 +5.65 kB (+9%) |
| app-page-exp...dev.js gzip | N/A | 325 kB | - |
| app-page-exp..prod.js gzip | N/A | 173 kB | - |
| app-page-nod...dev.js gzip | N/A | 321 kB | - |
| app-page-nod..prod.js gzip | N/A | 171 kB | - |
| app-page-tur...dev.js gzip | N/A | 324 kB | - |
| app-page-tur..prod.js gzip | N/A | 173 kB | - |
| app-page-tur...dev.js gzip | N/A | 321 kB | - |
| app-page-tur..prod.js gzip | N/A | 171 kB | - |
| app-route-ex...dev.js gzip | N/A | 77.8 kB | - |
| app-route-ex..prod.js gzip | N/A | 53.6 kB | - |
| app-route-no...dev.js gzip | N/A | 77.4 kB | - |
| app-route-no..prod.js gzip | N/A | 53.4 kB | - |
| app-route-tu...dev.js gzip | N/A | 77.8 kB | - |
| app-route-tu..prod.js gzip | N/A | 53.6 kB | - |
| app-route-tu...dev.js gzip | N/A | 77.4 kB | - |
| app-route-tu..prod.js gzip | N/A | 53.4 kB | - |
| dist_client_...dev.js gzip | N/A | 332 B | - |
| dist_client_...dev.js gzip | N/A | 324 B | - |
| dist_client_...dev.js gzip | N/A | 334 B | - |
| dist_client_...dev.js gzip | N/A | 326 B | - |
| Total | 2.8 MB | 5.39 MB |
📝 Changed Files (45 files)
Files with changes:
app-page-exp..ntime.dev.jsapp-page-exp..time.prod.jsapp-page-exp..ntime.dev.jsapp-page-exp..time.prod.jsapp-page-nod..ntime.dev.jsapp-page-nod..time.prod.jsapp-page-tur..ntime.dev.jsapp-page-tur..time.prod.jsapp-page-tur..ntime.dev.jsapp-page-tur..time.prod.jsapp-page-tur..ntime.dev.jsapp-page-tur..time.prod.jsapp-page-tur..ntime.dev.jsapp-page-tur..time.prod.jsapp-page.runtime.dev.jsapp-page.runtime.prod.jsapp-route-ex..ntime.dev.jsapp-route-ex..time.prod.jsapp-route-ex..ntime.dev.jsapp-route-ex..time.prod.js- ... and 25 more
View diffs
app-page-exp..ntime.dev.js
failed to diffapp-page-exp..time.prod.js
Diff too large to display
app-page-exp..ntime.dev.js
failed to diffapp-page-exp..time.prod.js
failed to diffapp-page-nod..ntime.dev.js
failed to diffapp-page-nod..time.prod.js
Diff too large to display
app-page-tur..ntime.dev.js
failed to diffapp-page-tur..time.prod.js
Diff too large to display
app-page-tur..ntime.dev.js
failed to diffapp-page-tur..time.prod.js
failed to diffapp-page-tur..ntime.dev.js
failed to diffapp-page-tur..time.prod.js
Diff too large to display
app-page-tur..ntime.dev.js
failed to diffapp-page-tur..time.prod.js
failed to diffapp-page.runtime.dev.js
failed to diffapp-page.runtime.prod.js
failed to diffapp-route-ex..ntime.dev.js
Diff too large to display
app-route-ex..time.prod.js
Diff too large to display
app-route-ex..ntime.dev.js
Diff too large to display
app-route-ex..time.prod.js
Diff too large to display
app-route-no..ntime.dev.js
Diff too large to display
app-route-no..time.prod.js
Diff too large to display
app-route-tu..ntime.dev.js
Diff too large to display
app-route-tu..time.prod.js
Diff too large to display
app-route-tu..ntime.dev.js
Diff too large to display
app-route-tu..time.prod.js
Diff too large to display
app-route-tu..ntime.dev.js
Diff too large to display
app-route-tu..time.prod.js
Diff too large to display
app-route-tu..ntime.dev.js
Diff too large to display
app-route-tu..time.prod.js
Diff too large to display
app-route.runtime.dev.js
Diff too large to display
app-route.ru..time.prod.js
Diff too large to display
dist_client_..ntime.dev.js
@@ -0,0 +1,2 @@
+"use strict";exports.ids=["dist_client_dev_noop-turbopack-hmr_js"],exports.modules={"./dist/client/dev/noop-turbopack-hmr.js"(module,exports1){function connect(){}Object.defineProperty(exports1,"__esModule",{value:!0}),Object.defineProperty(exports1,"connect",{enumerable:!0,get:function(){return connect}}),("function"==typeof exports1.default||"object"==typeof exports1.default&&null!==exports1.default)&&void 0===exports1.default.__esModule&&(Object.defineProperty(exports1.default,"__esModule",{value:!0}),Object.assign(exports1.default,exports1),module.exports=exports1.default)}};
+//# sourceMappingURL=dist_client_dev_noop-turbopack-hmr_js-experimental-nodestreams.runtime.dev.js.map
\ No newline at end of filedist_client_..ntime.dev.js
@@ -0,0 +1,2 @@
+"use strict";exports.ids=["dist_client_dev_noop-turbopack-hmr_js"],exports.modules={"./dist/client/dev/noop-turbopack-hmr.js"(module,exports1){function connect(){}Object.defineProperty(exports1,"__esModule",{value:!0}),Object.defineProperty(exports1,"connect",{enumerable:!0,get:function(){return connect}}),("function"==typeof exports1.default||"object"==typeof exports1.default&&null!==exports1.default)&&void 0===exports1.default.__esModule&&(Object.defineProperty(exports1.default,"__esModule",{value:!0}),Object.assign(exports1.default,exports1),module.exports=exports1.default)}};
+//# sourceMappingURL=dist_client_dev_noop-turbopack-hmr_js-nodestreams.runtime.dev.js.map
\ No newline at end of filedist_client_..ntime.dev.js
@@ -0,0 +1,2 @@
+"use strict";exports.ids=["dist_client_dev_noop-turbopack-hmr_js"],exports.modules={"./dist/client/dev/noop-turbopack-hmr.js"(module,exports1){function connect(){}Object.defineProperty(exports1,"__esModule",{value:!0}),Object.defineProperty(exports1,"connect",{enumerable:!0,get:function(){return connect}}),("function"==typeof exports1.default||"object"==typeof exports1.default&&null!==exports1.default)&&void 0===exports1.default.__esModule&&(Object.defineProperty(exports1.default,"__esModule",{value:!0}),Object.assign(exports1.default,exports1),module.exports=exports1.default)}};
+//# sourceMappingURL=dist_client_dev_noop-turbopack-hmr_js-turbo-experimental-nodestreams.runtime.dev.js.map
\ No newline at end of filedist_client_..ntime.dev.js
@@ -0,0 +1,2 @@
+"use strict";exports.ids=["dist_client_dev_noop-turbopack-hmr_js"],exports.modules={"./dist/client/dev/noop-turbopack-hmr.js"(module,exports1){function connect(){}Object.defineProperty(exports1,"__esModule",{value:!0}),Object.defineProperty(exports1,"connect",{enumerable:!0,get:function(){return connect}}),("function"==typeof exports1.default||"object"==typeof exports1.default&&null!==exports1.default)&&void 0===exports1.default.__esModule&&(Object.defineProperty(exports1.default,"__esModule",{value:!0}),Object.assign(exports1.default,exports1),module.exports=exports1.default)}};
+//# sourceMappingURL=dist_client_dev_noop-turbopack-hmr_js-turbo-nodestreams.runtime.dev.js.map
\ No newline at end of filepages-api-tu..ntime.dev.js
Diff too large to display
pages-api-tu..time.prod.js
Diff too large to display
pages-api.runtime.dev.js
Diff too large to display
pages-api.ru..time.prod.js
Diff too large to display
pages-turbo...ntime.dev.js
Diff too large to display
pages-turbo...time.prod.js
Diff too large to display
pages.runtime.dev.js
Diff too large to display
pages.runtime.prod.js
Diff too large to display
server.runtime.prod.js
Diff too large to display
4199831 to
932773c
Compare
0665da4 to
43acaf9
Compare
| const bridge = new PassThrough() | ||
| streams.push(bridge) | ||
|
|
||
| const combined = chainNodeStreams(...streams) |
There was a problem hiding this comment.
Instead of spreading maybe we can just pass the array here. Since we control both functions.
43acaf9 to
6fa3940
Compare
932773c to
3579d00
Compare
3579d00 to
4ccf817
Compare
6fa3940 to
8628e0b
Compare
4ccf817 to
6a196c8
Compare
117dd1d to
954314f
Compare
36ff037 to
35c5f12
Compare
954314f to
fa666f7
Compare
35c5f12 to
f043de0
Compare
b4003ab to
33a702e
Compare
f043de0 to
05531f7
Compare
33a702e to
ad081d7
Compare
05531f7 to
79b103c
Compare
Integrate the node stream building blocks (from prior PR) into the actual render paths, all gated behind experimental.useNodeStreams: - app-render.tsx: add useNodeStreams branching throughout render paths, teeDebugChannelForSsrAndBrowser and debugChannelClientForBrowser helpers - app-render-prerender-utils.ts: ReactServerResult accepts NodeReadable, createReactServerPrerenderResultFromPrerender dispatcher - instant-validation.tsx: node stream branches for validation renders - render-result.ts: accept Node Readable as response body with piping - flight-render-result.ts: accept Node Readable in constructor - app-page.ts: PPR resume path for node streams - stream-ops.ts/debug-channel-server.ts: widen types to include Readable
ad081d7 to
6257529
Compare
79b103c to
558f424
Compare
Add `node-stream-helpers.ts` with Node.js native stream utilities that parallel the WhatWG stream helpers in `node-web-streams-helper.ts`. These are the foundational building blocks needed for the node-streams rendering effort (PRs vercel#89566, vercel#89859, vercel#89860, vercel#90500). Key functions: - `chainNodeStreams()` - chains multiple Readable streams sequentially - `createBufferedTransformNode()` - batches small chunks before flushing - `createInlinedDataNodeStream()` - inlines flight data into HTML stream - `pipeNodeReadableToResponse()` - pipes Readable directly to ServerResponse - `nodeStreamToBuffer()` / `nodeStreamToString()` - collection utilities ALS context propagation uses `bindSnapshot()` from the existing `async-local-storage.ts` module, which wraps `AsyncLocalStorage.bind()`. This addresses the review feedback from @lubieowoce on PR vercel#89859 where ALS context was incorrectly propagated by wrapping callback return values instead of binding the callbacks themselves. This PR adds only the helper utilities as new files. No existing files are modified. Wiring into the render pipeline is a separate step. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Summary
Integrate node stream primitives into the actual render paths, all gated behind
experimental.useNodeStreams.Core render pipeline (
app-render.tsx):teeDebugChannelForSsrAndBrowser(): tees debug channel for SSR + browser HMR deliverydebugChannelClientForBrowser(): converts Node Readable to web ReadableStream for HMRuseNodeStreamsbranching throughout: Fizz renders, flight streams, prerender paths, PPR resumeAnyStreamnow includesReadableat the switcher boundaryPrerender support (
app-render-prerender-utils.ts):ReactServerResultacceptsNodeReadablewith AsyncLocalStorage context preservationcreateReactServerPrerenderResultFromPrerender()dispatcher for node/web pathsprocessNodePrelude()for node stream prelude emptiness detectionOther integrations:
instant-validation.tsx: node stream branches for validation rendersrender-result.ts: accept Node Readable as response body,pipeToNodeWritable()methodflight-render-result.ts: accept Node Readable in constructorapp-page.ts: PPR resume path using PassThrough +pipeToNodeWritable()Test plan
pnpm --filter=next typespassespnpm test-start-turbo test/e2e/app-dir/ssr-in-rsc/passespnpm testonly test/e2e/app-dir/user-bundle-node-stream-guard/passes__NEXT_USE_NODE_STREAMS=true: app-dir e2e tests pass