feat: add Node.js native stream helpers for render pipeline#91580
Open
benfavre wants to merge 4 commits intovercel:canaryfrom
Open
feat: add Node.js native stream helpers for render pipeline#91580benfavre wants to merge 4 commits intovercel:canaryfrom
benfavre wants to merge 4 commits intovercel:canaryfrom
Conversation
Add `node-stream-helpers.ts` with Node.js native stream utilities that parallel the WhatWG stream helpers in `node-web-streams-helper.ts`. These are the foundational building blocks needed for the node-streams rendering effort (PRs vercel#89566, vercel#89859, vercel#89860, vercel#90500). Key functions: - `chainNodeStreams()` - chains multiple Readable streams sequentially - `createBufferedTransformNode()` - batches small chunks before flushing - `createInlinedDataNodeStream()` - inlines flight data into HTML stream - `pipeNodeReadableToResponse()` - pipes Readable directly to ServerResponse - `nodeStreamToBuffer()` / `nodeStreamToString()` - collection utilities ALS context propagation uses `bindSnapshot()` from the existing `async-local-storage.ts` module, which wraps `AsyncLocalStorage.bind()`. This addresses the review feedback from @lubieowoce on PR vercel#89859 where ALS context was incorrectly propagated by wrapping callback return values instead of binding the callbacks themselves. This PR adds only the helper utilities as new files. No existing files are modified. Wiring into the render pipeline is a separate step. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Collaborator
|
Allow CI Workflow Run
Note: this should only be enabled once the PR is ready to go and can only be enabled by a maintainer |
- Use lazy require('node:stream') for Edge/DCE compatibility
- Use scheduleImmediate instead of raw setImmediate
- Use streaming TextDecoder in nodeStreamToString
- Add safePipe helper for proper error propagation
- Fix pipeNodeReadableToResponse onClose/onEnd semantics
- Add TODO for Buffer-based tag indexOf optimization
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- safePipe: use .once('error') instead of .on('error') — streams emit
at most one error, and once auto-removes the listener
- pipeNodeReadableToResponse: remove unused DetachedPromise (was
allocated per request but never awaited or returned)
- createInlinedDataNodeStream: skip startPulling call when already
started (avoids redundant function call per chunk)
benfavre
added a commit
to benfavre/next.js
that referenced
this pull request
Mar 18, 2026
…eams flag Add `experimental.useNodeStreams` config flag that switches the stream operations module (stream-ops.ts) to load native Node.js implementations for hot-path functions: - chainStreams: uses chainNodeStreams (PassThrough-based sequential piping) - streamToBuffer: uses nodeStreamToBuffer (for-await on Node Readable) - streamToString: uses nodeStreamToString (streaming TextDecoder) - renderToFizzStream: uses renderToPipeableStream instead of renderToReadableStream, avoiding web→node conversion overhead in React Complex transform chains (continueFizzStream, prerender continuations) still delegate to the web implementation as a stopgap — the native buffering and data inlining transforms from node-stream-helpers will be wired in a follow-up once the web transform chain is decomposed. Includes the node-stream-helpers module from PR vercel#91580 which provides the underlying native stream utilities. References: vercel#91580, vercel#89566, vercel#90500 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
6 tasks
When delayDataUntilFirstHtmlChunk=true and the HTML stream emits zero chunks, the transform callback is never called so startPulling() is never invoked. In flush(), dataExhausted is false, causing it to wait on dataComplete.promise which never resolves — a hang. Call startPulling() in flush() if it hasn't been called yet, so the data stream listeners are attached and can resolve the promise. Adds a regression test for the empty-HTML-stream edge case. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This was referenced Mar 18, 2026
Contributor
Author
Test Verification
All tests run on the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Node.js native stream utilities (node-stream-helpers.ts) as the foundation for replacing WhatWG stream polyfills in the SSR render pipeline. Profiling shows the web-to-node conversion layer accounts for 35%+ of CPU time in SSR workloads.
Helpers provided
Key design decisions
What this PR does NOT do
This PR only adds the helpers + tests. It does not wire them into the render pipeline yet -- that will be a follow-up PR that replaces the WhatWG stream path with these native equivalents behind a flag.
Test plan