Skip to content

Node.js streams: First pass#90500

Draft
timneutkens wants to merge 35 commits intocanaryfrom
02-24-node.js_streams_first_pass
Draft

Node.js streams: First pass#90500
timneutkens wants to merge 35 commits intocanaryfrom
02-24-node.js_streams_first_pass

Conversation

@timneutkens
Copy link
Copy Markdown
Member

No description provided.

Copy link
Copy Markdown
Member Author

timneutkens commented Feb 25, 2026

@nextjs-bot
Copy link
Copy Markdown
Collaborator

nextjs-bot commented Feb 25, 2026

Failing test suites

Commit: c51e106 | About building and testing Next.js

pnpm test-start test/e2e/app-dir/metadata-invalid-image-file/metadata-invalid-image-file.test.ts (job)

  • experimental.nextScriptWorkers > experimental.nextScriptWorkers: true with required Partytown dependency for external script > Partytown snippets are injected to head if enabled in configuration (DD)
  • experimental.nextScriptWorkers > experimental.nextScriptWorkers: true with required Partytown dependency for external script > Worker scripts are modified by Partytown to execute on a worker thread (DD)
  • experimental.nextScriptWorkers > experimental.nextScriptWorkers: true with required Partytown dependency for inline script > Inline worker script through children is modified by Partytown to execute on a worker thread (DD)
  • experimental.nextScriptWorkers > experimental.nextScriptWorkers: true with required Partytown dependency for inline script > Inline worker script through dangerouslySetInnerHtml is modified by Partytown to execute on a worker thread (DD)
Expand output

● experimental.nextScriptWorkers › experimental.nextScriptWorkers: true with required Partytown dependency for external script › Partytown snippets are injected to head if enabled in configuration

expect(received).not.toEqual(expected) // deep equality

Expected: not null

  319 |           )
  320 |
> 321 |           expect(snippetScript).not.toEqual(null)
      |                                     ^
  322 |
  323 |           // A default config is included that points to the correct folder that hosts partytown's static files
  324 |           expect(configScript).not.toEqual(null)

  at Object.toEqual (e2e/next-script/index.test.ts:321:37)

● experimental.nextScriptWorkers › experimental.nextScriptWorkers: true with required Partytown dependency for external script › Worker scripts are modified by Partytown to execute on a worker thread

TIMED OUT: success

0

undefined

  788 |   }
  789 |   console.error('TIMED OUT CHECK: ', { regex, content, lastErr })
> 790 |   throw new Error('TIMED OUT: ' + regex + '\n\n' + content + '\n\n' + lastErr)
      |         ^
  791 | }
  792 |
  793 | export class File {

  at check (lib/next-test-utils.ts:790:9)
  at Object.<anonymous> (e2e/next-script/index.test.ts:340:11)

● experimental.nextScriptWorkers › experimental.nextScriptWorkers: true with required Partytown dependency for inline script › Inline worker script through children is modified by Partytown to execute on a worker thread

TIMED OUT: 1

0

undefined

  788 |   }
  789 |   console.error('TIMED OUT CHECK: ', { regex, content, lastErr })
> 790 |   throw new Error('TIMED OUT: ' + regex + '\n\n' + content + '\n\n' + lastErr)
      |         ^
  791 | }
  792 |
  793 | export class File {

  at check (lib/next-test-utils.ts:790:9)
  at Object.<anonymous> (e2e/next-script/index.test.ts:404:11)

● experimental.nextScriptWorkers › experimental.nextScriptWorkers: true with required Partytown dependency for inline script › Inline worker script through dangerouslySetInnerHtml is modified by Partytown to execute on a worker thread

TIMED OUT: 1

0

undefined

  788 |   }
  789 |   console.error('TIMED OUT CHECK: ', { regex, content, lastErr })
> 790 |   throw new Error('TIMED OUT: ' + regex + '\n\n' + content + '\n\n' + lastErr)
      |         ^
  791 | }
  792 |
  793 | export class File {

  at check (lib/next-test-utils.ts:790:9)
  at Object.<anonymous> (e2e/next-script/index.test.ts:429:11)

pnpm test-start-turbo test/e2e/app-dir/app-prefetch/prefetching.test.ts (turbopack) (job)

  • Graceful Shutdown > production (next start) > should not accept new requests during shutdown cleanup > should stop accepting new requests when shutting down (DD)
  • Graceful Shutdown > production (standalone mode) > should not accept new requests during shutdown cleanup > should stop accepting new requests when shutting down (DD)
Expand output

● Graceful Shutdown › production (next start) › should not accept new requests during shutdown cleanup › should stop accepting new requests when shutting down

expect(received).toEqual(expected) // deep equality

- Expected  - 1
+ Received  + 1

  Array [
-   143,
    null,
+   "SIGTERM",
  ]

  227 |
  228 |         // App finally shuts down with signal-based exit code (128 + 15 for SIGTERM)
> 229 |         expect(await appKilledPromise).toEqual([143, null])
      |                                        ^
  230 |         expect(app.exitCode).toBe(143)
  231 |       })
  232 |     })

  at Object.toEqual (production/graceful-shutdown/index.test.ts:229:40)

● Graceful Shutdown › production (standalone mode) › should not accept new requests during shutdown cleanup › should stop accepting new requests when shutting down

expect(received).toEqual(expected) // deep equality

- Expected  - 1
+ Received  + 1

  Array [
-   143,
    null,
+   "SIGTERM",
  ]

  227 |
  228 |         // App finally shuts down with signal-based exit code (128 + 15 for SIGTERM)
> 229 |         expect(await appKilledPromise).toEqual([143, null])
      |                                        ^
  230 |         expect(app.exitCode).toBe(143)
  231 |       })
  232 |     })

  at Object.toEqual (production/graceful-shutdown/index.test.ts:229:40)

pnpm test-start test/e2e/app-dir/next-after-app-static/build-time/build-time.test.ts (job)

  • after() in static pages > runs after during build (DD)
Expand output

● after() in static pages › runs after during build

can not run export while server is running, use next.stop() first

  251 |   ) {
  252 |     if (this.childProcess) {
> 253 |       throw new Error(
      |             ^
  254 |         `can not run export while server is running, use next.stop() first`
  255 |       )
  256 |     }

  at NextStartInstance.build (lib/next-modes/next-start.ts:253:13)
  at Object.build (e2e/app-dir/next-after-app-static/build-time/build-time.test.ts:36:36)

pnpm test-dev test/e2e/app-dir/app-basepath/index.test.ts (job)

  • app dir - basepath > should successfully hard navigate from pages -> app (DD)
Expand output

● app dir - basepath › should successfully hard navigate from pages -> app

page.waitForSelector: Timeout 10000ms exceeded.
Call log:
  - waiting for locator('#page-2') to be visible

  545 |
  546 |     return this.startChain(async () => {
> 547 |       const el = await page.waitForSelector(selector, {
      |                             ^
  548 |         timeout,
  549 |         state,
  550 |       })

  at waitForSelector (lib/browsers/playwright.ts:547:29)
  at Playwright._chain (lib/browsers/playwright.ts:677:23)
  at Playwright._chain [as startChain] (lib/browsers/playwright.ts:658:17)
  at Playwright.startChain [as waitForElementByCss] (lib/browsers/playwright.ts:546:17)
  at Object.waitForElementByCss (e2e/app-dir/app-basepath/index.test.ts:16:19)

@nextjs-bot
Copy link
Copy Markdown
Collaborator

nextjs-bot commented Feb 25, 2026

Stats from current PR

🔴 1 regression

Metric Canary PR Change Trend
node_modules Size 477 MB 478 MB 🔴 +1.6 MB (+0%) ██▁▁▁
📊 All Metrics
📖 Metrics Glossary

Dev Server Metrics:

  • Listen = TCP port starts accepting connections
  • First Request = HTTP server returns successful response
  • Cold = Fresh build (no cache)
  • Warm = With cached build artifacts

Build Metrics:

  • Fresh = Clean build (no .next directory)
  • Cached = With existing .next directory

Change Thresholds:

  • Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
  • Size: Changes < 1KB AND < 1% are insignificant
  • All other changes are flagged to catch regressions

⚡ Dev Server

Metric Canary PR Change Trend
Cold (Listen) 455ms 455ms ▅█▁▁▁
Cold (Ready in log) 437ms 439ms ▄█▁▁▁
Cold (First Request) 1.221s 1.217s ▃█▁▁▁
Warm (Listen) 457ms 457ms ▅█▁▁▁
Warm (Ready in log) 442ms 441ms ▃█▁▁▁
Warm (First Request) 353ms 355ms ▃█▁▁▁
📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric Canary PR Change Trend
Cold (Listen) 456ms 456ms █▁▁▁▁
Cold (Ready in log) 447ms 450ms █▁▄▆▅
Cold (First Request) 1.904s 1.923s █▁▃▄▃
Warm (Listen) 456ms 456ms █▁▁▁▁
Warm (Ready in log) 447ms 448ms █▁▅▄▃
Warm (First Request) 1.914s 1.917s █▁▃▃▃

⚡ Production Builds

Metric Canary PR Change Trend
Fresh Build 3.824s 3.823s ▃█▁▁▁
Cached Build 3.910s 3.886s ▃█▁▁▁
📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric Canary PR Change Trend
Fresh Build 14.493s 14.517s █▁▁▁▁
Cached Build 14.484s 14.541s █▁▁▁▁
node_modules Size 477 MB 478 MB 🔴 +1.6 MB (+0%) ██▁▁▁
📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles: **402 kB** → **402 kB** ⚠️ +23 B

80 files with content-based hashes (individual files not comparable between builds)

Server

Middleware
Canary PR Change
middleware-b..fest.js gzip 766 B 765 B
Total 766 B 765 B ✅ -1 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 446 B 450 B
Total 446 B 450 B ⚠️ +4 B

📦 Webpack

Client

Main Bundles
Canary PR Change
5528-HASH.js gzip 5.54 kB N/A -
6280-HASH.js gzip 59.4 kB N/A -
6335.HASH.js gzip 169 B N/A -
912-HASH.js gzip 4.59 kB N/A -
e8aec2e4-HASH.js gzip 62.6 kB N/A -
framework-HASH.js gzip 59.7 kB 59.7 kB
main-app-HASH.js gzip 255 B 254 B
main-HASH.js gzip 39.1 kB 39.1 kB
webpack-HASH.js gzip 1.68 kB 1.68 kB
262-HASH.js gzip N/A 4.59 kB -
2889.HASH.js gzip N/A 169 B -
5602-HASH.js gzip N/A 5.55 kB -
6948ada0-HASH.js gzip N/A 62.6 kB -
9544-HASH.js gzip N/A 60.2 kB -
Total 233 kB 234 kB ⚠️ +726 B
Polyfills
Canary PR Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB
Total 39.4 kB 39.4 kB
Pages
Canary PR Change
_app-HASH.js gzip 194 B 194 B
_error-HASH.js gzip 183 B 180 B 🟢 3 B (-2%)
css-HASH.js gzip 331 B 330 B
dynamic-HASH.js gzip 1.81 kB 1.81 kB
edge-ssr-HASH.js gzip 256 B 256 B
head-HASH.js gzip 351 B 352 B
hooks-HASH.js gzip 384 B 383 B
image-HASH.js gzip 580 B 581 B
index-HASH.js gzip 260 B 260 B
link-HASH.js gzip 2.51 kB 2.51 kB
routerDirect..HASH.js gzip 320 B 319 B
script-HASH.js gzip 386 B 386 B
withRouter-HASH.js gzip 315 B 315 B
1afbb74e6ecf..834.css gzip 106 B 106 B
Total 7.98 kB 7.98 kB ✅ -1 B

Server

Edge SSR
Canary PR Change
edge-ssr.js gzip 125 kB 125 kB
page.js gzip 256 kB 258 kB 🔴 +2.49 kB (+1%)
Total 380 kB 383 kB ⚠️ +2.9 kB
Middleware
Canary PR Change
middleware-b..fest.js gzip 617 B 615 B
middleware-r..fest.js gzip 156 B 155 B
middleware.js gzip 43.9 kB 44.1 kB
edge-runtime..pack.js gzip 842 B 842 B
Total 45.5 kB 45.8 kB ⚠️ +208 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 715 B 718 B
Total 715 B 718 B ⚠️ +3 B
Build Cache
Canary PR Change
0.pack gzip 4.07 MB 4.08 MB 🔴 +9.7 kB (+0%)
index.pack gzip 104 kB 103 kB 🟢 1.08 kB (-1%)
index.pack.old gzip 103 kB 102 kB 🟢 1.05 kB (-1%)
Total 4.27 MB 4.28 MB ⚠️ +7.56 kB

🔄 Shared (bundler-independent)

Runtimes
Canary PR Change
app-page-exp...dev.js gzip 322 kB 327 kB 🔴 +5.23 kB (+2%)
app-page-exp..prod.js gzip 171 kB 175 kB 🔴 +3.37 kB (+2%)
app-page-tur...dev.js gzip 322 kB 327 kB 🔴 +5.22 kB (+2%)
app-page-tur..prod.js gzip 171 kB 174 kB 🔴 +3.36 kB (+2%)
app-page-tur...dev.js gzip 318 kB 323 kB 🔴 +5.15 kB (+2%)
app-page-tur..prod.js gzip 169 kB 173 kB 🔴 +3.4 kB (+2%)
app-page.run...dev.js gzip 319 kB 324 kB 🔴 +5.15 kB (+2%)
app-page.run..prod.js gzip 169 kB 173 kB 🔴 +3.4 kB (+2%)
app-route-ex...dev.js gzip 70.9 kB 71.1 kB
app-route-ex..prod.js gzip 49.3 kB 49.5 kB
app-route-tu...dev.js gzip 70.9 kB 71.1 kB
app-route-tu..prod.js gzip 49.3 kB 49.5 kB
app-route-tu...dev.js gzip 70.5 kB 70.7 kB
app-route-tu..prod.js gzip 49 kB 49.3 kB
app-route.ru...dev.js gzip 70.4 kB 70.7 kB
app-route.ru..prod.js gzip 49 kB 49.3 kB
dist_client_...dev.js gzip 324 B 324 B
dist_client_...dev.js gzip 326 B 326 B
dist_client_...dev.js gzip 318 B 318 B
dist_client_...dev.js gzip 317 B 317 B
pages-api-tu...dev.js gzip 43.2 kB 43.5 kB
pages-api-tu..prod.js gzip 32.9 kB 33.2 kB
pages-api.ru...dev.js gzip 43.2 kB 43.5 kB
pages-api.ru..prod.js gzip 32.9 kB 33.1 kB
pages-turbo....dev.js gzip 52.6 kB 52.9 kB
pages-turbo...prod.js gzip 38.5 kB 38.8 kB
pages.runtim...dev.js gzip 52.6 kB 52.8 kB
pages.runtim..prod.js gzip 38.5 kB 38.7 kB
server.runti..prod.js gzip 62 kB 62.2 kB
Total 2.84 MB 2.88 MB ⚠️ +38.5 kB
📝 Changed Files (25 files)

Files with changes:

  • app-page-exp..ntime.dev.js
  • app-page-exp..time.prod.js
  • app-page-tur..ntime.dev.js
  • app-page-tur..time.prod.js
  • app-page-tur..ntime.dev.js
  • app-page-tur..time.prod.js
  • app-page.runtime.dev.js
  • app-page.runtime.prod.js
  • app-route-ex..ntime.dev.js
  • app-route-ex..time.prod.js
  • app-route-tu..ntime.dev.js
  • app-route-tu..time.prod.js
  • app-route-tu..ntime.dev.js
  • app-route-tu..time.prod.js
  • app-route.runtime.dev.js
  • app-route.ru..time.prod.js
  • pages-api-tu..ntime.dev.js
  • pages-api-tu..time.prod.js
  • pages-api.runtime.dev.js
  • pages-api.ru..time.prod.js
  • ... and 5 more
View diffs
app-page-exp..ntime.dev.js
failed to diff
app-page-exp..time.prod.js
failed to diff
app-page-tur..ntime.dev.js
failed to diff
app-page-tur..time.prod.js
failed to diff
app-page-tur..ntime.dev.js
failed to diff
app-page-tur..time.prod.js
failed to diff
app-page.runtime.dev.js
failed to diff
app-page.runtime.prod.js
failed to diff
app-route-ex..ntime.dev.js

Diff too large to display

app-route-ex..time.prod.js

Diff too large to display

app-route-tu..ntime.dev.js

Diff too large to display

app-route-tu..time.prod.js

Diff too large to display

app-route-tu..ntime.dev.js

Diff too large to display

app-route-tu..time.prod.js

Diff too large to display

app-route.runtime.dev.js

Diff too large to display

app-route.ru..time.prod.js

Diff too large to display

pages-api-tu..ntime.dev.js

Diff too large to display

pages-api-tu..time.prod.js

Diff too large to display

pages-api.runtime.dev.js

Diff too large to display

pages-api.ru..time.prod.js

Diff too large to display

pages-turbo...ntime.dev.js

Diff too large to display

pages-turbo...time.prod.js

Diff too large to display

pages.runtime.dev.js

Diff too large to display

pages.runtime.prod.js

Diff too large to display

server.runtime.prod.js

Diff too large to display

📎 Tarball URL
https://vercel-packages.vercel.app/next/commits/2d9ecef39425cb2cc4e08fb8f31c37aeb8e72fa8/next

@timneutkens timneutkens force-pushed the 02-24-node.js_streams_first_pass branch from 83a60e0 to e730890 Compare March 3, 2026 11:29
@timneutkens timneutkens force-pushed the feedthejim/node-stream-05-ci branch from 09f55e4 to 2653f7a Compare March 3, 2026 11:29
@timneutkens timneutkens force-pushed the 02-24-node.js_streams_first_pass branch 2 times, most recently from 5d08c0b to f5d7e30 Compare March 4, 2026 09:49
@timneutkens timneutkens force-pushed the feedthejim/node-stream-05-ci branch 2 times, most recently from 4101209 to c8e8ea6 Compare March 4, 2026 10:09
@timneutkens timneutkens force-pushed the 02-24-node.js_streams_first_pass branch 2 times, most recently from 1ca3490 to ab198ee Compare March 4, 2026 15:02
@vercel
Copy link
Copy Markdown
Contributor

vercel bot commented Mar 4, 2026

Deployment failed with the following error:

Creating the Deployment Timed Out.

@timneutkens timneutkens force-pushed the feedthejim/node-stream-05-ci branch from b0ba55d to 89ef013 Compare March 5, 2026 09:05
@timneutkens timneutkens force-pushed the 02-24-node.js_streams_first_pass branch 2 times, most recently from dc83079 to b06d35e Compare March 5, 2026 14:25
@timneutkens timneutkens force-pushed the feedthejim/node-stream-05-ci branch 2 times, most recently from 114e0d2 to 449ef4c Compare March 6, 2026 09:43
@timneutkens timneutkens force-pushed the 02-24-node.js_streams_first_pass branch from 4d2e1f2 to 85a490b Compare March 6, 2026 09:43
Copy link
Copy Markdown
Contributor

@gnoff gnoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add a better description before we merge but looks good as a jumping off point

benfavre added a commit to benfavre/next.js that referenced this pull request Mar 18, 2026
Add `node-stream-helpers.ts` with Node.js native stream utilities that
parallel the WhatWG stream helpers in `node-web-streams-helper.ts`.
These are the foundational building blocks needed for the node-streams
rendering effort (PRs vercel#89566, vercel#89859, vercel#89860, vercel#90500).

Key functions:
- `chainNodeStreams()` - chains multiple Readable streams sequentially
- `createBufferedTransformNode()` - batches small chunks before flushing
- `createInlinedDataNodeStream()` - inlines flight data into HTML stream
- `pipeNodeReadableToResponse()` - pipes Readable directly to ServerResponse
- `nodeStreamToBuffer()` / `nodeStreamToString()` - collection utilities

ALS context propagation uses `bindSnapshot()` from the existing
`async-local-storage.ts` module, which wraps `AsyncLocalStorage.bind()`.
This addresses the review feedback from @lubieowoce on PR vercel#89859 where
ALS context was incorrectly propagated by wrapping callback return values
instead of binding the callbacks themselves.

This PR adds only the helper utilities as new files. No existing files
are modified. Wiring into the render pipeline is a separate step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
benfavre added a commit to benfavre/next.js that referenced this pull request Mar 18, 2026
…eams flag

Add `experimental.useNodeStreams` config flag that switches the stream
operations module (stream-ops.ts) to load native Node.js implementations
for hot-path functions:

- chainStreams: uses chainNodeStreams (PassThrough-based sequential piping)
- streamToBuffer: uses nodeStreamToBuffer (for-await on Node Readable)
- streamToString: uses nodeStreamToString (streaming TextDecoder)
- renderToFizzStream: uses renderToPipeableStream instead of
  renderToReadableStream, avoiding web→node conversion overhead in React

Complex transform chains (continueFizzStream, prerender continuations)
still delegate to the web implementation as a stopgap — the native
buffering and data inlining transforms from node-stream-helpers will be
wired in a follow-up once the web transform chain is decomposed.

Includes the node-stream-helpers module from PR vercel#91580 which provides
the underlying native stream utilities.

References: vercel#91580, vercel#89566, vercel#90500

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@timneutkens timneutkens force-pushed the 02-24-node.js_streams_first_pass branch from 2d9ecef to c67bd1d Compare March 19, 2026 12:53
@timneutkens timneutkens force-pushed the 02-24-node.js_streams_first_pass branch from fa84d62 to c51e106 Compare March 19, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants