Skip to content

Support sync and async iterables with Response/Request#5476

Merged
jasnell merged 1 commit intomainfrom
jasnell/fetch-body-async-iterator
Nov 15, 2025
Merged

Support sync and async iterables with Response/Request#5476
jasnell merged 1 commit intomainfrom
jasnell/fetch-body-async-iterator

Conversation

@jasnell
Copy link
Copy Markdown
Collaborator

@jasnell jasnell commented Nov 7, 2025

While passing sync/async iterables to Request and Response are not part of the standard, Node.js implements it and we've been asked to also.

const enc = new TextEncoder();
// Curently, the generator must yield ArrayBuffer or ArrayBufferView
async function* gen() {
  yield enc.encode('Hello ');
  yield enc.encode('World!');
}
return new Response(gen());

// or

const req = new Request('...', { body: gen() });

Note that this will NOT give us 100% compat with Node.js on this feature yet. Node.js supports writing strings to the response, e.g. yield 'foo'; while we still require the values to be ArrayBuffer or ArrayBufferView. We will be supporting strings as part of a different change that'll take a bit longer to land.

@jasnell jasnell requested a review from a team November 7, 2025 02:25
@jasnell jasnell requested review from a team as code owners November 7, 2025 02:25
@github-actions

This comment has been minimized.

@jasnell jasnell requested a review from a team as a code owner November 7, 2025 14:53
Copy link
Copy Markdown
Contributor

@petebacondarwin petebacondarwin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved for typings changes.

@jasnell jasnell force-pushed the jasnell/fetch-body-async-iterator branch from ded0063 to ae576ac Compare November 7, 2025 15:40
@jasnell jasnell marked this pull request as draft November 7, 2025 16:11
@jasnell
Copy link
Copy Markdown
Collaborator Author

jasnell commented Nov 7, 2025

Moving this back to draft. It's going to require a compatibility flag which is going to require some additional work.

@vicb
Copy link
Copy Markdown
Contributor

vicb commented Nov 7, 2025

Moving this back to draft. It's going to require a compatibility flag which is going to require some additional work.

Do we really need a compat flag here?
I can not really think about a use case in which users were passing an (async) iterable and expect .toString() to be called on it.
I would rather consider this a bug fix because the new behavior is what Node does

@vicb
Copy link
Copy Markdown
Contributor

vicb commented Nov 7, 2025

Does the new behavior only apply to nodejs_compat - I don't immediately see that in the code

@jasnell
Copy link
Copy Markdown
Collaborator Author

jasnell commented Nov 7, 2025

Yes, a compat flag is needed. The key issue is that stuff like new Response('hello') and new Response(['a','b','c']) are currently supported, it's also currently possible to pass something like new Response({ toString() { return 'abc' } })... specifically, strings and arrays are both iterables. This mechanism as written now handles strings as a special case but it becomes ambiguous when the user passes an object (like an Array) that implements both stringify and iterable semantics.

I can not really think about a use case in which users were passing an (async) iterable...

We have tests that do exactly this. Specifically, we have tests that are passing an Array to new Response(...). Arrays are both stringifiable (which is how they are handled now) and sync iterable. We cannot assume that users aren't doing similar things.

I would rather consider this a bug fix because the new behavior is what Node does

This really doesn't matter. Even as a bug fix it would require a compat flag.

Does the new behavior only apply to nodejs_compat - I don't immediately see that in the code

No, it's being enabled generally, not just for nodejs_compat.

@jasnell
Copy link
Copy Markdown
Collaborator Author

jasnell commented Nov 7, 2025

Testing Node.js further, it looks like they cheat around the issue by actually not supporting sync iterables.

> function* gen() { yield 'a'; yield 'b'; }
undefined
> r = new Response(gen());
Response {
  status: 200,
  statusText: '',
  headers: Headers { 'content-type': 'text/plain;charset=UTF-8' },
  body: ReadableStream { locked: false, state: 'readable', supportsBYOB: true },
  bodyUsed: false,
  ok: true,
  redirected: false,
  type: 'default',
  url: ''
}
> r.text().then(console.log)
Promise {
  <pending>,
  Symbol(async_id_symbol): 226,
  Symbol(trigger_async_id_symbol): 218
}
> [object Generator]
> async function* gen() { yield 'a'; yield 'b'; }
undefined
> r = new Response(gen());
Response {
  status: 200,
  statusText: '',
  headers: Headers {},
  body: ReadableStream { locked: false, state: 'readable', supportsBYOB: true },
  bodyUsed: false,
  ok: true,
  redirected: false,
  type: 'default',
  url: ''
}
> r.text().then(console.log)
Promise {
  <pending>,
  Symbol(async_id_symbol): 360,
  Symbol(trigger_async_id_symbol): 349
}
> ab

So actually, I think we may be able to safely enable that case but there's still an ambiguity that probably still requires a compat flag... If an obj the user is currently passing is has a Symbol.asyncIterator property that is ignored currently, it would suddenly start being treated differently. Because we really don't know if those exist, we have to use a compat flag.

@jasnell jasnell force-pushed the jasnell/fetch-body-async-iterator branch from ae576ac to a9ef841 Compare November 7, 2025 22:30
@codspeed-hq

This comment was marked as outdated.

@jasnell jasnell marked this pull request as ready for review November 7, 2025 23:14
@jasnell jasnell enabled auto-merge November 10, 2025 20:06
@jasnell jasnell disabled auto-merge November 15, 2025 00:11
@jasnell jasnell force-pushed the jasnell/fetch-body-async-iterator branch from a9ef841 to 0cb1779 Compare November 15, 2025 00:16
@jasnell jasnell enabled auto-merge November 15, 2025 00:16
@jasnell jasnell added feature Implementation of a new feature nodejs compat labels Nov 15, 2025
Copy link
Copy Markdown
Contributor

@guybedford guybedford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is there no WPT coverage diff on this PR? I would have expected we'd see an improvement, or does it need to be turned on for the WPT tests? Can we do that here?

@jasnell
Copy link
Copy Markdown
Collaborator Author

jasnell commented Nov 15, 2025

It's a node.js extension. As far as I know it's not supported by spec.

@guybedford
Copy link
Copy Markdown
Contributor

Given this is a Node.js compat feature, could we put it behind nodejs_compat?

@jasnell
Copy link
Copy Markdown
Collaborator Author

jasnell commented Nov 15, 2025

Given this is a Node.js compat feature, could we put it behind nodejs_compat

We've had people request it independently of nodejs_compat. I was planning to imply it on when nodejs_compat is on but also wanted the ability to switch it on/off independently and have it available without nodejs_compat.

@jasnell jasnell force-pushed the jasnell/fetch-body-async-iterator branch from 0cb1779 to ff1b1a5 Compare November 15, 2025 03:01
@guybedford
Copy link
Copy Markdown
Contributor

Right, and I know you mentioned this before but just to reiterate - the only risky compat case seems to be fetch('foo', { method: 'POST', body: [1,2] }). Is there any framing where we could special case that case outside of node_compat? Might be an option to keep in mind if this ever comes up.

@jasnell
Copy link
Copy Markdown
Collaborator Author

jasnell commented Nov 15, 2025

Well, in that case they can simply disable the compat flag to switch it off. Not sure what other special casing would be necessary?

@guybedford
Copy link
Copy Markdown
Contributor

Yeah agreed, it seems like it should be fine, but there's options if we hit any compat questions down the line. I mean specifically separating treatment of body on fetch from the init on request/response if such a compat case came up.

@jasnell jasnell merged commit 5a05537 into main Nov 15, 2025
20 checks passed
@jasnell jasnell deleted the jasnell/fetch-body-async-iterator branch November 15, 2025 03:33
@vicb
Copy link
Copy Markdown
Contributor

vicb commented Nov 15, 2025

Thanks @jasnell 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature Implementation of a new feature nodejs compat

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants