Skip to content

fix(ai-openrouter): handle usage-only terminal stream chunks#6117

Merged
IMax153 merged 1 commit intoEffect-TS:mainfrom
nickbreaton:fix/openrouter-usage-only-stream-chunk
Mar 10, 2026
Merged

fix(ai-openrouter): handle usage-only terminal stream chunks#6117
IMax153 merged 1 commit intoEffect-TS:mainfrom
nickbreaton:fix/openrouter-usage-only-stream-chunk

Conversation

@nickbreaton
Copy link
Contributor

@nickbreaton nickbreaton commented Mar 8, 2026

Summary

Closes #6116.

It appears OpenRouter recently made a change where it can end a streamed response with a final chunk that contains usage data but no choices:

data: { "choices": [], "usage": {"prompt_tokens": 13, "completion_tokens": 16, "total_tokens": 29, "cost": 0.000093, "is_byok": false, "prompt_tokens_details": {"cached_tokens": 0, "cache_write_tokens": 0, "audio_tokens": 0, "video_tokens": 0}, "cost_details": {"upstream_inference_cost": 0.000093, "upstream_inference_prompt_cost": 0.000013, "upstream_inference_completions_cost": 0.00008}, "completion_tokens_details": {"reasoning_tokens": 0, "image_tokens": 0, "audio_tokens": 0}} }

Right now makeStreamResponse assumes every streamed chunk has at least one choice. That means this terminal chunk gets treated as malformed, and the stream fails at the very end even though the model output was already streamed successfully.

This PR makes the streaming adapter a little more permissive in that one case. Chunks with neither a choice nor usage are still treated as malformed, but a terminal usage-only chunk is now allowed through so the existing finish and usage finalization logic can run normally.

OpenRouter documents that the final chunk includes usage stats, but they do not clearly say whether an empty choices array is valid (evidently it now is):
https://openrouter.ai/docs/api/reference/streaming

This appears to be the same OpenRouter streaming issue that was recently reported and fixed in Zed's agent:

Tests?

I've validated the fix within my project impacted by the issue and it appears to work.

I did not see any unit tests within the repo that I could add to. Please let me know if I've missed that, or if adding some baseline tests for the package would be desired.

@github-project-automation github-project-automation bot moved this to Discussion Ongoing in PR Backlog Mar 8, 2026
@changeset-bot
Copy link

changeset-bot bot commented Mar 8, 2026

🦋 Changeset detected

Latest commit: 13fae61

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@effect/ai-openrouter Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@nickbreaton nickbreaton marked this pull request as ready for review March 8, 2026 03:26
@nickbreaton nickbreaton requested a review from IMax153 as a code owner March 8, 2026 03:26
@nickbreaton nickbreaton marked this pull request as draft March 8, 2026 03:27
@nickbreaton nickbreaton marked this pull request as ready for review March 8, 2026 03:31
Copy link
Member

@IMax153 IMax153 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nickbreaton !

@IMax153 IMax153 merged commit 7103e24 into Effect-TS:main Mar 10, 2026
11 checks passed
@github-project-automation github-project-automation bot moved this from Discussion Ongoing to Done in PR Backlog Mar 10, 2026
@github-actions github-actions bot mentioned this pull request Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

@effect/ai-openrouter streamText fails at end of stream

2 participants