feat: image content by cristinaponcela · Pull Request #3887 · earendil-works/pi

cristinaponcela · 2026-04-28T11:52:30Z

This PR adds a new API, closely mirroring the stream API, to support image blocks and image models (via Google/ OpenRouter) so the agent can output images.

With a simple (clanked) extension, you can test in the TUI:

badlogic · 2026-05-04T20:44:52Z

clsoing this out, as we have a new plan.

badlogic · 2026-05-04T20:45:14Z

wait, shit, i thought that was the old PR :D

badlogic

overall squeaky clean! great job. left some minor comments.

badlogic · 2026-05-04T20:46:10Z

@@ -0,0 +1,202 @@
+import OpenAI from "openai";


hmmm, i wonder if we should mvoe imagegen provider impls to their own subfolder, to separate them from the normal providers. how do you feel about that @cristinaponcela ?

Yep, cleaner. ✅ 60e885f

badlogic · 2026-05-04T20:50:39Z

+	return provider.images(model, context, options as ImagesOptions);
+}
+
+export async function completeImages<TApi extends ImagesApi>(


rename to generateImages

✅ 0d96b9b

badlogic · 2026-05-04T20:53:47Z

@@ -0,0 +1,111 @@
+import { beforeEach, describe, expect, it, vi } from "vitest";


we'll need an end-to-end test as well. see stream.test.ts as a sort of template.

list of functions all provider specific tests can call into

one describe per provider with "sub tests" calling into the functions with a known-good model from that provider

Makes sense. Done, only 1 describe bc 1 provider (OpenRouter), using Gemini 2.5 flash image preview.

✅ 3728e4b

badlogic · 2026-05-04T20:54:22Z

@@ -1249,10 +1249,11 @@ Create a new provider file (for example `amazon-bedrock.ts`) that exports:
 - Add credential detection in `env-api-keys.ts` for the new provider


ai/README.md needs a dedicated section on image generation explaining all the ins and outs and don'ts, following the style of the rest of the README.md.

✅ 074747c

Do lmk if this needs more detail/ is too verbose, first time writing public docs :3

badlogic · 2026-05-07T10:03:43Z

@cristinaponcela I think we should remove images() entirely here.

The OpenRouter provider sends stream: false, so images() is not actually streaming anything. It waits for the full HTTP response and then emits synthetic events after the fact. That makes the API misleading, especially because generateImages() can return both image and text blocks, but the current events only expose image blocks.

Let's keep this as a one-shot API only:

generateImages(model, context, options) returns the final result
result output can contain image blocks and text blocks
no AssistantImagesEventStream
no images() helper
no image event protocol

Abort support is still needed. The current provider already passes options.signal into the OpenAI request options, so please keep that behavior when moving generateImages() to call the provider directly.

Please also update packages/ai/README.md accordingly.

badlogic · 2026-05-08T13:57:09Z

looking good, cheers!

补充缺失条目： - packages/ai: 图片输出功能 (PR earendil-works#3887), Bun WebSocket 代理修复, Fireworks 会话亲和性修复 - packages/tui: 列表项缩进包裹, 复选框渲染, 大文件 markdown 鲁棒性, 图片放置修复 - packages/coding-agent: TTY 异常恢复, .agents 来源保留, 跨包同步图片输出功能 ziye Co-Authored-By: ziye <ziye0180@outlook.com>

…e-outputs feat: image content

cristinaponcela added 2 commits April 28, 2026 13:43

feat: image content

c3c1073

fix: test (openRouterImageGeneration)

364ac0f

github-actions Bot mentioned this pull request Apr 29, 2026

📊 AI CLI 工具社区动态日报 2026-04-29 borq168/radar-forge#67

Open

cristinaponcela added 11 commits April 29, 2026 09:49

fix

e9414b0

fix: modalities from openrouter

59a89e0

revert

cbf3c33

feat: images api types

e3d066d

feat: images registry

62d9132

feat: images stream

e9b0af0

feat: openrouter images

d9adc53

feat: image models

63c61aa

cleanup

b9c1913

dog

ab14238

Merge branch 'main' into feat/image-outputs

5731e13

badlogic closed this May 4, 2026

badlogic reopened this May 4, 2026

badlogic reviewed May 4, 2026

View reviewed changes

cristinaponcela added 4 commits May 5, 2026 09:30

refactor: image providers into subfolder

60e885f

rename: generateImages

0d96b9b

e2e: add images test

3728e4b

docs: readme image section

074747c

cristinaponcela requested a review from badlogic May 5, 2026 08:08

badlogic added the inprogress Issue is being worked on label May 7, 2026

cristinaponcela added 4 commits May 7, 2026 16:21

delete: images()

32b2cd0

Merge remote-tracking branch 'upstream/main' into feat/image-outputs

ffdf426

fix: lazy loading

0e4f845

fix: no fallback for apiKey

8fa1194

badlogic merged commit 9751057 into earendil-works:main May 8, 2026
1 check passed

larsboes pushed a commit to larsboes/pi-mono that referenced this pull request May 17, 2026

Merge pull request earendil-works#3887 from cristinaponcela/feat/imag…

4b1db00

…e-outputs feat: image content

		@@ -0,0 +1,111 @@
		import { beforeEach, describe, expect, it, vi } from "vitest";

		@@ -1249,10 +1249,11 @@ Create a new provider file (for example `amazon-bedrock.ts`) that exports:
		- Add credential detection in `env-api-keys.ts` for the new provider

Conversation

cristinaponcela commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

badlogic commented May 4, 2026

Uh oh!

badlogic commented May 4, 2026

Uh oh!

badlogic left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

badlogic commented May 7, 2026

Uh oh!

Uh oh!

badlogic commented May 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cristinaponcela commented Apr 28, 2026 •

edited

Loading