Skip to content

Route standalone image generation through host finalization md#25176

Merged
won-openai merged 3 commits into
mainfrom
extension_turn_itme
Jun 2, 2026
Merged

Route standalone image generation through host finalization md#25176
won-openai merged 3 commits into
mainfrom
extension_turn_itme

Conversation

@won-openai

Copy link
Copy Markdown
Collaborator

Why

Standalone image-generation extensions emitted turn items through the low-level event path, bypassing host-owned finalization such as image persistence and contributor processing. At the same time, the generated-image save-path hint must remain visible to the model through the extension tool's FunctionCallOutput, rather than the legacy built-in developer-message path.

What changed

  • Extended ExtensionTurnItem to support image-generation items while keeping the extension-facing emitter API limited to emit_started and emit_completed.
  • Routed extension completion through core finalize_turn_item, so standalone image-generation items receive host-owned processing and persisted saved_path values before publication.
  • Kept legacy built-in image generation on its existing developer-message hint path, while standalone image generation returns its deterministic saved-path hint in FunctionCallOutput.
  • Shared the image artifact path and output-hint formatting used by core and the image-generation extension.
  • Passed thread identity through extension tool calls so standalone image generation can construct the same intended artifact path as core.
  • Added an app-server integration test covering real standalone image generation, saved artifact publication, model-visible output hint wiring, and absence of the legacy developer-message hint.

Validation

  • just fmt
  • just test -p codex-image-generation-extension
  • just test -p codex-web-search-extension
  • just test -p codex-goal-extension
  • just test -p codex-memories-extension
  • Targeted codex-core tests for image save history, extension completion finalization, and contributor execution
  • just test -p codex-app-server standalone_image_generation_returns_saved_path_hint_to_model
  • just fix -p codex-core
  • just fix -p codex-image-generation-extension
  • just bazel-lock-update
  • just bazel-lock-check

@won-openai won-openai requested a review from a team as a code owner May 29, 2026 23:50
let conversation_history =
ConversationHistory::new(invocation.session.clone_history().await.into_raw_items());
ExtensionToolCall {
thread_id: invocation.session.conversation_id.to_string(),

@won-openai won-openai May 30, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to know where it's saved

@won-openai

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bf947390ba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +96 to +100
call.turn_item_emitter
.emit_started(ExtensionTurnItem::ImageGeneration(ImageGenerationItem {
id: call.call_id.clone(),
status: "in_progress".to_string(),
revised_prompt: None,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Emit a terminal image item on generation failure

When the image backend fails or returns no image data after this emit_started, handle returns FunctionCallError before any matching completed/failed image-generation item is emitted. The generic tool error path only returns a failed FunctionCallOutput, so app-server/TUI clients that already received this started item can leave the image generation permanently in progress for quota, network, or upstream failures; before this change the image item was only started after a successful result was available.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the error is sent to the model, no need for separate ui on this imo

Comment thread codex-rs/ext/image-generation/src/tool.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2e9606e804

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codex-rs/ext/memories/src/tests.rs Outdated
result,
output_hint,
}))
Ok(Box::new(GeneratedImageOutput { result }))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Return the saved image path to the model

After successful standalone image generation, this now returns a GeneratedImageOutput that only serializes the image bytes, while the persisted saved_path is only emitted to clients via the turn item and is not added to the model-facing function output or conversation history. In the common follow-up where the model needs to inspect, copy, or edit the generated file from the shell, it no longer knows the deterministic artifact path that was previously included by image_generation_completed when persistence succeeded.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is highly relevant to the PR but i'm putting this separately for sake of reviewer

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree

@jif-oai jif-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See Slack comment + I agree with the remaining open comment but the rest lgtm

@won-openai won-openai merged commit 57f337a into main Jun 2, 2026
31 checks passed
@won-openai won-openai deleted the extension_turn_itme branch June 2, 2026 19:00
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 2, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants