fix(agents): suppress commentary partial visibility and buffer unphased early deltas#61457
fix(agents): suppress commentary partial visibility and buffer unphased early deltas#61457100yenadmin wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refines the OpenAI Responses WebSocket streaming + replay path to prevent commentary leakage and to correctly phase-map streamed text when response.output_text.delta arrives before response.output_item.added.
Changes:
- Buffer early
response.output_text.*events until the corresponding output item (and its phase) is known, then emit only appropriately phased partials. - Make replay/input-item conversion phase-aware by splitting assistant text on per-block
textSignature.phaseboundaries instead of flattening. - Adjust stored assistant message reconstruction to omit top-level
phasewhen output contains mixed phases or mixed phased/unphased text, and add tests for these behaviors.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/agents/openai-ws-stream.ts | Adds buffering + phase-aware emission of WS text deltas, and encodes per-item text signatures on partials. |
| src/agents/openai-ws-stream.test.ts | Adds unit coverage for phase-aware replay splitting and WS late phase-mapping/buffering behaviors. |
| src/agents/openai-ws-message-conversion.ts | Updates replay conversion to split assistant text by block-level phase signatures; omits top-level phase for mixed-phase responses. |
| const itemPhase = normalizeAssistantPhase(outputItemPhaseById.get(params.itemId)); | ||
| if (!itemPhase) { | ||
| return; | ||
| } | ||
| const partialBase = buildAssistantMessageWithZeroUsage({ | ||
| model, | ||
| content: [ | ||
| { | ||
| type: "text", | ||
| text: params.fullText, | ||
| textSignature: encodeAssistantTextSignature({ | ||
| id: params.itemId, | ||
| phase: itemPhase, | ||
| }), | ||
| }, | ||
| ], | ||
| stopReason: "stop", | ||
| }); | ||
| const partialMsg: AssistantMessageWithPhase = { | ||
| ...partialBase, | ||
| phase: itemPhase, | ||
| }; | ||
| eventStream.push({ | ||
| type: "text_delta", | ||
| contentIndex: params.contentIndex, | ||
| delta: params.deltaText, | ||
| partial: partialMsg, | ||
| }); | ||
| }; | ||
| const flushPendingUnknownPhaseText = (itemId: string) => { | ||
| const itemPhase = normalizeAssistantPhase(outputItemPhaseById.get(itemId)); | ||
| if (!itemPhase) { | ||
| pendingUnknownPhaseTextByPart.forEach((_, key) => { | ||
| if (key.startsWith(`${itemId}:`)) { | ||
| pendingUnknownPhaseTextByPart.delete(key); | ||
| } | ||
| }); | ||
| return; | ||
| } |
There was a problem hiding this comment.
emitTextDelta() currently drops all response.output_text.* events unless the output item phase is exactly commentary or final_answer. Since OutputItem.phase is optional (unphased/legacy items are valid), this change will suppress streaming for normal assistant output when the server omits phase, and buffered deltas get discarded in flushPendingUnknownPhaseText() when the mapped phase stays undefined. Consider treating undefined phase as visible (emit deltas with no partial.phase and a signature that omits phase), and only suppress when phase resolves to commentary.
| const pendingEntries = [...pendingUnknownPhaseTextByPart.entries()] | ||
| .filter(([key]) => key.startsWith(`${itemId}:`)) | ||
| .sort((a, b) => a[0].localeCompare(b[0])); | ||
| for (const [key, pending] of pendingEntries) { | ||
| pendingUnknownPhaseTextByPart.delete(key); | ||
| emitTextDelta({ | ||
| fullText: pending.fullText, | ||
| deltaText: pending.fullText, | ||
| itemId: pending.itemId, | ||
| contentIndex: pending.contentIndex, | ||
| }); |
There was a problem hiding this comment.
flushPendingUnknownPhaseText() sorts pending parts with a[0].localeCompare(b[0]) on keys like ${itemId}:${contentIndex}. This can misorder content indices lexicographically (e.g., :10 before :2), which could replay buffered text out of order when there are many content parts. Prefer sorting by numeric contentIndex (store it separately or parse it from the key) instead of string compare.
| type AssistantMessageWithPhase = AssistantMessage & { phase?: OpenAIResponsesAssistantPhase }; | ||
|
|
||
| function normalizeAssistantPhase(value: unknown): OpenAIResponsesAssistantPhase | undefined { | ||
| return value === "commentary" || value === "final_answer" ? value : undefined; | ||
| } | ||
|
|
||
| function encodeAssistantTextSignature(params: { | ||
| id: string; | ||
| phase?: OpenAIResponsesAssistantPhase; | ||
| }): string { | ||
| return JSON.stringify({ | ||
| v: 1, | ||
| id: params.id, | ||
| ...(params.phase ? { phase: params.phase } : {}), | ||
| }); | ||
| } |
There was a problem hiding this comment.
normalizeAssistantPhase() and encodeAssistantTextSignature() are duplicated here and in src/agents/openai-ws-message-conversion.ts. To avoid the two implementations drifting (signature format needs to stay consistent with parseAssistantTextSignature()), consider exporting/reusing the shared helpers instead of re-declaring them in both modules.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8e10cf0192
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (!itemPhase) { | ||
| pendingUnknownPhaseTextByPart.forEach((_, key) => { | ||
| if (key.startsWith(`${itemId}:`)) { | ||
| pendingUnknownPhaseTextByPart.delete(key); | ||
| } | ||
| }); |
There was a problem hiding this comment.
Keep buffered deltas until phase is known
In flushPendingUnknownPhaseText, the !itemPhase branch deletes all buffered text for that item, but this function is called on both response.output_item.added and response.output_item.done. Because phase is optional on output items, an early response.output_text.delta can be buffered, then dropped when an unphased output_item.added arrives; if no further delta is appended before output_item.done carries the phase, the initial streamed text is never emitted as text_delta. This causes missing leading output in exactly the late-map case this change is trying to fix.
Useful? React with 👍 / 👎.
Greptile SummaryThis PR addresses two blockers from the #59643 maintainer review: suppressing commentary-phase streaming partials and buffering early (unphased) deltas until item-phase metadata arrives. The phase-aware replay splitting in However, the core fix for Blocker 1 — suppressing commentary deltas in the WS streaming path — contains a logic error that makes the suppression inoperative:
Confidence Score: 2/5Not safe to merge — the primary stated fix (commentary suppression) is broken and the accompanying test would fail. The
Prompt To Fix All With AIThis is a comment left during a code review.
Path: src/agents/openai-ws-stream.ts
Line: 928-931
Comment:
**Commentary phase not suppressed — `emitTextDelta` emits for all known phases**
The guard `if (!itemPhase)` only suppresses events when the phase is *unknown* (`undefined`). Because `normalizeAssistantPhase("commentary")` returns the string `"commentary"` (truthy), this check evaluates to `false` for commentary items and the `text_delta` event is pushed to `eventStream`.
This directly contradicts the PR's stated Blocker 1 goal ("commentary buffered text stays suppressed") and the test `"buffers early text deltas until phase metadata arrives and suppresses commentary output"`, which asserts zero `text_delta` events after a commentary item is identified. Both the late-map flush path and the early-map direct-emit path flow through this function, so commentary text leaks in both scenarios.
The fix is to change the condition from `!itemPhase` to `itemPhase !== "final_answer"` so that only `final_answer` text is forwarded:
```suggestion
const itemPhase = normalizeAssistantPhase(outputItemPhaseById.get(params.itemId));
if (itemPhase !== "final_answer") {
return;
}
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: src/agents/openai-ws-stream.ts
Line: 957-965
Comment:
**`flushPendingUnknownPhaseText` emits buffered commentary text instead of discarding it**
When the resolved phase is `"commentary"` (truthy), `if (!itemPhase)` evaluates to `false` and execution falls through to the `emitTextDelta` loop. `emitTextDelta` has the same `!itemPhase` guard (see companion comment), so the accumulated commentary text is emitted as a `text_delta` event.
The suppression guard here should mirror the fix to `emitTextDelta`:
```suggestion
const flushPendingUnknownPhaseText = (itemId: string) => {
const itemPhase = normalizeAssistantPhase(outputItemPhaseById.get(itemId));
if (itemPhase !== "final_answer") {
pendingUnknownPhaseTextByPart.forEach((_, key) => {
if (key.startsWith(`${itemId}:`)) {
pendingUnknownPhaseTextByPart.delete(key);
}
});
return;
}
```
With this change, unknown-phase items *and* commentary items are both discarded from the pending buffer, while `final_answer` items are flushed and emitted as before.
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "fix(agents): add phase-resolution fallba..." | Re-trigger Greptile |
| const itemPhase = normalizeAssistantPhase(outputItemPhaseById.get(params.itemId)); | ||
| if (!itemPhase) { | ||
| return; | ||
| } |
There was a problem hiding this comment.
Commentary phase not suppressed —
emitTextDelta emits for all known phases
The guard if (!itemPhase) only suppresses events when the phase is unknown (undefined). Because normalizeAssistantPhase("commentary") returns the string "commentary" (truthy), this check evaluates to false for commentary items and the text_delta event is pushed to eventStream.
This directly contradicts the PR's stated Blocker 1 goal ("commentary buffered text stays suppressed") and the test "buffers early text deltas until phase metadata arrives and suppresses commentary output", which asserts zero text_delta events after a commentary item is identified. Both the late-map flush path and the early-map direct-emit path flow through this function, so commentary text leaks in both scenarios.
The fix is to change the condition from !itemPhase to itemPhase !== "final_answer" so that only final_answer text is forwarded:
| const itemPhase = normalizeAssistantPhase(outputItemPhaseById.get(params.itemId)); | |
| if (!itemPhase) { | |
| return; | |
| } | |
| const itemPhase = normalizeAssistantPhase(outputItemPhaseById.get(params.itemId)); | |
| if (itemPhase !== "final_answer") { | |
| return; | |
| } |
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/openai-ws-stream.ts
Line: 928-931
Comment:
**Commentary phase not suppressed — `emitTextDelta` emits for all known phases**
The guard `if (!itemPhase)` only suppresses events when the phase is *unknown* (`undefined`). Because `normalizeAssistantPhase("commentary")` returns the string `"commentary"` (truthy), this check evaluates to `false` for commentary items and the `text_delta` event is pushed to `eventStream`.
This directly contradicts the PR's stated Blocker 1 goal ("commentary buffered text stays suppressed") and the test `"buffers early text deltas until phase metadata arrives and suppresses commentary output"`, which asserts zero `text_delta` events after a commentary item is identified. Both the late-map flush path and the early-map direct-emit path flow through this function, so commentary text leaks in both scenarios.
The fix is to change the condition from `!itemPhase` to `itemPhase !== "final_answer"` so that only `final_answer` text is forwarded:
```suggestion
const itemPhase = normalizeAssistantPhase(outputItemPhaseById.get(params.itemId));
if (itemPhase !== "final_answer") {
return;
}
```
How can I resolve this? If you propose a fix, please make it concise.| const flushPendingUnknownPhaseText = ( | ||
| itemId: string, | ||
| fallbackPhase?: OpenAIResponsesAssistantPhase, | ||
| ) => { | ||
| const itemPhase = | ||
| normalizeAssistantPhase(outputItemPhaseById.get(itemId)) ?? fallbackPhase; | ||
| if (!itemPhase) { | ||
| return; | ||
| } |
There was a problem hiding this comment.
flushPendingUnknownPhaseText emits buffered commentary text instead of discarding it
When the resolved phase is "commentary" (truthy), if (!itemPhase) evaluates to false and execution falls through to the emitTextDelta loop. emitTextDelta has the same !itemPhase guard (see companion comment), so the accumulated commentary text is emitted as a text_delta event.
The suppression guard here should mirror the fix to emitTextDelta:
| const flushPendingUnknownPhaseText = ( | |
| itemId: string, | |
| fallbackPhase?: OpenAIResponsesAssistantPhase, | |
| ) => { | |
| const itemPhase = | |
| normalizeAssistantPhase(outputItemPhaseById.get(itemId)) ?? fallbackPhase; | |
| if (!itemPhase) { | |
| return; | |
| } | |
| const flushPendingUnknownPhaseText = (itemId: string) => { | |
| const itemPhase = normalizeAssistantPhase(outputItemPhaseById.get(itemId)); | |
| if (itemPhase !== "final_answer") { | |
| pendingUnknownPhaseTextByPart.forEach((_, key) => { | |
| if (key.startsWith(`${itemId}:`)) { | |
| pendingUnknownPhaseTextByPart.delete(key); | |
| } | |
| }); | |
| return; | |
| } |
With this change, unknown-phase items and commentary items are both discarded from the pending buffer, while final_answer items are flushed and emitted as before.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/openai-ws-stream.ts
Line: 957-965
Comment:
**`flushPendingUnknownPhaseText` emits buffered commentary text instead of discarding it**
When the resolved phase is `"commentary"` (truthy), `if (!itemPhase)` evaluates to `false` and execution falls through to the `emitTextDelta` loop. `emitTextDelta` has the same `!itemPhase` guard (see companion comment), so the accumulated commentary text is emitted as a `text_delta` event.
The suppression guard here should mirror the fix to `emitTextDelta`:
```suggestion
const flushPendingUnknownPhaseText = (itemId: string) => {
const itemPhase = normalizeAssistantPhase(outputItemPhaseById.get(itemId));
if (itemPhase !== "final_answer") {
pendingUnknownPhaseTextByPart.forEach((_, key) => {
if (key.startsWith(`${itemId}:`)) {
pendingUnknownPhaseTextByPart.delete(key);
}
});
return;
}
```
With this change, unknown-phase items *and* commentary items are both discarded from the pending buffer, while `final_answer` items are flushed and emitted as before.
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7097cf5865
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const itemPhase = | ||
| event.item.type === "message" | ||
| ? normalizeAssistantPhase((event.item as { phase?: unknown }).phase) | ||
| : undefined; | ||
| outputItemPhaseById.set(event.item.id, itemPhase); |
There was a problem hiding this comment.
Keep prior phase when output item updates are unphased
This handler resets outputItemPhaseById to undefined whenever an item update arrives without item.phase, even if an earlier response.output_item.added already established final_answer. In that case, later response.output_text.done events for the same item are treated as unknown-phase, buffered, and then flushed on completion with deltaText equal to the full accumulated text, which can replay already-streamed content and duplicate user-visible output.
Useful? React with 👍 / 👎.
| const finalAssistantPhase = | ||
| assistantPhases.size === 1 && !hasUnphasedAssistantText ? [...assistantPhases][0] : undefined; |
There was a problem hiding this comment.
Keep done-message phase usable for mixed commentary/final output
This change removes the top-level assistant phase whenever a response includes mixed phases, but downstream delivery logic suppresses visibility based on the first phased text block when top-level phase is absent. For the common ordering of commentary first then final answer, handleMessageEnd can classify the whole message as commentary and return early; if no prior streamed assistant update was emitted, the final answer is dropped entirely.
Useful? React with 👍 / 👎.
|
Closing — #59643 landed with maintainer fixes that resolve the core blockers addressed here. The embedded subscribe commentary suppression (blocker 1) was already on main via #61282, and steipete tightened the WS-side phase handling (blocker 2) directly in the merge. I'll check whether any narrow hardening / test-coverage delta remains and spin it as a clean follow-up PR if needed, rather than keeping overlap around. |
Summary
Addresses the two specific blockers from the maintainer review on #59643, building on the architectural direction of that PR while fixing the behaviors that blocked its merge.
Blocker 1: Commentary partials must stay suppressed
The merged fix from #61282 already suppresses commentary-phase output at the embedded subscribe delivery boundary on current
main. This PR preserves that behavior and does not reintroduce the commentary-preview-then-replace pattern that was rejected in #59643.Blocker 2: Late-map unphased deltas must be buffered
In
src/agents/openai-ws-stream.ts, whenresponse.output_text.deltaarrives beforeresponse.output_item.added, the delta is now buffered instead of emitted as an unphased visible partial. When item metadata arrives:final_answerbuffered text is emitted with correct phase/signaturecommentarybuffered text stays suppressedArchitectural improvements (from #59643 direction)
openai-ws-message-conversion.ts: stored/replayed assistant text respects per-blocktextSignature.phaseand splits by phase instead of flatteningtextSignaturephaseonly set when response is consistently single-phaseWhat this does NOT cover (known follow-up scope)
src/agents/tools/sessions-helpers.tsstill uses phase-blindextractAssistantText()— follow-up PR recommendedtui-formatters.ts,tui-stream-assembler.ts) still phase-blind — follow-up PR recommendedThese are explicitly out of scope per the maintainer direction to make the core WS → embedded subscribe path correct first.
Change Type
Scope
Linked Issues
Tests
Tests added for:
Credits
Architectural direction from @ringlochid in #59643. Review feedback from @steipete that identified the two specific blockers addressed here.