fix(gateway): flush undelivered tail before segment reset (#8124)#12414
Merged
Conversation
… streamed text (#8124) When a streaming edit fails mid-stream (flood control, transport error) and a tool boundary arrives before the fallback threshold is reached, the pre-boundary tail in `_accumulated` was silently discarded by `_reset_segment_state`. The user saw a frozen partial message and missing words on the other side of the tool call. Flush the undelivered tail as a continuation message before the reset, computed relative to the last successfully-delivered prefix so we don't duplicate content the user already saw.
This was referenced Apr 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Text generated between a mid-stream edit failure and a tool boundary is no longer silently dropped. Root cause:
_reset_segment_state()on a tool boundary wiped_accumulatedeven when the most recent edit had failed, discarding un-delivered content.The fix flushes the undelivered tail as a continuation message before the segment reset, computed relative to the last successfully-delivered prefix so it doesn't duplicate what the user already saw. Best-effort cursor strip on the partial message is also attempted when fallback mode hasn't already done so.
Changes
gateway/stream_consumer.py— new_flush_segment_tail_on_edit_failure()helper called before_reset_segment_state()on segment breaks, guarded on_accumulated and not current_update_visible and _message_id and _message_id != "__no_edit__".tests/gateway/test_stream_consumer.py— newtest_segment_break_after_mid_stream_edit_failure_preserves_tail(matches austinmw's repro script verbatim) + updated existingtest_segment_break_clears_failed_edit_fallback_statewhich had inadvertently codified the drop-the-tail behavior.Validation
tests/gateway/test_stream_consumer.pyUser received: 'Hello world ▉ Here is the tool result.'(" more" dropped)User received: 'Hello world ▉ more Here is the tool result.'(all text delivered)Closes
Credit
1bb83a98via cherry-pick._fallback_final_send=True, which only latches after_MAX_FLOOD_STRIKES=3consecutive failures — in the actual bug scenario fallback isn't yet armed when the tool boundary arrives, so that approach doesn't fully cover the bug. fix(gateway): flush undelivered tail before segment reset (#8124) #11974's condition (_accumulated AND not current_update_visible) fires on any unsuccessful segment-break edit and handles the common pre-fallback case.