fix(gateway): flush undelivered tail before segment reset (#8124)#11974
Closed
konsisumer wants to merge 1 commit into
Closed
fix(gateway): flush undelivered tail before segment reset (#8124)#11974konsisumer wants to merge 1 commit into
konsisumer wants to merge 1 commit into
Conversation
… streamed text (NousResearch#8124) When a streaming edit fails mid-stream (flood control, transport error) and a tool boundary arrives before the fallback threshold is reached, the pre-boundary tail in `_accumulated` was silently discarded by `_reset_segment_state`. The user saw a frozen partial message and missing words on the other side of the tool call. Flush the undelivered tail as a continuation message before the reset, computed relative to the last successfully-delivered prefix so we don't duplicate content the user already saw.
Contributor
|
Your implementation is landed in #12414 — commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #8124.
When a streaming edit fails mid-stream (e.g. Telegram flood control) and a tool boundary arrives before the fallback-strike threshold promotes the consumer to fallback mode, the pre-boundary tail in
_accumulatedwas silently discarded by_reset_segment_state. The user saw a frozen partial message with a stuck cursor and missing words on the other side of the tool call.The fix flushes the undelivered tail as a continuation message before the segment reset, computed relative to the last successfully-delivered prefix so it doesn't duplicate content the user already saw. Best-effort cursor strip on the partial message is also attempted when fallback mode hasn't already done so.
Test plan
ghcr.io/astral-sh/uv:python3.11-trixie).NOT REPRODUCED—"more"is delivered as a continuation message.test_segment_break_after_mid_stream_edit_failure_preserves_tailthat asserts the pre-boundary tail reaches the user.test_segment_break_clears_failed_edit_fallback_statewhich inadvertently codified the drop-the-tail behavior; it now also asserts the tail flush without prefix duplication.tests/gateway/test_stream_consumer.pycases pass in Docker.