fix: prevent false StreamIdleTimeout when ToolCallTextFilter suppresses SSE events#720
Merged
Merged
Conversation
…es SSE events (#717) When the ToolCallTextFilter detects <tool_call> XML in streaming text, it suppresses all subsequent SSE updates via `continue`, bypassing the `yield return`. This creates a watchdog blackout — no LlmResponseDeltaReceived messages reach the session actor, so ProcessingWatchdog.Refresh() is never called and the StreamIdleTimeout (default 120s) fires even though the GPU is actively generating tokens. Fix: yield a content-free keepalive ChatResponseUpdate when text is suppressed, and send a keepalive LlmResponseDeltaReceived with empty TextContent when no content is dispatched for an SSE event. The actor's delta handler refreshes the watchdog unconditionally before the content-type switch, so empty TextContent resets the timer without emitting visible output.
This was referenced Apr 22, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #717
ToolCallTextFilterdetects<tool_call>XML in streaming text, it suppresses all subsequent SSE updates viacontinue, bypassingyield return. This creates a watchdog blackout — noLlmResponseDeltaReceivedmessages reach the session actor, soProcessingWatchdog.Refresh()is never called and theStreamIdleTimeout(default 120s) fires even though the GPU is actively generating tokens.ChatResponseUpdatewhen text is suppressed, and sends a keepaliveLlmResponseDeltaReceivedwith emptyTextContentwhen no content is dispatched for an SSE event. The actor's delta handler refreshes the watchdog unconditionally before the content-type switch, so emptyTextContentresets the timer without emitting visible output.Changes
OpenAiCompatibleChatClient.csChatResponseUpdatewhenToolCallTextFiltersuppresses textSessionLlmInvoker.csdispatchedflag; send keepalive delta when no content dispatchedOpenAiCompatibleChatClientTests.csTest plan
StreamingYieldsKeepaliveUpdates_WhenToolCallFilterSuppressesText— verifies keepalive updates are yielded when filter suppresses textStreamingKeepalive_ToolCallStillExtracted_AfterSuppression— verifies tool call extraction still works with keepalivesOpenAiCompatibleChatClientTestspassLlmSessionWatchdogTestspass (watchdog integration)