Skip to content

[Bug]: Streaming text silently dropped when tool boundary arrives during fallback mode #8124

@austinmw

Description

@austinmw

Bug Description

When streaming to Telegram, if a message edit fails (flood control) and the agent then calls a tool before the stream finishes, text that was already generated is silently dropped. The user sees a frozen partial message with ▉ and the missing words are never delivered.

This is distinct from #7183, which covers only the final cursor-removal edit failing after full text was already delivered. This bug loses actual content.

Steps to Reproduce

Save the script below as repro.py in the repo root and run:

source venv/bin/activate && python repro.py
Reproduction script (verified on v0.8.0)
import asyncio, sys
sys.path.insert(0, ".")
from gateway.platforms.base import SendResult
from gateway.stream_consumer import GatewayStreamConsumer, StreamConsumerConfig

class FakeTelegram:
    """Simulates Telegram with persistent flood control once triggered."""
    MAX_MESSAGE_LENGTH = 4096
    def __init__(self):
        self.visible_messages = {}
        self.message_order = []
        self.edit_log = []
        self._next_id = 1
        self._flood_active = False
    async def send(self, *, chat_id, content, reply_to=None, metadata=None):
        mid = f"msg_{self._next_id}"; self._next_id += 1
        self.visible_messages[mid] = content
        self.message_order.append(mid)
        return SendResult(success=True, message_id=mid)
    async def edit_message(self, *, chat_id, message_id, content):
        self.edit_log.append(f"EDIT {message_id}: {content!r}")
        if self._flood_active:
            self.edit_log[-1] += "  <-- FAILED (flood still active)"
            return SendResult(success=False, error="flood_control:6.0")
        if "more" in content and content.endswith(" ▉"):
            self._flood_active = True
            self.edit_log[-1] += "  <-- FAILED (flood control triggered)"
            return SendResult(success=False, error="flood_control:6.0")
        self.visible_messages[message_id] = content
        return SendResult(success=True, message_id=message_id)
    def truncate_message(self, text, limit):
        return [text] if len(text) <= limit else [text[i:i+limit] for i in range(0, len(text), limit)]

async def main():
    tg = FakeTelegram()
    consumer = GatewayStreamConsumer(tg, "chat_123",
        StreamConsumerConfig(edit_interval=0.01, buffer_threshold=5, cursor=" ▉"))
    consumer.on_delta("Hello")
    task = asyncio.create_task(consumer.run())
    await asyncio.sleep(0.08)
    consumer.on_delta(" world"); await asyncio.sleep(0.08)
    consumer.on_delta(" more"); await asyncio.sleep(0.08)  # this edit fails
    consumer.on_delta(None)  # tool boundary
    consumer.on_delta("Here is the tool result.")
    consumer.finish()
    await task
    visible = " ".join(tg.visible_messages[m] for m in tg.message_order)
    for e in tg.edit_log: print(e)
    for mid in tg.message_order: print(f"{mid}: {tg.visible_messages[mid]!r}")
    print(f"Agent generated: \"Hello world more\" + [tool] + \"Here is the tool result.\"")
    print(f"User received:   {visible!r}")
    print("REPRODUCED" if "more" not in visible else "NOT REPRODUCED")

asyncio.run(main())

Expected Behavior

All generated text is delivered to the user, even when progressive edits fail and tool boundaries occur mid-stream.

Actual Behavior

Text buffered after the edit failure but before the tool boundary is silently dropped:

EDIT msg_1: 'Hello world ▉'
EDIT msg_1: 'Hello world more ▉'  <-- FAILED (flood control triggered)
EDIT msg_1: 'Hello world more'  <-- FAILED (flood still active)

msg_1: 'Hello world ▉'
msg_2: 'Here is the tool result.'

Agent generated: "Hello world more" + [tool] + "Here is the tool result."
User received:   'Hello world ▉ Here is the tool result.'

REPRODUCED

The word "more" was generated but never delivered. The user also sees a stuck ▉ cursor in msg_1.

Affected Component

Gateway (Telegram/Discord/Slack/WhatsApp)

Messaging Platform

Telegram

Operating System

Ubuntu 24.04 (aarch64 — NVIDIA DGX Spark)

Python Version

3.11.15

Hermes Version

v0.8.0 (2026.4.8)

Root Cause Analysis

Verified sequence with line numbers from gateway/stream_consumer.py:

  1. Edit fails → _send_or_edit() enters fallback mode (lines 545-547): sets _fallback_prefix, _fallback_final_send=True, _edit_supported=False
  2. Text continues accumulating in _accumulated via queue drain (line 159) regardless of _edit_supported
  3. _NEW_SEGMENT arrives → run() calls _reset_segment_state(preserve_no_edit=True) at line 272
  4. _reset_segment_state() (lines 113-117) clears _accumulated, _fallback_final_send, and _fallback_prefix. The preserve_no_edit guard only protects the __no_edit__ sentinel — not normal fallback state
  5. _DONE arrives → the check at line 241 (if self._fallback_final_send) is now False, and _accumulated only contains the post-boundary text. Pre-boundary text is gone.

Related: #7183 (covers only the narrower final-cursor-removal case)

Proposed Fix

Before resetting segment state on _NEW_SEGMENT (line 272), flush the pending fallback continuation. Or add a guard in _reset_segment_state() to preserve fallback state when _fallback_final_send is True.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions