Skip to content

[Bug] Telegram: Pool timeout causes silent send failures — no retry or fallback on exhausted general pool #35610

@q3874758

Description

@q3874758

Bug Description

Error from gateway.log:

Frequency: Recurring — observed multiple times on 2026-05-23 and 2026-05-30.

Affected version: Hermes Agent v0.15.1 (2026.5.29), Windows 10


Root Cause Analysis

The Telegram adapter's httpx connection pool (, via ) can become saturated under the following conditions:

  1. Temporary network latency spikes (proxy/firewall packet loss)
  2. Slow Telegram API responses during high traffic periods
  3. Connection pool exhaustion during or after reconnect storms

When the general pool ( in PTB terminology) is saturated:

  • New / calls immediately fail with
  • The error is caught, logged as , and the message is dropped silently
  • The flag (from PR fix(telegram): detect wedged send path after reconnect storms (#31165) #31441) is only set during reconnect events — not on general pool exhaustion — so it provides no protection for this case
  • The existing of 8s is already an improvement over PTB's 1s default, but when the pool is fully occupied, any new request times out immediately rather than waiting or retrying

Existing mitigations that don't fully cover this:


Expected Behavior

When the general pool is exhausted during a send operation:

  1. Immediate retry — try once more on the same pool (same behavior as all retryable errors)
  2. Fallback to standalone HTTP — if retry fails, fall back to a fresh HTTP session (like the path does during reconnect) rather than dropping the message
  3. Exponential back-off — respect Telegram's 429 rate-limit headers rather than spinning

Impact

  • User-facing silent drops: Trade confirmations, cron reports, and agent responses silently fail to deliver
  • No user notification: The message only appears in gateway logs, not to the user
  • Requires manual intervention: Gateway must be manually restarted to clear the wedged pool state in some cases

Environment

  • OS: Windows 10 (x86_64)
  • Hermes Agent: v0.15.1 (2026.5.29)
  • Python: 3.11.15
  • Platform: Telegram (bot token configured, polling mode)
  • Related logs: lines showing

Related Issues / PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High — major feature broken, no workaroundcomp/gatewayGateway runner, session dispatch, deliveryplatform/telegramTelegram bot adaptertype/bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions