Skip to content

Regression on v2026.5.22: event-loop starvation returns (87s session-lock phase, 31s loop delay) — ref #80695 #86509

Closed
#86783
Closed
Regression on v2026.5.22: event-loop starvation returns (87s session-lock phase, 31s loop delay) — ref #80695#86509
#86783
Labels
P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.
@vdruts

Description

@vdruts
vdruts
opened on May 25, 2026

Summary

Re-filing because the prior ticket (#80695, closed) covered the same root pattern on v2026.5.7 and the regression is now present on v2026.5.22. Rolled back to v2026.5.20 to recover; 5.22 added to local BLOCKED_VERSIONS.

Environment

Symptoms observed after upgrading 5.20 → 5.22

  1. Telegram getMe calls timing out at 10s despite 31s elapsed (timer delayed, loop blocked).
  2. Health monitor flapping — Alan, Alex-Demo, Clawd, Cllawway all marked stopped within 8s of each other → cascade restart.
  3. Interactive agent responses very slow even outside the cascade window.
  4. dir_list failed: unknown node: gateway — tool can no longer resolve gateway as a workspace node (possibly a registry change between 5.20 and 5.22).

Trigger pattern

Two agents active concurrently:

  • agent:tony:main — interactive embedded_run, 49s age
  • agent:alex:cron:alex-daily-state-refresh — model_call, 31s age

Sidecar phase that blocks the loop: sidecars.session-locks:87319ms (87 seconds on the main thread).

Log excerpt

2026-05-25T08:32:12.367-04:00 [fetch-timeout] fetch timeout after 9999ms (elapsed 31195ms) timer delayed 21196ms, likely event-loop starvation operation=fetchWithTimeout url=https://api.telegram.org/bot875230…bMY4/getMe
2026-05-25T08:32:12.373-04:00 [diagnostic] liveness warning: reasons=event_loop_delay,event_loop_utilization,cpu interval=32s eventLoopDelayP99Ms=31172.1 eventLoopDelayMaxMs=31172.1 eventLoopUtilization=1 cpuCoreRatio=1.014 active=2 waiting=0 queued=0 recentPhases=sidecars.session-locks:87319ms,post-ready.maintenance:1307ms,channels.telegram.is-configured:0ms,channels.telegram.runtime:0ms,channels.telegram.approval-bootstrap:0ms,channels.telegram.start-account-handoff:1ms work=[active=agent:tony:main(processing/embedded_run,q=1,age=49s last=embedded_run:started)|agent:alex:cron:alex-daily-state-refresh:run:8a5c2c67-f857-445c-b65e-6160a888a641(processing/model_call,q=0,age=31s last=model_call:started)]
2026-05-25T08:32:37.784-04:00 [fetch-timeout] fetch timeout after 10000ms (elapsed 23789ms) timer delayed 13789ms, likely event-loop starvation operation=fetchWithTimeout url=https://api.telegram.org/bot874417…SIuo/getMe
2026-05-25T08:32:39.885-04:00 [health-monitor] [telegram:alan] health-monitor: restarting (reason: stopped)
2026-05-25T08:32:43.021-04:00 [health-monitor] [telegram:alex-demo] health-monitor: restarting (reason: stopped)
2026-05-25T08:32:45.176-04:00 [health-monitor] [telegram:clawd] health-monitor: restarting (reason: stopped)
2026-05-25T08:32:47.293-04:00 [health-monitor] [telegram:cllawway] health-monitor: restarting (reason: stopped)
2026-05-25T08:32:52.547-04:00 [fetch-timeout] fetch timeout after 10000ms (elapsed 14759ms) operation=fetchWithTimeout url=https://api.telegram.org/bot858735…o_mg/getMe
2026-05-25T08:32:52.553-04:00 [agent/embedded] [trace:embedded-run] startup stages: runId=d71fa514-689b-4424-9fa2-8efea9847768 sessionId=28b0d924-8324-43db-92f0-1f20e518af70 phase=attempt-dispatch totalMs=20723 stages=workspace:1ms@1ms,runtime-plugins:2ms@3ms,hooks:0ms@3ms,model-resolution:886ms@889ms,auth:5060ms@5949ms,context-engine:0ms@5949ms,attempt-workspace:14774ms@20723ms,attempt-prompt:0ms@20723ms,attempt-runtime-plan:0ms@20723ms,attempt-dispatch:0ms@20723ms
2026-05-25T08:32:56.667-04:00 [tools] dir_list failed: unknown node: gateway raw_params={"node":"gateway","path":"C:\Users\vdrut\clawd\agents\alex-clawdmozi"}
2026-05-25T08:33:18.711-04:00 [agent/embedded] [trace:embedded-run] prep stages: runId=d71fa514-689b-4424-9fa2-8efea9847768 sessionId=28b0d924-8324-43db-92f0-1f20e518af70 phase=stream-ready totalMs=26155 stages=workspace-sandbox:25ms@25ms,skills:0ms@25ms,core-plugin-tools:4060ms@4085ms,bootstrap-context:2300ms@6385ms,bundle-tools:1092ms@7477ms,system-prompt:9706ms@17183ms,session-resource-loader:8956ms@26139ms,agent-session:1ms@26140ms,stream-setup:15ms@26155ms

Two stage breakdowns worth flagging on 5.22:

  • attempt-workspace:14774ms on dispatch — workspace setup blocking nearly 15s per run.
  • system-prompt:9706ms + session-resource-loader:8956ms on stream-ready — ~19s of synchronous prep on the hot path.

Suspected root cause(s)

Same as #80695 — concurrent agent work saturates the event loop because:

  1. sidecars.session-locks runs heavy I/O on the main thread (87s here vs. ~hundreds of ms on 5.20).
  2. attempt-workspace and session-resource-loader look like they regressed in 5.22 — both run synchronously enough to block the loop while another agent is also running.
  3. Whatever absorbed the multi-account start-account starvation fix in Telegram cascade on v2026.5.7: polling stall + outbound HTTP agent never rebuilt; multi-account start-account causes event-loop starvation #80695 appears to have regressed.

Repro

  1. Two agents configured with overlapping schedules (e.g. one interactive Telegram agent + a cron firing a model_call on another agent).
  2. Trigger them within ~30s of each other.
  3. Telegram polling stops responding to getMe; health monitor declares bots stopped; cascade restart.

Workaround

  • Rolled back to v2026.5.20 locally and added 5.22 to BLOCKED_VERSIONS in our update guard.
  • Will re-test against any future release that claims to fix this and update this issue.

Cross-refs

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions