Skip to content

[Bug] Ollama Cloud rate-limit cooldown permanently blocks agents — not released after API recovery #87608

@timo-reuter

Description

@timo-reuter

Bug type

Regression (worked before, now fails)

Beta release blocker

No

Summary

After an Ollama Cloud 429 rate-limit error, OpenClaw's internal rate-limit cooldown permanently blocks agent sessions. Even after the Ollama API fully recovers (confirmed by direct API test returning HTTP 200), blocked agents cannot start new sessions. Only agents with already-active sessions continue to work. Gateway restarts do not clear the cooldown state. Observed in v2026.5.22, persists after upgrade to v2026.5.26.

Steps to reproduce

  1. Set up OpenClaw with 4 agents (Tim, Tea, Kaho, Tom) all using Ollama Cloud (ollama.com) with the same API key and glm-5.1:cloud as primary model.
  2. Configure cron jobs for multiple agents that trigger at the same time (e.g., daily 04:00 memory sanitization).
  3. Wait for a burst of concurrent API calls that triggers a temporary Ollama 429 rate-limit response.
  4. Observe that OpenClaw enters rate-limit cooldown for the affected agents.
  5. Confirm the Ollama API has recovered by making a direct API call (returns HTTP 200 with valid response).
  6. Check the Ollama dashboard: shows only 0.6% weekly usage, 3.5% session usage — nowhere near any real limit.
  7. Attempt to send messages to the blocked agents — they fail with FailoverError: ⚠️ API rate limit reached.
  8. Restart the gateway (openclaw gateway restart) — blocked agents remain blocked.
  9. Fully restart OpenClaw (kill process + restart) — blocked agents remain blocked.
  10. Upgrade from v2026.5.22 to v2026.5.26 — blocked agents remain blocked.
  11. Only agents with pre-existing active sessions (Tim) continue to work on the same API key.

Expected behavior

After a temporary 429 rate-limit error clears, OpenClaw should retry API calls and resume normal operation. Agents should be able to start new sessions and respond to messages once the Ollama API returns to normal (HTTP 200). Gateway restarts should clear any internal rate-limit cooldown state. In v2026.5.22 and earlier, temporary 429 errors were handled with exponential backoff and agents recovered automatically.

Actual behavior

Agents Kaho and Tom are permanently blocked with FailoverError: ⚠️ API rate limit reached. Please try again later. Direct API test with the same key returns HTTP 200. Gateway logs show EmbeddedAttemptSessionTakeoverError: session file changed while embedded prompt lock was released. The Ollama dashboard confirms 0.6% weekly usage — no real limit reached. Only agents with already-active sessions (Tim, Tea) continue to function on the same API key.

OpenClaw version

2026.5.26

Operating system

Windows 11 (10.0.26200) x64

Install method

npm global (npm install -g openclaw)

Model

ollama/glm-5.1:cloud (primary), ollama/kimi-k2.5:cloud, ollama/minimax-m2.7:cloud (fallbacks)

Provider / routing chain

openclaw -> ollama.com (Ollama Cloud)

Additional provider/model setup details

4 agents share the same Ollama Cloud API key (Pro tier). Config in ~/.openclaw/openclaw.json under models.providers.ollama. Each agent has model set to ollama/glm-5.1:cloud with fallbacks. Kaho was updated to use model object with fallbacks (primary + fallbacks[]) but the fallbacks also return 429. Direct Node.js API test with the same key returns HTTP 200, confirming the issue is in OpenClaw's internal rate-limit state, not in Ollama.

Logs, screenshots, and evidence

Impact and severity

  • Affected: All multi-agent setups using Ollama Cloud on shared API keys
  • Severity: High (blocks 2 out of 4 agents completely; they cannot respond to any user messages or run cron jobs)
  • Frequency: Always once triggered (permanent block until manual intervention or unknown cooldown expiry)
  • Consequence: Agents Kaho and Tom are completely unresponsive. Cron jobs (memory sanitization, scheduled tasks) fail with 3 consecutive errors. User-facing messages go unanswered.

Additional information

Last known good: v2026.5.22 (before upgrade, agents worked intermittently). First known bad: v2026.5.22 (after burst of 429 errors, agents became permanently blocked).

Workaround attempted:

  • Deleted and recreated cron jobs (cleared consecutive error counters)
  • Added fallback models to agent config
  • Multiple gateway restarts
  • Full process restart
  • Upgrade to v2026.5.26
  • None resolved the issue

Suggested fix:

  1. Add a configurable cooldown reset mechanism (e.g., openclaw gateway reset-ratelimit)
  2. Clear all rate-limit cooldown state on gateway restart
  3. Implement proper exponential backoff with automatic retry instead of permanent block
  4. Respect Retry-After headers from Ollama instead of indefinite cooldown
  5. Direct API test proof: Node.js https.request to ollama.com/v1/chat/completions with the same API key returns HTTP 200 immediately, while OpenClaw's internal tracker continues returning FailoverError

Metadata

Metadata

Assignees

Labels

P1High-priority user-facing bug, regression, or broken workflow.bugSomething isn't workingclawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions