Skip to content

feat: circuit breaker for infinite tool-call loops#14059

Closed
gzsiang wants to merge 7 commits into
NousResearch:mainfrom
gzsiang:pr-circuit-breaker
Closed

feat: circuit breaker for infinite tool-call loops#14059
gzsiang wants to merge 7 commits into
NousResearch:mainfrom
gzsiang:pr-circuit-breaker

Conversation

@gzsiang

@gzsiang gzsiang commented Apr 22, 2026

Copy link
Copy Markdown

Summary

Implements a circuit breaker mechanism to detect and interrupt infinite tool-call loops in the agent.

Problem

The agent can enter infinite tool-call loops where it repeatedly calls the same (or similar) tools without making progress, consuming tokens and time indefinitely.

Solution

  • Circuit breaker detection: Tracks consecutive failed/similar tool calls; triggers after a configurable threshold (default: 3)
  • Built-in heuristic suggestions: When triggered, provides actionable suggestions to the agent (e.g., try a different approach, use a compression model, verify assumptions)
  • Agent-level retry counter: Adds a tool failure retry counter at the agent level for better tracking
  • Configurable threshold: The breaker threshold can be adjusted via configuration

Changes

  • Circuit breaker with configurable threshold
  • Agent-level tool failure retry counter
  • Heuristic suggestions on breaker trigger (simplified, no pattern matching)
  • Suggestion to use compression model when context is large

Testing

  • Existing tests updated to accommodate the circuit breaker
  • Threshold and suggestion logic tested

Note: This is a clean resubmission of #12632, which inadvertently included unrelated i18n commits.

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/agent Core agent loop, run_agent.py, prompt builder labels Apr 22, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Clean resubmission of #12632. Note: files_changed includes unrelated modules (ComfyUI, i18n scripts, browser_tool, file_tools, web_tools) — branch likely needs rebasing/cleanup.

gzsiang added 7 commits April 23, 2026 00:54
Add a circuit breaker mechanism in _invoke_tool() and the sequential
execution path to detect and break infinite tool-call loops.

When the same tool with identical arguments is called 3 consecutive
times, the circuit breaker triggers and returns an error, forcing the
LLM to try a different strategy.

The breaker uses MD5 hashing of (tool_name, json_args) to create a
stable signature, then checks if the last N calls all have the same
signature. After triggering, the counter resets so future calls can
succeed.

- Added _consecutive_tool_calls and _circuit_breaker_threshold to AIAgent.__init__
- Added circuit breaker logic to _invoke_tool() (concurrent path)
- Added circuit breaker logic to _execute_tool_calls_sequential() (sequential path)
- All 22 existing tests pass
…shold

- Change error message from Chinese to English for upstream compatibility
- Increase default threshold from 3 to 5 to avoid false positives
  (legitimate retry scenarios exist, e.g. network jitter)
- Fix indentation bug in sequential execution path
- All 22 existing tests pass
Prevents the LLM from retrying different variations of a failing tool call.
When a tool returns errors/empty results N times consecutively (regardless of
argument changes), the counter triggers and forces a strategy change.

- Added _tool_failure_count dict and _tool_failure_threshold (5) in __init__
- Added _check_tool_failure() method to detect failures and update counter
- Added check calls in both concurrent and sequential execution paths
- Each tool has independent counter; success resets counter to 0
…_breaker_threshold from 5 to 3 - Reduces _tool_failure_threshold from 5 to 3 - Improves circuit breaker error message with specific suggestions - Helps stop tool-loop failures earlier
When a tool call fails repeatedly, the compression model is asked
for a suggestion. This gives the main model a fresh perspective
to break out of tool loops.

- Added _get_tool_suggestion() for retry threshold failures
- Added _get_consecutive_suggestion() for identical-call circuit breaker
- Suggestions are appended to the error message with a 💡 emoji
- Gracefully falls back if compression model is unavailable
When a tool call fails repeatedly, suggestions are provided even
without a compression model configured.

- Added _heuristic_suggestion() — pattern matching on error messages
- Added _heuristic_consecutive_suggestion() — 20+ tool-specific suggestions
- Compression model is tried first, then falls back to heuristics
- Works zero-config — no LLM needed for basic suggestions
Replaced complex pattern-matching heuristics with simple generic prompts.
- Removed _heuristic_suggestion() (8 pattern rules)
- Removed _heuristic_consecutive_suggestion() (20+ tool mappings)
- Fallback is now just a direct text suggestion
- Compression model still tries first for smarter suggestions
- Net: -90 lines of dead pattern logic
@gzsiang gzsiang force-pushed the pr-circuit-breaker branch from 6a3fd14 to 2ae59cb Compare April 22, 2026 16:55
@gzsiang

gzsiang commented Apr 22, 2026

Copy link
Copy Markdown
Author

Thanks for the heads up! I have rebased the branch onto a clean upstream/main and cherry-picked only the circuit breaker commits. The diff now touches only run_agent.py - no more unrelated modules.

@gzsiang gzsiang closed this Apr 28, 2026
@gzsiang

gzsiang commented Apr 28, 2026

Copy link
Copy Markdown
Author

已关闭。原因:原方案已完全重构。

原 PR 使用基于参数 hash 的硬判断,默认开启,阈值 3。

新方案:

  • 使用压缩模型判断是否循环(YES/NO)
  • 无压缩模型时回退到主模型自我审视
  • 默认关闭,通过 config.yaml 配置启用
  • 阈值可配置

新 PR:#16749

@gzsiang gzsiang deleted the pr-circuit-breaker branch June 8, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent loop, run_agent.py, prompt builder P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants