feat: circuit breaker with compression model judgment#16749
Conversation
1d51371 to
775c3f7
Compare
|
Thanks for the review, @alt-glitch! I've rebased the branch onto latest The i18n changes have been moved to a separate branch and will be submitted as a separate PR. Regarding related issues:
The PR is now ready for review. |
|
Hey @gzsiang, nice work on the circuit breaker concept! The compression-model-as-judge approach is clever. A couple of things I noticed: 1. In # Circuit breaker: track consecutive calls (before execution)
...
_cb_retry_msg = self._check_tool_failure(function_name, function_result) # ← function_result not yet defined
if _cb_retry_msg is not None:
return json.dumps({"error": _cb_retry_msg}, ensure_ascii=False)
if function_name == "todo": # tool execution starts hereAt this point 2. Failure detection is inactive in sequential mode In 3. Minor: I don't see them called anywhere in the diff — they may be dead code. Hope this helps! |
663a2fb to
af8ce08
Compare
|
The two-layer design is thoughtful: compression model as primary judge, main model self-reflection as fallback. This avoids a circular dependency where the main model would need to judge its own looping behavior.
Clean single-file implementation. Worth merging with documentation. |
|
Thanks for the thorough review, @liuhao1024! All three issues are now fixed in the latest commit: 1. NameError fix — Removed the misplaced 2. Sequential path now active — Added a 3. Dead code removed — All circuit-breaker tests still pass ( |
|
@ether-btc Thanks for the thoughtful questions! Addressing each: 1. Threshold calibration Regarding consecutive vs cumulative: the current implementation is strictly consecutive — the counter resets when a different tool is called. This is intentional: cumulative counting would produce false positives on agents that legitimately use the same tool at different stages of a task. 2. State persistence across restarts 3. Self-reflection depth / loop risk 4. Separate opt-out for compression model judgment Appreciate the review! 🙏 |
e94797d to
f8b6fe0
Compare
…avior Replace hard-coded circuit breaker with two-layer approach: 1. Consecutive call tracking: Count consecutive calls to the same tool (regardless of success/failure). When threshold (5) is reached, trigger a loop check. 2. Compression model judgment: Ask the compression model to analyze if the agent is looping/repeating useless work. This is closer to how a human would notice repetitive behavior. 3. Failure tracking: Keep the existing failure counter for tools that keep returning errors. Key changes: - Remove _circuit_breaker_threshold (old identical-args breaker) - Add _consecutive_threshold = 5 (new consecutive call threshold) - Add _check_tool_loop() method that asks compression model: "Is the agent looping?" based on recent tool calls and results - If compression model says YES -> trigger circuit breaker - If compression model says NO -> reset counter and allow continuation - Examples in prompt: reading same file repeatedly = YES, reading different sections = NO, same failing command = YES, different params = NO This avoids false positives (e.g. reading a large file in chunks) while catching actual loops (e.g. repeatedly reading the same file).
When compression model is unavailable (not configured), fall back to letting the main model self-reflect — show it the consecutive call count and last result, ask it to assess if it's looping. Two paths: - With compression model: fast judgment by auxiliary model (YES/NO) - Without compression model: main model self-reflection prompt Both paths avoid hard-coded rules, using language model judgment instead of mechanical thresholds.
…eaker
Circuit breaker is now disabled by default. Enable via config.yaml:
circuit_breaker:
enabled: true
consecutive_threshold: 5 # calls before asking compression model
failure_threshold: 5 # consecutive failures before breaker
Both layers (consecutive call check and failure check) are gated by
this config option. When disabled, no tracking occurs, adding zero
overhead.
… model - Replace compression model references with auxiliary model - Update method names: _try_compression_model_* -> _try_auxiliary_model_* - Update documentation
…e dead code - Fix NameError: _check_tool_failure() was called before tool dispatch in _invoke_tool(), where function_result was not yet defined. Moved the call to after the tool result is available (handle_function_call path). - Fix silent no-op in sequential mode: _execute_tool_calls_sequential() incremented the consecutive counter but never called _check_tool_failure(), so Layer 2 failure detection was inactive in sequential execution. Added the call after _detect_tool_failure() and before result is appended. - Remove dead code: _get_consecutive_suggestion() and _try_auxiliary_model_consecutive() were never called anywhere in the diff; the same logic is handled by _check_tool_loop(). Removed both methods.
- Layer 2 (failure check): detect Hermes non-JSON error markers
([error], Error executing tool, Error:, startswith(Error))
instead of resetting failure counter on JSONDecodeError
- Self-reflection fallback: hard kill message ("YOU ARE STUCK IN A LOOP.
DO NOT retry this tool.") instead of soft suggestion
f8b6fe0 to
86405c4
Compare
… reasoning_content parsing
Circuit Breaker Redesign: From Loop Detection to Suggestion-Based BreakThis commit significantly refactors the circuit breaker, shifting from a loop detection model to an auxiliary model suggestion mechanism. Core Changes1. Phase 1 bug fix 2. Phase 2 redesign: from detector to breaker
3. Fixed auxiliary model response parsing 4. All circuit breaker messages localized to Chinese Test ResultsSetup: Hermes Agent + Qwen3.6-35B-A3B (main model, IQ4_XS quantized) + Qwen3.5-4B (auxiliary model) Scenario: The main model was stuck in a loop repeatedly calling with persistent errors during an HTML parsing task. Actual session log excerpt: Key finding:
|
1. "error" in data 在 error=null 时也返回 True(键存在即判定为失败),
导致 exit_code=0 的正常结果被误判为失败。
修复:改为 data.get("error") is not None
2. terminal/execute_code 工具的结果包含 exit_code 字段,但原代码
non-JSON 分支检查的是 "exit code 1"(空格),与 JSON 中的
"exit_code"(下划线)不匹配,导致 exit_code!=0 时也无法被检测。
修复:在 JSON 分支中增加 exit_code != 0 的检查。
详见会话 20260511_121344_c5f11b:
- 真实失败 2 次(exit_code=1)
- 误判失败 2 次(exit_code=0,error=null,但 "error" in data = True)
- 熔断在第 4 次计数时触发(阈值为 5,第 5 次调用被阻断)
Resolve merge conflicts in run_agent.py (4 conflicts): - Accept upstream changes for compression config, aux context, stream diagnostics, and provider profile path. - Circuit breaker code remains intact.
|
After extensive real-world usage, I've found a number of issues that need addressing, and the feature's use case is quite niche. I don't plan to maintain it going forward. Closing this PR. |
Added Chinese description of fork features: - Circuit breaker (NousResearch#16749) - CLI Chinese localization (NousResearch#15282) - Message embedding (NousResearch#18059) - Emergency compression (NousResearch#18607)
Summary
Two-layer circuit breaker (disabled by default). Enable via config.yaml:
Changes