Skip to content

Fix half-open circuit breaker retry handling#120

Merged
SantiagoDePolonia merged 1 commit intomainfrom
fix/circuit-breaker
Mar 6, 2026
Merged

Fix half-open circuit breaker retry handling#120
SantiagoDePolonia merged 1 commit intomainfrom
fix/circuit-breaker

Conversation

@SantiagoDePolonia
Copy link
Copy Markdown
Contributor

@SantiagoDePolonia SantiagoDePolonia commented Mar 6, 2026

Summary by CodeRabbit

  • Bug Fixes
    • Request retries now work more reliably during transient service failures, improving overall system resilience.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 6, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: f31fe040-cd20-49f2-b7df-ecff9505dbcb

📥 Commits

Reviewing files that changed from the base of the PR and between 46e3f34 and 4b755fc.

📒 Files selected for processing (2)
  • internal/llmclient/client.go
  • internal/llmclient/client_test.go

📝 Walkthrough

Walkthrough

Refactors circuit breaker from simple Allow check to an acquire-based approach returning (bool, bool) indicating permission and half-open probe status. Implements probe short-circuiting in DoRaw and retry logic to prevent concurrent probes during open states. Adds comprehensive tests for retry-circuit-breaker interactions and half-open probe behavior.

Changes

Cohort / File(s) Summary
Circuit Breaker Acquire Mechanism
internal/llmclient/client.go
Replaces direct circuit breaker Allow check with acquire() method returning (allowed, isHalfOpenProbe). Implements half-open probe signaling and short-circuit logic in DoRaw to prevent multiple concurrent probes. Integrates probe handling into retry logic to end early with last error when in half-open state. Updates internal state transitions to support single half-open probe tracking.
Retry and Circuit Breaker Tests
internal/llmclient/client_test.go
Adds TestClient_Do_RetriesContinueAfterCircuitTrips validating retry continuation through circuit breaker transitions and successful recovery. Adds TestCircuitBreaker_HalfOpenProbeDoesNotRetry verifying that half-open probe prevents concurrent retries, returns GatewayError on immediate retry, and maintains open state. Tests configure retry backoff, jitter, and circuit breaker thresholds with assertions on upstream attempt counts and error types.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant CB as CircuitBreaker
    participant Retry
    participant Upstream

    Client->>CB: acquire()
    alt Circuit Open (Half-Open Probe)
        CB-->>Client: (true, true)<br/>allowed=true, isProbe=true
        Client->>Upstream: send request
        Upstream-->>Client: error/status
        Client->>Retry: check if retryable
        alt Retryable + In Probe
            Retry->>Client: invoke end-hook<br/>return last error
        end
    else Circuit Closed
        CB-->>Client: (true, false)
        Client->>Upstream: send request
        Upstream-->>Client: response
        alt Failure + Retryable
            Client->>Retry: attempt retry
            Retry->>CB: acquire()
            CB-->>Retry: (allow, probe)
        end
    else Circuit Open (Blocked)
        CB-->>Client: (false, _)
        Client->>Client: invoke end-hook<br/>return last error
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 A circuit breaker stands so tall,
With half-open probes that heed the call,
No retries when the probe takes flight,
State transitions done just right,
Control flow dancing left and right! 🔌

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title 'Fix half-open circuit breaker retry handling' directly addresses the main change in the changeset, which involves fixing retry behavior when a circuit breaker is in the half-open state.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix/circuit-breaker

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@SantiagoDePolonia
Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 6, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@SantiagoDePolonia SantiagoDePolonia merged commit d5ddd58 into main Mar 6, 2026
13 checks passed
@SantiagoDePolonia SantiagoDePolonia deleted the fix/circuit-breaker branch March 22, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant