-
-
Notifications
You must be signed in to change notification settings - Fork 56.7k
Description
Problem
When the upstream API returns persistent errors (e.g., 429 rate limit, 402 payment required, credit exhaustion), the gateway retries correctly but then sends empty messages to the user rather than explaining what happened.
From the user's perspective, the assistant simply stops responding — or worse, sends blank/ghost messages. There's no indication that credits are exhausted or rate limits have been hit.
Observed Behavior
Session history shows:
- User sends message
- API returns
429 rate_limit_error: "This request would exceed your account's rate limit" - Gateway retries 4 times (all fail with same error)
- Gateway sends response with
content: [],stopReason: "error" - User receives empty/blank message
- User has no idea what's happening
Expected Behavior
When retries are exhausted on a 429/402/similar error, the gateway should send a helpful message to the user:
⚠️ API rate limit reached. Your account may need more credits. Check console.anthropic.com or try again later.
This should be configurable (e.g., errorMessages.rateLimitExhausted) but have sensible defaults.
Impact
- Users think the assistant has crashed or is ignoring them
- No actionable information provided
- Poor UX during a recoverable situation
Suggested Fix
In the response handling path, detect when:
stopReason === 'error'- Error is a rate limit (429) or payment issue (402)
- Retries exhausted
Then inject a user-facing error message rather than sending empty content.