fix(telegram): retry network errors wrapped in grammY HttpError#51895
Conversation
Greptile SummaryThis PR fixes a real, well-described production bug: grammY wraps transient network failures (ECONNRESET, ETIMEDOUT) inside an Key observations:
Confidence Score: 4/5
Prompt To Fix All With AIThis is a comment left during a code review.
Path: src/infra/errors.ts
Line: 73-74
Comment:
**Cycle detection misses the root error itself**
`err` is never added to `seen`, so a cycle that includes `err` is only broken after `err.message` has been appended a second time. For the test case `a.cause = b; b.cause = a`, the loop produces `"error A | error B | error A"` — `err.message` appears twice. The existing test uses `.toContain` and therefore doesn't catch the duplicate.
Initialize `seen` with `err` to close that gap:
```suggestion
let cause: unknown = err.cause;
const seen = new Set<unknown>([err]);
```
How can I resolve this? If you propose a fix, please make it concise.Last reviewed commit: "fix(telegram): trave..." |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 882980d7f1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
7c99269 to
ae44696
Compare
|
@obviyus could you kindly review this when you have a moment? |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ae4469687a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 68e43b0878
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e0972e67af
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…match grammY HttpError grammY wraps network failures in HttpError with message 'Network request for ... failed!' and the original error in .cause. formatErrorMessage only checked err.message, so shouldRetry never fired for the most common transient failure class. Changes: - formatErrorMessage now traverses .cause chain, appending nested error messages (with cycle protection) - Added 'Network request' to TELEGRAM_RETRY_RE as belt-and-suspenders - Added tests for .cause traversal, circular references, and grammY HttpError retry behavior Fixes openclaw#51525
6694b44 to
068a177
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 068a177fa2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
obviyus
left a comment
There was a problem hiding this comment.
Reviewed latest changes; landing now.
|
Landed on main. Thanks @chinar-amrutkar. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cd03366d4d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| export function isTelegramRateLimitError(err: unknown): boolean { | ||
| return ( | ||
| hasTelegramErrorCode(err, (code) => code === 429) || | ||
| (hasTelegramRetryAfter(err) && /(?:^|\b)429\b|too many requests/i.test(formatErrorMessage(err))) |
There was a problem hiding this comment.
Propagate wrapped
retry_after delay into strict send retries
isTelegramRateLimitError now treats nested wrappers as retriable (via hasTelegramRetryAfter), but the strict send path in createTelegramNonIdempotentRequestWithDiag still relies on createTelegramRetryRunner's shallow getTelegramRetryAfterMs parser, which only reads top-level parameters/response/error.parameters. For wrapped shapes like HttpError -> error.response.parameters.retry_after, this change will retry immediately using default backoff instead of Telegram’s required delay, so attempts can be exhausted under throttling even though a valid retry_after was present.
Useful? React with 👍 / 👎.
@chinar-amrutkar) * fix(telegram): traverse error .cause chain in formatErrorMessage and match grammY HttpError grammY wraps network failures in HttpError with message 'Network request for ... failed!' and the original error in .cause. formatErrorMessage only checked err.message, so shouldRetry never fired for the most common transient failure class. Changes: - formatErrorMessage now traverses .cause chain, appending nested error messages (with cycle protection) - Added 'Network request' to TELEGRAM_RETRY_RE as belt-and-suspenders - Added tests for .cause traversal, circular references, and grammY HttpError retry behavior Fixes openclaw#51525 * style: fix oxfmt formatting in retry-policy.ts * fix: add braces to satisfy oxlint requirement * fix(telegram): keep send retries strict * test(telegram): cover wrapped retry paths * fix(telegram): retry rate-limited sends safely * fix: retry safe wrapped Telegram send failures (openclaw#51895) (thanks @chinar-amrutkar) * fix: preserve wrapped Telegram rate-limit retries (openclaw#51895) (thanks @chinar-amrutkar) --------- Co-authored-by: chinar-amrutkar <chinar-amrutkar@users.noreply.github.com> Co-authored-by: Ayaan Zaidi <hi@obviy.us>
@chinar-amrutkar) * fix(telegram): traverse error .cause chain in formatErrorMessage and match grammY HttpError grammY wraps network failures in HttpError with message 'Network request for ... failed!' and the original error in .cause. formatErrorMessage only checked err.message, so shouldRetry never fired for the most common transient failure class. Changes: - formatErrorMessage now traverses .cause chain, appending nested error messages (with cycle protection) - Added 'Network request' to TELEGRAM_RETRY_RE as belt-and-suspenders - Added tests for .cause traversal, circular references, and grammY HttpError retry behavior Fixes openclaw#51525 * style: fix oxfmt formatting in retry-policy.ts * fix: add braces to satisfy oxlint requirement * fix(telegram): keep send retries strict * test(telegram): cover wrapped retry paths * fix(telegram): retry rate-limited sends safely * fix: retry safe wrapped Telegram send failures (openclaw#51895) (thanks @chinar-amrutkar) * fix: preserve wrapped Telegram rate-limit retries (openclaw#51895) (thanks @chinar-amrutkar) --------- Co-authored-by: chinar-amrutkar <chinar-amrutkar@users.noreply.github.com> Co-authored-by: Ayaan Zaidi <hi@obviy.us>
@chinar-amrutkar) * fix(telegram): traverse error .cause chain in formatErrorMessage and match grammY HttpError grammY wraps network failures in HttpError with message 'Network request for ... failed!' and the original error in .cause. formatErrorMessage only checked err.message, so shouldRetry never fired for the most common transient failure class. Changes: - formatErrorMessage now traverses .cause chain, appending nested error messages (with cycle protection) - Added 'Network request' to TELEGRAM_RETRY_RE as belt-and-suspenders - Added tests for .cause traversal, circular references, and grammY HttpError retry behavior Fixes openclaw#51525 * style: fix oxfmt formatting in retry-policy.ts * fix: add braces to satisfy oxlint requirement * fix(telegram): keep send retries strict * test(telegram): cover wrapped retry paths * fix(telegram): retry rate-limited sends safely * fix: retry safe wrapped Telegram send failures (openclaw#51895) (thanks @chinar-amrutkar) * fix: preserve wrapped Telegram rate-limit retries (openclaw#51895) (thanks @chinar-amrutkar) --------- Co-authored-by: chinar-amrutkar <chinar-amrutkar@users.noreply.github.com> Co-authored-by: Ayaan Zaidi <hi@obviy.us>
@chinar-amrutkar) * fix(telegram): traverse error .cause chain in formatErrorMessage and match grammY HttpError grammY wraps network failures in HttpError with message 'Network request for ... failed!' and the original error in .cause. formatErrorMessage only checked err.message, so shouldRetry never fired for the most common transient failure class. Changes: - formatErrorMessage now traverses .cause chain, appending nested error messages (with cycle protection) - Added 'Network request' to TELEGRAM_RETRY_RE as belt-and-suspenders - Added tests for .cause traversal, circular references, and grammY HttpError retry behavior Fixes openclaw#51525 * style: fix oxfmt formatting in retry-policy.ts * fix: add braces to satisfy oxlint requirement * fix(telegram): keep send retries strict * test(telegram): cover wrapped retry paths * fix(telegram): retry rate-limited sends safely * fix: retry safe wrapped Telegram send failures (openclaw#51895) (thanks @chinar-amrutkar) * fix: preserve wrapped Telegram rate-limit retries (openclaw#51895) (thanks @chinar-amrutkar) --------- Co-authored-by: chinar-amrutkar <chinar-amrutkar@users.noreply.github.com> Co-authored-by: Ayaan Zaidi <hi@obviy.us>
Summary
Network request for 'sendMessage' failed!and the original error in.cause.formatErrorMessageonly checkederr.message, soshouldRetrynever fired — ~40 silent failures/day on high-latency paths.formatErrorMessagetraverses.causechain (with cycle protection). AddedNetwork requesttoTELEGRAM_RETRY_RE.Change Type
Scope
Linked Issue/PR
User-visible / Behavior Changes
Telegram message sends now retry on transient network errors that grammY wraps in HttpError. Previously silently dropped.
Security Impact
Repro + Verification
Steps
throw Object.assign(new Error("Network request for 'sendMessage' failed!"), { cause: new Error("ECONNRESET") })Expected
shouldRetry returns true, retry executes.
Actual (before fix)
shouldRetry returns false, no retry.
Evidence
Human Verification
Compatibility / Migration
Failure Recovery
git revert HEADon branch