β¨ feat(model-runtime): split ProviderBizError into finer codes + reclassify catch-all at write time#15286
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
π‘ Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 12d7af9ad4
βΉοΈ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with π.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // rate-limits, network drops, quota, etc. Specific codes pass through. | ||
| const refined = refineErrorCode({ | ||
| errorType: String(formatted.type), | ||
| httpStatus: extractHttpStatus(formatted.body), |
There was a problem hiding this comment.
Use top-level HTTP status during refinement
When the idempotent path receives an already-normalized ChatMessageError with type: ProviderBizError, a generic/opaque message, and a top-level httpStatus, this passes undefined to refineErrorCode unless the same status is duplicated under body.status. In that scenario the new 402/429/5xx fallback never runs and the error remains ProviderBizError; use formatted.httpStatus as a fallback to the body extraction so normalized errors can still be refined.
Useful? React with πΒ / π.
Codecov Reportβ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## canary #15286 +/- ##
========================================
Coverage 71.12% 71.12%
========================================
Files 3174 3175 +1
Lines 317373 317484 +111
Branches 33708 33729 +21
========================================
+ Hits 225718 225823 +105
- Misses 91485 91489 +4
- Partials 170 172 +2
Flags with carried forward coverage won't be shown. Click here to find out more.
π New features to boost your workflow:
|
β¦assify catch-all at write time Add UpstreamGatewayError (E8010), UpstreamMalformedResponse (E8011), and UpstreamHttpError (E8012), migrating the matching patterns out of the ProviderBizError catch-all. Add a refineErrorCode() step (message-pattern match + HTTP-status fallback) wired into formatErrorForState so generic ProviderBizError is reclassified into the correct existing code (rate-limit / quota / network / service-unavailable / model-not-found) instead of collapsing into one opaque 8xxx bucket. Production sampling showed ~72% of ProviderBizError actually belongs to existing codes and only ~5% is a true residual.
β¦ckets Add an `isFallback` boolean to ErrorCodeSpec / ChatMessageError, set on the catch-all codes (ProviderBizError, UpstreamHttpError, AgentRuntimeError, DatabasePersistError). It flows onto agent_operations.error via the write-path enrichment so monitoring can track how much volume still lands in fallback buckets β the signal for where finer codes are still worth carving out.
β¦ mocks formatErrorForState now imports refineErrorCode, so the partial module mocks in AgentRuntimeService / RuntimeExecutors must expose it or vitest throws on access.
β¦ter canary 8010 collision canary claimed 8010 for ProviderContentPolicyViolation, so the Upstream* codes shifted to 8011/8012/8013 during rebase; update the refinement test assertion.
a1d61e4 to
56092d6
Compare
# π LobeHub Release (20260528) **Release Date:** May 28, 2026 **Since v2.2.0:** 220 merged PRs Β· 15 contributors > This cycle brings heterogeneous "platform agents" you can dispatch to local or remote devices, a rebuilt onboarding flow, document-centric chat, and a unified model-runtime error model β with new DeepSeek V4 and Gemini 3.5 Flash support along the way. --- ## β¨ Highlights - **More Hetero Agents (OpenClaw / Hermes)** β Create heterogeneous agents and dispatch them to local or remote devices through the device gateway, with an execution-target switcher in the composer and persistent CLI sessions. (#15065, #15179, #15022) - **iMessage on Desktop** β New iMessage setup and bridge on desktop, plus bot attachments across every platform. (#15228, #15227, #15029) - **Skills in the Composer** β Drag skill chips into chat, trigger installed skills from the slash menu mid-line, and surface project-level skills in the homogeneous agent runtime. (#15095, #15061, #15110) - **New Models** β DeepSeek V4 Flash/Pro and Gemini 3.5 Flash across providers, with thinking params for structured output and chat cost estimates. (#15031, #15001, #15051, #14876) - **Agent Runtime Observability** β OpenTelemetry GenAI semantic conventions plus per-call generation tracing. (#15123, #15124) --- ## π€ Agents & Heterogeneous Runtime - **Platform agent creation** β OpenClaw/Hermes creation UI, device guard, and remote dispatch backend. (#15065) - **Execution-target switcher** β Pick local vs remote execution directly in the composer; device-selection UX with actionable guidance. (#15179, #15111) - **CLI hetero dispatch** β OpenClaw/Hermes dispatch with persistent sessions and a notify protocol. (#15022) - **Gateway snapshot as source of truth** β Consume the gateway `uiMessages` snapshot at step boundaries to keep chat state consistent. (#15153, #15152) - **Client sub-agent as a normal tool call** β Simplifies the sub-agent execution path. (#15281) - **Hermes agent chain** β Implements the Hermes agent chain logic. (#15189) - **Device registry** β TRPC endpoints to register, list, update, and remove devices. (#15299) - **Desktop device routing** β Route gateway agent runs through `lh hetero exec`; restore `userId` in gateway dispatch and gate local-system by execution target. (#15132, #15232) - **Agent signals** β Anchor agent-signal receipts to messages and isolate memory-agent messages into a child thread. (#14969, #14921) --- ## π Onboarding - **Simplified first screen** β Defer topic creation to first send. (#15090) - **Market Agent Picker** β Added as a classic onboarding step, with template prefetch. (#14980, #15041) - **Welcome guidance** β Show agent welcome guidance on first run. (#15098) - **Mobile** β Adapt agent onboarding UI and restore Classic-step padding on mobile. (#15019, #15032) - **Discovery** β Streamline discovery to a single profession question. (#14987) - **Analytics** β Track onboarding step events and create-agent modal source. (#15133, #15028) --- ## π Documents, Pages & Knowledge - **Thread chat in preview** β Embed thread chat in the document preview portal. (#15216) - **Non-markdown rendering** β Render non-markdown docs as a read-only highlight. (#15272) - **Multi-select** β Multi-select delete in the document tree. (#15125) - **Page-agent streaming** β Preview `initPage` streaming arguments. (#15039) - **Per-agent topics** β Per-agent topic management page. (#15207) - **Server-side category** β Derive document category server-side and drop frontend predicates. (#15076) --- ## π§© Skills & Tools - **Drag skill chips** β Drag skills into chat input and register agent-document skills. (#15095) - **Slash menu** β Installed skills appear in the slash menu with a mid-line trigger. (#15061) - **Project skills** β Recognize project-level skills in the homogeneous agent runtime and surface them regardless of active device. (#15110, #15177) - **VFS archiving** β Archive oversized tool results to VFS instead of truncating. (#15074) - **@localfile mentions** β Drag folders into chat input as `@localFile` mentions on desktop. (#15071) --- ## π§ Model Runtime & Providers - **Error spec registry** β Unify error codes into a spec + pattern registry, split `ProviderBizError` into finer codes, classify Cloud-only codes via a tier digit, and add `DatabasePersistError`. (#15262, #15286, #15278, #15279) - **New models** β DeepSeek V4 Flash/Pro (opencode-go) and Gemini 3.5 Flash; DeepSeek V4 Pro on SiliconCloud. (#15031, #15001, #15017, #15267) - **Structured output** β Thinking params for structured output, Bedrock structured generation, and DeepSeek `generateObject` tool choice. (#15051, #15174, #15054) - **Cost** β Chat cost estimate support; preserve usage cost in custom streams. (#14876, #15218) --- ## π¬ Chat & User Experience - **Follow-up chips** β Extend follow-up chip suggestions to general chat with scene-specific model config. (#15101, #14797) - **Input drafts** β Persist unsent input drafts across tab switches and prevent repeated draft restore. (#14992, #15024) - **Command menu** β Order topic/message search by recency and promote inline type filters. (#15094, #14986) - **Zoom HUD** β Show a zoom-level HUD on Cmd +/β and Cmd 0. (#15294) - **Copy** β Unescape markdown escapes when copying user messages. (#15253) --- ## π₯οΈ Desktop - **App Nap fix** β Prevent App Nap from dropping the gateway WebSocket during display sleep. (#14994) - **File preview** β Preview `.cjs`/`.mjs`/no-extension files instead of binary fallback and expand `~` when opening local files. (#15168, #15284) - **Cross-platform settings** β Open settings via main-window navigation on Windows/Linux and restore the route after an update restart. (#15036, #14922) - **Token refresh** β Prevent frequent logout from token-refresh retries. (#14928) --- ## π Observability - **OTel GenAI** β Instrument Agent Runtime with OpenTelemetry GenAI semantic conventions. (#15123) - **Generation tracing** β Per-call `llm_generation_tracing` with a pre-allocated tracingId and recordFeedback router. (#15124, #15146) - **Error classification** β Persist `ERROR_CODE_SPECS` classification on operation errors. (#15273) --- ## ποΈ Database Migrations - **Batch migrations** β Topic usage stats, push tokens, `tasks.editor_data`, and document shares. (#15280) - **Tracing & eval tables** β Add `llm_generation_tracing` and agent eval experiment tables. (#15126) > Self-hosted operators should run the database migration (`pnpm db:migrate`, or restart with auto-migrate enabled) after upgrading. The changes are additive and backwards-compatible. --- ## π Security & Reliability - **Security:** Remove the `getPlaintextCred` tool to prevent plaintext credential exposure. (#14998) - **Security:** Prompt account selection for Google OAuth and add `prompt=consent` to the OIDC authorization URL to fix missing refresh tokens. (#15234, #15010) - **Reliability:** Preserve streamed content across a mid-stream cancel. (#15173) - **Reliability:** Bound the Redis command timeout and configure the Anthropic client timeout. (#15091, #15042) - **Reliability:** Prevent infinite recursion in the assistant chain. (#15288) --- ## π₯ Contributors Huge thanks to **15 contributors** who shipped **220 merged PRs** this cycle. @AnotiaWang Β· @sxjeru Β· @algojogacor Β· @hardy-one Β· @arvinxx Β· @Innei Β· @tjx666 Β· @lijian Β· @AmAzing129 Β· @rdmclin2 Β· @neko Β· @cy948 Β· @CanisMinor Β· @sudongyuer Β· @rivertwilight Plus @lobehubbot and renovate[bot] for maintenance. --- **Full Changelog**: v2.2.0...release/weekly-20260528
π» Change Type
π Related Issue
Related to LOBE-9683 (split the
ProviderBizErrorcatch-all). Follow-up to the unified error registry from #15262.π Description of Change
ProviderBizError(E8002) was a catch-all absorbing everything from gateway HTML to upstream Python crashes to bare HTTP statuses. Production sampling ofagent_operations.error(14d) showed that ~72% ofProviderBizErroractually belongs to existing codes (rate-limit / quota / network / service-unavailable / model-not-found), only ~22% maps to genuinely new "upstream" buckets, and only ~5% is a true residual.Root cause: the write path (
formatErrorForState) only enriched viagetErrorCodeSpec(error.type)and never ran the message-pattern registry or looked at the HTTP status, so every generic upstream failure collapsed into one opaque 8xxx bucket.This PR does two things:
1. Add three finer provider codes (additive, append-only numericIds):
UpstreamGatewayError(E8010) β proxy/gateway-layer failure (openresty, litellm, HTML 5xx, Cloudflare 525).retryable.UpstreamMalformedResponse(E8011) β provider returned an unparseable payload (Go re-marshal failure, bad tool-call JSON, upstreamTypeError). Not retryable.UpstreamHttpError(E8012) β bare upstream HTTP error with no context (e.g."400 status code"). The residual provider bucket.The matching patterns are migrated out of the
ProviderBizErrorsection inpatterns.ts.2. Reclassify the catch-all at write time via a new
refineErrorCode()step (model-runtime/src/errors/refine.ts), wired intoformatErrorForState. For a genericProviderBizErrorit:matchErrorPatternover the message (covers the rich cases + the migratedUpstream*patterns), then429βRateLimitExceeded,402βInsufficientQuota,5xxβProviderServiceUnavailable, other4xxβUpstreamHttpError).Specific codes assigned by an adapter are never touched β only the
ProviderBizErrorcatch-all is refined. Also added two high-volume patterns the data surfaced:Connection error.βProviderNetworkErrorandWeekly usage limit reachedβInsufficientQuota.ProviderBizError(retryable=false) now reclassify into retryable codes (RateLimitExceeded / ProviderServiceUnavailable / ProviderNetworkError). This means a chunk of upstream failures that weren't retried will now be retried β intended, but worth a look for retry-cost impact.Notes / follow-ups:
modelRuntimekeys were added (no translation creds locally). Other languages fall back to en-US until a maintainer runspnpm i18n.UpstreamHttpErrorcurrently holds bare 400/413/422; 400/422 are candidates for a futurerequest-category split (tracked separately).π§ͺ How to Test
Tested locally
Added/updated tests
packages/model-runtime/src/errors/refine.test.ts(new) β message-pattern + status-fallback cases.src/server/modules/AgentRuntime/formatErrorForState.test.tsβ reclassification + residual-preservation cases.Existing
errors(66) +llmErrorClassification(31) suites pass;type-checkclean.πΈ Screenshots / Videos
N/A β no UI changes.