β¨ feat(agent-runtime): persist ERROR_CODE_SPECS classification on operation errors#15273
Conversation
β¦ration errors Look up the runtime error's spec in `ERROR_CODE_SPECS` at the single catch chokepoint and merge `attribution` / `category` / `severity` / `httpStatus` / `retryable` / `countAsFailure` / `numericId` onto the normalized `ChatMessageError`. The enriched object flows through to all three downstream sinks β `agent_operations.error` JSONB, S3 trace snapshot, and the agent-gateway WS push β without each consumer having to re-run pattern matching. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
ERROR_CODE_SPECS classification on operation errors
There was a problem hiding this comment.
π‘ Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e2615f07f5
βΉοΈ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with π.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codecov Reportβ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## canary #15273 +/- ##
========================================
Coverage 70.95% 70.96%
========================================
Files 3160 3161 +1
Lines 317287 317334 +47
Branches 34491 33581 -910
========================================
+ Hits 225127 225181 +54
+ Misses 91991 91984 -7
Partials 169 169
Flags with carried forward coverage won't be shown. Click here to find out more.
π New features to boost your workflow:
|
Model-runtime failures caught inside `runtime.step()` resolve normally with `newState.status = 'error'` instead of throwing, so the prior commit's outer `executeStep` catch never sees common provider errors like `InvalidProviderAPIKey` / `InsufficientQuota`. Those were reaching `agent_operations.error` JSONB and the success-path trace snapshot raw β without `attribution` / `category` / `severity` / β¦ Run `formatErrorForState` on `stepResult.newState.error` immediately after `runtime.step()` returns, before the state is saved to Redis, hooks are dispatched, or the trace is finalized. Made the helper idempotent (recognizes already-normalized `ChatMessageError` shape) so a second pass through the outer catch can't collapse it back to `AgentRuntimeError`. Success-path `traceRecorder.finalize` now forwards the classification fields too. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
Good catch β fixed in 56118e0. The inner Now:
Tests: 69 pass (61 existing AgentRuntimeService + 8 in |
# π LobeHub Release (20260528) **Release Date:** May 28, 2026 **Since v2.2.0:** 220 merged PRs Β· 15 contributors > This cycle brings heterogeneous "platform agents" you can dispatch to local or remote devices, a rebuilt onboarding flow, document-centric chat, and a unified model-runtime error model β with new DeepSeek V4 and Gemini 3.5 Flash support along the way. --- ## β¨ Highlights - **More Hetero Agents (OpenClaw / Hermes)** β Create heterogeneous agents and dispatch them to local or remote devices through the device gateway, with an execution-target switcher in the composer and persistent CLI sessions. (#15065, #15179, #15022) - **iMessage on Desktop** β New iMessage setup and bridge on desktop, plus bot attachments across every platform. (#15228, #15227, #15029) - **Skills in the Composer** β Drag skill chips into chat, trigger installed skills from the slash menu mid-line, and surface project-level skills in the homogeneous agent runtime. (#15095, #15061, #15110) - **New Models** β DeepSeek V4 Flash/Pro and Gemini 3.5 Flash across providers, with thinking params for structured output and chat cost estimates. (#15031, #15001, #15051, #14876) - **Agent Runtime Observability** β OpenTelemetry GenAI semantic conventions plus per-call generation tracing. (#15123, #15124) --- ## π€ Agents & Heterogeneous Runtime - **Platform agent creation** β OpenClaw/Hermes creation UI, device guard, and remote dispatch backend. (#15065) - **Execution-target switcher** β Pick local vs remote execution directly in the composer; device-selection UX with actionable guidance. (#15179, #15111) - **CLI hetero dispatch** β OpenClaw/Hermes dispatch with persistent sessions and a notify protocol. (#15022) - **Gateway snapshot as source of truth** β Consume the gateway `uiMessages` snapshot at step boundaries to keep chat state consistent. (#15153, #15152) - **Client sub-agent as a normal tool call** β Simplifies the sub-agent execution path. (#15281) - **Hermes agent chain** β Implements the Hermes agent chain logic. (#15189) - **Device registry** β TRPC endpoints to register, list, update, and remove devices. (#15299) - **Desktop device routing** β Route gateway agent runs through `lh hetero exec`; restore `userId` in gateway dispatch and gate local-system by execution target. (#15132, #15232) - **Agent signals** β Anchor agent-signal receipts to messages and isolate memory-agent messages into a child thread. (#14969, #14921) --- ## π Onboarding - **Simplified first screen** β Defer topic creation to first send. (#15090) - **Market Agent Picker** β Added as a classic onboarding step, with template prefetch. (#14980, #15041) - **Welcome guidance** β Show agent welcome guidance on first run. (#15098) - **Mobile** β Adapt agent onboarding UI and restore Classic-step padding on mobile. (#15019, #15032) - **Discovery** β Streamline discovery to a single profession question. (#14987) - **Analytics** β Track onboarding step events and create-agent modal source. (#15133, #15028) --- ## π Documents, Pages & Knowledge - **Thread chat in preview** β Embed thread chat in the document preview portal. (#15216) - **Non-markdown rendering** β Render non-markdown docs as a read-only highlight. (#15272) - **Multi-select** β Multi-select delete in the document tree. (#15125) - **Page-agent streaming** β Preview `initPage` streaming arguments. (#15039) - **Per-agent topics** β Per-agent topic management page. (#15207) - **Server-side category** β Derive document category server-side and drop frontend predicates. (#15076) --- ## π§© Skills & Tools - **Drag skill chips** β Drag skills into chat input and register agent-document skills. (#15095) - **Slash menu** β Installed skills appear in the slash menu with a mid-line trigger. (#15061) - **Project skills** β Recognize project-level skills in the homogeneous agent runtime and surface them regardless of active device. (#15110, #15177) - **VFS archiving** β Archive oversized tool results to VFS instead of truncating. (#15074) - **@localfile mentions** β Drag folders into chat input as `@localFile` mentions on desktop. (#15071) --- ## π§ Model Runtime & Providers - **Error spec registry** β Unify error codes into a spec + pattern registry, split `ProviderBizError` into finer codes, classify Cloud-only codes via a tier digit, and add `DatabasePersistError`. (#15262, #15286, #15278, #15279) - **New models** β DeepSeek V4 Flash/Pro (opencode-go) and Gemini 3.5 Flash; DeepSeek V4 Pro on SiliconCloud. (#15031, #15001, #15017, #15267) - **Structured output** β Thinking params for structured output, Bedrock structured generation, and DeepSeek `generateObject` tool choice. (#15051, #15174, #15054) - **Cost** β Chat cost estimate support; preserve usage cost in custom streams. (#14876, #15218) --- ## π¬ Chat & User Experience - **Follow-up chips** β Extend follow-up chip suggestions to general chat with scene-specific model config. (#15101, #14797) - **Input drafts** β Persist unsent input drafts across tab switches and prevent repeated draft restore. (#14992, #15024) - **Command menu** β Order topic/message search by recency and promote inline type filters. (#15094, #14986) - **Zoom HUD** β Show a zoom-level HUD on Cmd +/β and Cmd 0. (#15294) - **Copy** β Unescape markdown escapes when copying user messages. (#15253) --- ## π₯οΈ Desktop - **App Nap fix** β Prevent App Nap from dropping the gateway WebSocket during display sleep. (#14994) - **File preview** β Preview `.cjs`/`.mjs`/no-extension files instead of binary fallback and expand `~` when opening local files. (#15168, #15284) - **Cross-platform settings** β Open settings via main-window navigation on Windows/Linux and restore the route after an update restart. (#15036, #14922) - **Token refresh** β Prevent frequent logout from token-refresh retries. (#14928) --- ## π Observability - **OTel GenAI** β Instrument Agent Runtime with OpenTelemetry GenAI semantic conventions. (#15123) - **Generation tracing** β Per-call `llm_generation_tracing` with a pre-allocated tracingId and recordFeedback router. (#15124, #15146) - **Error classification** β Persist `ERROR_CODE_SPECS` classification on operation errors. (#15273) --- ## ποΈ Database Migrations - **Batch migrations** β Topic usage stats, push tokens, `tasks.editor_data`, and document shares. (#15280) - **Tracing & eval tables** β Add `llm_generation_tracing` and agent eval experiment tables. (#15126) > Self-hosted operators should run the database migration (`pnpm db:migrate`, or restart with auto-migrate enabled) after upgrading. The changes are additive and backwards-compatible. --- ## π Security & Reliability - **Security:** Remove the `getPlaintextCred` tool to prevent plaintext credential exposure. (#14998) - **Security:** Prompt account selection for Google OAuth and add `prompt=consent` to the OIDC authorization URL to fix missing refresh tokens. (#15234, #15010) - **Reliability:** Preserve streamed content across a mid-stream cancel. (#15173) - **Reliability:** Bound the Redis command timeout and configure the Anthropic client timeout. (#15091, #15042) - **Reliability:** Prevent infinite recursion in the assistant chain. (#15288) --- ## π₯ Contributors Huge thanks to **15 contributors** who shipped **220 merged PRs** this cycle. @AnotiaWang Β· @sxjeru Β· @algojogacor Β· @hardy-one Β· @arvinxx Β· @Innei Β· @tjx666 Β· @lijian Β· @AmAzing129 Β· @rdmclin2 Β· @neko Β· @cy948 Β· @CanisMinor Β· @sudongyuer Β· @rivertwilight Plus @lobehubbot and renovate[bot] for maintenance. --- **Full Changelog**: v2.2.0...release/weekly-20260528
π» Change Type
π Related Issue
Follow-up to #15262, which introduced
ERROR_CODE_SPECSbut only consumed it internally (HTTP status mapping + retry decisions). This PR makes the classification a first-class field on every persisted operation error.π Description of Change
When an agent operation fails, the catch block in
AgentRuntimeService.executeStepnormalizes the thrown value into aChatMessageErrorand writes it to three sinks:agent_operations.errorJSONB (viaagentOperationModel.recordCompletion)OperationTraceRecorder.finalize)GatewayStreamNotifier.publishAgentRuntimeEnd, which forwardsfinalState.errorverbatim)Today the persisted shape is just
{ message, type, body }. Downstream consumers (DC observability dashboards, agent-gateway error triage, support tooling) that want to know whether an error is user-attributable, what category it belongs to, and whether it counts as a failure have to re-derive that themselves β typically by maintaining a parallel mirror ofERROR_CODE_SPECS.This PR does the lookup once, at the chokepoint:
packages/types/src/message/common/base.tsβChatMessageErrorgains optionalattribution/category/severity/httpStatus/retryable/countAsFailure/numericIdfields plus exportedChatMessageErrorAttributionandChatMessageErrorSeverityliteral types. Zod schema mirrored.src/server/modules/AgentRuntime/formatErrorForState.ts(new) β extracted fromAgentRuntimeService.tsand wraps the existing normalization with anenrichWithSpecstep that mergesgetErrorCodeSpec(formatted.type)onto the result. Codes outside the spec table (e.g. rawInternalServerError) pass through unchanged.src/server/services/agentRuntime/AgentRuntimeService.tsβ imports the extracted helper; the error finalize call totraceRecorder.finalizeforwards the classification fields so the S3 snapshot carries them too.src/server/services/agentRuntime/OperationTraceRecorder.tsβFinalizeParams.errorwidened to accept the classification fields.The agent-gateway notifier needs no changes β it forwards
finalState.erroras-is, so the new fields ride along automatically.π§ͺ How to Test
```bash
bun run type-check # passes
bunx vitest run src/server/modules/AgentRuntime/formatErrorForState.test.ts # 7 new cases
bunx vitest run src/server/services/agentRuntime/AgentRuntimeService.test.ts # 60 existing pass
bunx vitest run src/server/modules/AgentRuntime/tests/formatErrorEventData.test.ts \
src/server/modules/AgentRuntime/tests/llmErrorClassification.test.ts # 41 pass
```
New test cases cover:
ChatCompletionErrorPayload/ standardError/ unknown).InsufficientQuota(user / quota / non-retryable / not-a-failure) andRateLimitExceeded(provider / capacity / retryable).QuotaLimitReached β RateLimitExceededalias is resolved through to the canonical spec.πΈ Screenshots / Videos
N/A (no UI changes)
π Additional Information
Backward-compatible. All new fields on
ChatMessageErrorare optional; the Zod schema additions are also optional. Old persisted error rows inagent_operations.errorsimply lack the new keys.Forward path. Once this lands, DC's operation tracing can render attribution / severity / category badges directly from
agent_operations.errorwithout maintaining a spec mirror. Agent-gateway already importsisUserSideErrorfrom its own mirror; this PR doesn't touch that path.π€ Generated with Claude Code