β¨ feat(model-runtime): add DatabasePersistError code for failed DB queries#15279
Conversation
β¦eries Drizzle stringifies a failed query/transaction as `Failed query: <sql> params: <values>`. These are harness-side persistence failures, but they were landing in the operation dashboards as `unknown` β and worse, the embedded SQL/parameter text (model names, error_log rows, user messages) contains substrings that trip unrelated provider patterns, so naive message-matching misclassified them as CapabilityNotSupported / InsufficientQuota / ModelNotFound. - `agentRuntime.ts` β new `DatabasePersistError` code. - `specs.ts` β E7004 under the 7xxx Stream/Runtime (harness) bucket, `attribution: harness`, `countAsFailure: true`, httpStatus 500. - `patterns.ts` β `Failed query:` substring pattern placed **first** in the registry. matchErrorPattern is first-match-wins, so claiming it up front both classifies these correctly and stops the embedded blob from matching anything below. - `match.test.ts` β assert the wrap classifies as DatabasePersistError and that a blob embedding `InsufficientQuota` / `context length exceeded` still resolves to DatabasePersistError. - `modelRuntime.ts` β en-US `DatabasePersistError` copy (others auto-translate). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
π‘ Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0448114de8
βΉοΈ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with π.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| code: AgentRuntimeErrorType.DatabasePersistError, | ||
| match: sub('Failed query:'), |
There was a problem hiding this comment.
Route Drizzle Error instances through the new code
When a failed Drizzle query is thrown as a normal Error in the agent-runtime path, this new pattern is never consulted: formatErrorForState still maps all Error instances to ChatErrorType.InternalServerError before publishing/persisting the state (src/server/modules/AgentRuntime/formatErrorForState.ts, lines 78-83, used by AgentRuntimeService). In that scenario the message may be Failed query: ..., but downstream clients and operation records continue to see type 500 rather than DatabasePersistError, so the new code/spec does not actually surface for the DB failures it is meant to classify.
Useful? React with πΒ / π.
Codecov Reportβ
All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## canary #15279 +/- ##
=========================================
Coverage 71.02% 71.03%
=========================================
Files 3172 3172
Lines 317388 317454 +66
Branches 27919 34576 +6657
=========================================
+ Hits 225437 225503 +66
Misses 91782 91782
Partials 169 169
Flags with carried forward coverage won't be shown. Click here to find out more.
π New features to boost your workflow:
|
β¦edis aborts as provider-network `Command aborted due to connection close` is an ioredis error β the Redis/Upstash agent-state store dropping a queued command, not the LLM provider's network. It was mapped to `ProviderNetworkError`, which misattributed our own infra failures to upstream providers. - `agentRuntime.ts` β new `StateStorePersistError` (sibling of `DatabasePersistError`: DB layer vs state-store layer). - `specs.ts` β E7005 under 7xxx Stream/Runtime (harness), countAsFailure true. - `patterns.ts` β repoint `Command aborted due to connection close` to StateStorePersistError, and add the other Upstash state-store signatures (`max request size exceeded`, `database has been suspended`). - `match.test.ts` + `modelRuntime.ts` β test + en-US locale. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
β¦ash patterns Classify the harness-side crashes that were landing as `unknown`: - `ContextEnginePipelineError` (E7006, 7xxx Stream/Runtime, harness) β the context-engine pipeline processor crash, surfaced as "Processor [<name>] execution failed". The context-engine throws `PipelineError` (its `error.name`), so a CODE_ALIASES entry resolves `PipelineError` β ContextEnginePipelineError for stored / live records. - patterns: `Processor [` β ContextEnginePipelineError, placed before the generic JS-crash fallbacks so a processor crash with a nested TypeError is attributed to the pipeline, not the bare `Cannot read properties` rule. - patterns: bare V8 crashes (`is not a function`, `Cannot read properties of`, `Maximum call stack size exceeded`) β AgentRuntimeError, kept LAST so specific provider/harness patterns win first. - test + en-US locale. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
β¦user The broken conversation chain (`parent_id` no longer exists) is usually the user deleting the topic / parent message mid-operation β an expected race, not a harness bug. Flip attribution harness β user, countAsFailure true β false (so it drops out of failure metrics), severity error β warning. numericId 7003 / category `stream` stay put (append-only); attribution and category are orthogonal, so a stream-bucket code can be user-attributed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
β¦ntimeError A message of literally "[object Object]" means the harness stringified an error object instead of extracting its message β a harness serialization bug. Add it to the JS-crash fallbacks (last, lowest priority) so it resolves to AgentRuntimeError instead of staying unknown. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# π LobeHub Release (20260528) **Release Date:** May 28, 2026 **Since v2.2.0:** 220 merged PRs Β· 15 contributors > This cycle brings heterogeneous "platform agents" you can dispatch to local or remote devices, a rebuilt onboarding flow, document-centric chat, and a unified model-runtime error model β with new DeepSeek V4 and Gemini 3.5 Flash support along the way. --- ## β¨ Highlights - **More Hetero Agents (OpenClaw / Hermes)** β Create heterogeneous agents and dispatch them to local or remote devices through the device gateway, with an execution-target switcher in the composer and persistent CLI sessions. (#15065, #15179, #15022) - **iMessage on Desktop** β New iMessage setup and bridge on desktop, plus bot attachments across every platform. (#15228, #15227, #15029) - **Skills in the Composer** β Drag skill chips into chat, trigger installed skills from the slash menu mid-line, and surface project-level skills in the homogeneous agent runtime. (#15095, #15061, #15110) - **New Models** β DeepSeek V4 Flash/Pro and Gemini 3.5 Flash across providers, with thinking params for structured output and chat cost estimates. (#15031, #15001, #15051, #14876) - **Agent Runtime Observability** β OpenTelemetry GenAI semantic conventions plus per-call generation tracing. (#15123, #15124) --- ## π€ Agents & Heterogeneous Runtime - **Platform agent creation** β OpenClaw/Hermes creation UI, device guard, and remote dispatch backend. (#15065) - **Execution-target switcher** β Pick local vs remote execution directly in the composer; device-selection UX with actionable guidance. (#15179, #15111) - **CLI hetero dispatch** β OpenClaw/Hermes dispatch with persistent sessions and a notify protocol. (#15022) - **Gateway snapshot as source of truth** β Consume the gateway `uiMessages` snapshot at step boundaries to keep chat state consistent. (#15153, #15152) - **Client sub-agent as a normal tool call** β Simplifies the sub-agent execution path. (#15281) - **Hermes agent chain** β Implements the Hermes agent chain logic. (#15189) - **Device registry** β TRPC endpoints to register, list, update, and remove devices. (#15299) - **Desktop device routing** β Route gateway agent runs through `lh hetero exec`; restore `userId` in gateway dispatch and gate local-system by execution target. (#15132, #15232) - **Agent signals** β Anchor agent-signal receipts to messages and isolate memory-agent messages into a child thread. (#14969, #14921) --- ## π Onboarding - **Simplified first screen** β Defer topic creation to first send. (#15090) - **Market Agent Picker** β Added as a classic onboarding step, with template prefetch. (#14980, #15041) - **Welcome guidance** β Show agent welcome guidance on first run. (#15098) - **Mobile** β Adapt agent onboarding UI and restore Classic-step padding on mobile. (#15019, #15032) - **Discovery** β Streamline discovery to a single profession question. (#14987) - **Analytics** β Track onboarding step events and create-agent modal source. (#15133, #15028) --- ## π Documents, Pages & Knowledge - **Thread chat in preview** β Embed thread chat in the document preview portal. (#15216) - **Non-markdown rendering** β Render non-markdown docs as a read-only highlight. (#15272) - **Multi-select** β Multi-select delete in the document tree. (#15125) - **Page-agent streaming** β Preview `initPage` streaming arguments. (#15039) - **Per-agent topics** β Per-agent topic management page. (#15207) - **Server-side category** β Derive document category server-side and drop frontend predicates. (#15076) --- ## π§© Skills & Tools - **Drag skill chips** β Drag skills into chat input and register agent-document skills. (#15095) - **Slash menu** β Installed skills appear in the slash menu with a mid-line trigger. (#15061) - **Project skills** β Recognize project-level skills in the homogeneous agent runtime and surface them regardless of active device. (#15110, #15177) - **VFS archiving** β Archive oversized tool results to VFS instead of truncating. (#15074) - **@localfile mentions** β Drag folders into chat input as `@localFile` mentions on desktop. (#15071) --- ## π§ Model Runtime & Providers - **Error spec registry** β Unify error codes into a spec + pattern registry, split `ProviderBizError` into finer codes, classify Cloud-only codes via a tier digit, and add `DatabasePersistError`. (#15262, #15286, #15278, #15279) - **New models** β DeepSeek V4 Flash/Pro (opencode-go) and Gemini 3.5 Flash; DeepSeek V4 Pro on SiliconCloud. (#15031, #15001, #15017, #15267) - **Structured output** β Thinking params for structured output, Bedrock structured generation, and DeepSeek `generateObject` tool choice. (#15051, #15174, #15054) - **Cost** β Chat cost estimate support; preserve usage cost in custom streams. (#14876, #15218) --- ## π¬ Chat & User Experience - **Follow-up chips** β Extend follow-up chip suggestions to general chat with scene-specific model config. (#15101, #14797) - **Input drafts** β Persist unsent input drafts across tab switches and prevent repeated draft restore. (#14992, #15024) - **Command menu** β Order topic/message search by recency and promote inline type filters. (#15094, #14986) - **Zoom HUD** β Show a zoom-level HUD on Cmd +/β and Cmd 0. (#15294) - **Copy** β Unescape markdown escapes when copying user messages. (#15253) --- ## π₯οΈ Desktop - **App Nap fix** β Prevent App Nap from dropping the gateway WebSocket during display sleep. (#14994) - **File preview** β Preview `.cjs`/`.mjs`/no-extension files instead of binary fallback and expand `~` when opening local files. (#15168, #15284) - **Cross-platform settings** β Open settings via main-window navigation on Windows/Linux and restore the route after an update restart. (#15036, #14922) - **Token refresh** β Prevent frequent logout from token-refresh retries. (#14928) --- ## π Observability - **OTel GenAI** β Instrument Agent Runtime with OpenTelemetry GenAI semantic conventions. (#15123) - **Generation tracing** β Per-call `llm_generation_tracing` with a pre-allocated tracingId and recordFeedback router. (#15124, #15146) - **Error classification** β Persist `ERROR_CODE_SPECS` classification on operation errors. (#15273) --- ## ποΈ Database Migrations - **Batch migrations** β Topic usage stats, push tokens, `tasks.editor_data`, and document shares. (#15280) - **Tracing & eval tables** β Add `llm_generation_tracing` and agent eval experiment tables. (#15126) > Self-hosted operators should run the database migration (`pnpm db:migrate`, or restart with auto-migrate enabled) after upgrading. The changes are additive and backwards-compatible. --- ## π Security & Reliability - **Security:** Remove the `getPlaintextCred` tool to prevent plaintext credential exposure. (#14998) - **Security:** Prompt account selection for Google OAuth and add `prompt=consent` to the OIDC authorization URL to fix missing refresh tokens. (#15234, #15010) - **Reliability:** Preserve streamed content across a mid-stream cancel. (#15173) - **Reliability:** Bound the Redis command timeout and configure the Anthropic client timeout. (#15091, #15042) - **Reliability:** Prevent infinite recursion in the assistant chain. (#15288) --- ## π₯ Contributors Huge thanks to **15 contributors** who shipped **220 merged PRs** this cycle. @AnotiaWang Β· @sxjeru Β· @algojogacor Β· @hardy-one Β· @arvinxx Β· @Innei Β· @tjx666 Β· @lijian Β· @AmAzing129 Β· @rdmclin2 Β· @neko Β· @cy948 Β· @CanisMinor Β· @sudongyuer Β· @rivertwilight Plus @lobehubbot and renovate[bot] for maintenance. --- **Full Changelog**: v2.2.0...release/weekly-20260528
π» Change Type
π Related Issue
Follow-up to the error-classification work (#15262 / #15273). Surfaced while backfilling DC's operation dashboards: a large
unknownbucket was DrizzleFailed query:wraps.π Description of Change
Drizzle stringifies a failed query/transaction as
Failed query: <sql> params: <values>. These are harness-side persistence failures (DB write/read/txn could not complete), but:unknown.error_logsrows, user messages) contains substrings that match unrelated provider patterns β so message-based classification misfiled them asCapabilityNotSupported/InsufficientQuota/ModelNotFound.This adds a dedicated code:
agentRuntime.tsβ newDatabasePersistError.specs.tsβE7004, under the 7xxx Stream / Runtime (harness) bucket alongsideStreamChunkError/ConversationParentMissing.attribution: harness,severity: error,countAsFailure: true,httpStatus: 500.patterns.tsβFailed query:substring pattern placed first inERROR_PATTERNS.matchErrorPatternis first-match-wins, so claiming it up front both classifies these correctly and prevents the embedded blob from matching any pattern below it.match.test.tsβ asserts the wrap βDatabasePersistError, and that a blob embeddingInsufficientQuota/context length exceededstill resolves toDatabasePersistError(the false-positive guard).modelRuntime.tsβ en-US locale copy (other languages auto-translate).π§ͺ How to Test
```bash
bun run type-check # passes
cd packages/model-runtime && bunx vitest run src/errors/ # 49 pass
```
π Additional Information
The agent-gateway mirror is updated in parallel (
codes.ts+specs.tsE7004 + the first-positionFailed query:pattern).π€ Generated with Claude Code