Skip to content

✨ feat(tool): archive oversized tool results to VFS instead of truncating#15074

Merged
Innei merged 6 commits into
canaryfrom
feat/tool-result-archive
May 21, 2026
Merged

✨ feat(tool): archive oversized tool results to VFS instead of truncating#15074
Innei merged 6 commits into
canaryfrom
feat/tool-result-archive

Conversation

@Innei

@Innei Innei commented May 21, 2026

Copy link
Copy Markdown
Member

πŸ’» Change Type

  • ✨ feat
  • πŸ› fix
  • ♻️ refactor
  • πŸ’„ style
  • πŸ‘· build
  • ⚑️ perf
  • βœ… test
  • πŸ“ docs
  • πŸ”¨ chore

πŸ”— Related Issue

Design spec: docs/superpowers/specs/2026-05-21-tool-result-archive-design.md

πŸ”€ Description of Change

When a tool execution result exceeds the configured toolResultMaxLength, the full content is now persisted to the agent VFS under ./.tool-results/<toolCallId>.md and the LLM receives a truncated preview with an archive path pointer. This prevents data loss from unconditional truncation while keeping context windows bounded.

Key changes:

  1. archiveToolResult.ts (new) β€” Core archival logic:

    • Persists oversized content via AgentDocumentVfsService.write
    • Associates the document with the current topic via TopicDocumentModel
    • Graceful fallback to truncation-only when VFS context is unavailable or writing fails
  2. ToolExecutionContext.skipResultTruncation β€” New flag that lets the AgentRuntime boundary receive full content for archival before any truncation occurs. The ToolExecutionService respects this flag in both success and error paths.

  3. VFS line-range reads (loc parameter) β€” Extends AgentDocumentVfsService.read() with an optional [startLine, endLine) range, enabling the agent to inspect archived files incrementally. The router schema validates the tuple and the AgentDocumentReadResult type now includes lineCount, charCount, loc, totalLineCount, and totalCharCount.

  4. Runtime executor wiring β€” Both single-tool and batch-tool execution paths now pass skipResultTruncation: true and run archiveRuntimeToolResult() on the raw result before passing it downstream.

πŸ§ͺ How to Test

  • Added/updated tests

New test files:

  • archiveToolResult.test.ts β€” Covers: under-limit passthrough, successful archive, duplicate association guard, missing-context fallback, write-failure graceful degradation
  • index.test.ts (toolExecution) β€” Covers: skipResultTruncation bypasses truncation, default behavior preserves truncation

Updated test files:

  • RuntimeExecutors.test.ts β€” Asserts skipResultTruncation: true in executor call
  • agentDocumentVfs.integration.test.ts β€” Integration test for line-range read via router
  • agentDocumentVfs/index.test.ts β€” Unit tests for loc slicing on both ordinary files and mounted skill files

Run:

bunx vitest run --silent="passed-only" src/server/services/toolExecution/__tests__/
bunx vitest run --silent="passed-only" src/server/services/agentDocumentVfs/index.test.ts
bunx vitest run --silent="passed-only" src/server/routers/lambda/__tests__/integration/agentDocumentVfs.integration.test.ts

πŸ“ Additional Information

The archival directory ./.tool-results/ is a VFS path that resolves to agent documents scoped by agentId + topicId. The design spec is included in this PR for reference.

@vercel

vercel Bot commented May 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
lobehub Ready Ready Preview, Comment May 21, 2026 5:45pm

Request Review

@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label May 21, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @Innei, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@dosubot dosubot Bot added the feature:tool Tool calling and function execution label May 21, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

πŸ’‘ Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 851a8fa2d5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with πŸ‘.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

userId?: string;
}

const buildArchivePath = (toolCallId: string) => `${TOOL_RESULTS_DIR}/${toolCallId}.md`;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Include execution scope in archive file names

Deriving archive paths from only toolCallId (./.tool-results/<id>.md) makes unrelated executions overwrite each other whenever the same call id is reused (tool call ids are not globally unique across conversations/topics, and VFS write updates existing files at the same path). That means an older tool message can point to content from a later run, silently corrupting the archive history this feature is meant to preserve. Please namespace the filename with stable scope (for example topic/thread/operation/message) instead of relying on bare toolCallId.

Useful? React with πŸ‘Β / πŸ‘Ž.

@codecov

codecov Bot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.67797% with 22 lines in your changes missing coverage. Please review.
βœ… Project coverage is 70.80%. Comparing base (b45cb41) to head (4591c34).

Additional details and impacted files
@@            Coverage Diff             @@
##           canary   #15074      +/-   ##
==========================================
+ Coverage   70.77%   70.80%   +0.02%     
==========================================
  Files        3106     3108       +2     
  Lines      308827   309042     +215     
  Branches    33447    33491      +44     
==========================================
+ Hits       218581   218819     +238     
+ Misses      90079    90056      -23     
  Partials      167      167              
Flag Coverage Ξ”
app 61.49% <90.67%> (+0.05%) ⬆️
database 92.15% <ΓΈ> (ΓΈ)
packages/agent-runtime 80.48% <ΓΈ> (ΓΈ)
packages/builtin-tool-lobe-agent 19.87% <ΓΈ> (ΓΈ)
packages/context-engine 84.07% <ΓΈ> (ΓΈ)
packages/conversation-flow 91.28% <ΓΈ> (ΓΈ)
packages/file-loaders 87.89% <ΓΈ> (ΓΈ)
packages/memory-user-memory 75.01% <ΓΈ> (ΓΈ)
packages/model-bank 99.99% <ΓΈ> (ΓΈ)
packages/model-runtime 83.69% <ΓΈ> (ΓΈ)
packages/prompts 71.60% <ΓΈ> (ΓΈ)
packages/python-interpreter 92.90% <ΓΈ> (ΓΈ)
packages/ssrf-safe-fetch 0.00% <ΓΈ> (ΓΈ)
packages/types 35.20% <ΓΈ> (ΓΈ)
packages/utils 88.02% <ΓΈ> (ΓΈ)
packages/web-crawler 87.74% <ΓΈ> (ΓΈ)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Ξ”
Store 68.03% <96.29%> (+0.02%) ⬆️
Services 54.66% <56.66%> (+<0.01%) ⬆️
Server 72.10% <96.57%> (+0.09%) ⬆️
Libs 56.39% <ΓΈ> (ΓΈ)
Utils 85.01% <ΓΈ> (ΓΈ)
πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Innei added 6 commits May 22, 2026 01:15
…ting

When tool execution results exceed the configured max length, the full
content is now persisted to the agent's VFS under ./.tool-results/ and
the LLM receives a truncated preview with an archive path pointer.

Key changes:
- Add archiveToolResultIfNeeded() to persist oversized results via VFS
- Add skipResultTruncation flag to ToolExecutionContext so the runtime
  can receive full content for archival before truncation
- Add line-range (loc) support to VFS reads for inspecting archived files
- Extend AgentDocumentReadResult with line/char count and loc metadata
- Wire archival into both single-tool and batch-tool executor paths
…documents reads

Server-only AgentRuntime archive missed the main webapi chat loop where tool
execution happens in the browser. Route oversized tool results from the client
plugin executors through a new aiChat.archiveToolResult tRPC mutation that
reuses archiveToolResultIfNeeded, so calculator/MCP/klavis/lobehub-skill calls
all archive to the VFS instead of just being truncated.

Flatten the archive layout to ./.tool-results/<topicId>_<toolCallId>.md to dodge
a nested-folder edge case in the VFS resolver, surface the agent_documents.id
in the model-facing hint so the LLM can call lobe-agent-documents.readDocument
directly, and bypass archive entirely for lobe-agent-documents tool results so
reading the archive does not loop back into another archive write.

Also harden truncateToolResult against splitting a UTF-16 surrogate pair: when
the cutoff lands on a high surrogate, step back one code unit so JSON.stringify
no longer emits a lone \\uD83D escape that DeepSeek / Anthropic reject as
'unexpected end of hex escape'.

Includes a small ApprovalMode dropdown placement + trigger styling tweak.
The path is already excluded by .gitignore line 149; the design spec was only
in the index because an earlier commit forced it in. Remove it from tracking
while keeping the local copy so the ignore rule actually takes effect.
…osed emoji

A single surrogate pair was easy to get right; the real-world worry is ZWJ
sequences like πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦ where four surrogate pairs are stitched with ZWJs
into one grapheme. Sweep every cutoff position across that family emoji and
assert the result never leaves a lone high surrogate and always round-trips
through JSON.stringify / JSON.parse.
Thinking accordion and assistant content loading dot kept spinning after
the user aborted a stream or the run ended without closing the inline
`<think>` tag. Gate the markdown thinking plugins on
`isMessageGenerating(id)` and bail out of `ContentLoading` when no
running operation exists for the message.
@Innei Innei force-pushed the feat/tool-result-archive branch from 9e456c3 to 4591c34 Compare May 21, 2026 17:22
@Innei Innei merged commit c056760 into canary May 21, 2026
35 checks passed
@Innei Innei deleted the feat/tool-result-archive branch May 21, 2026 18:07
arvinxx added a commit that referenced this pull request May 29, 2026
# πŸš€ LobeHub Release (20260528)

**Release Date:** May 28, 2026  
**Since v2.2.0:** 220 merged PRs Β· 15 contributors

> This cycle brings heterogeneous "platform agents" you can dispatch to
local or remote devices, a rebuilt onboarding flow, document-centric
chat, and a unified model-runtime error model β€” with new DeepSeek V4 and
Gemini 3.5 Flash support along the way.

---

## ✨ Highlights

- **More Hetero Agents (OpenClaw / Hermes)** β€” Create heterogeneous
agents and dispatch them to local or remote devices through the device
gateway, with an execution-target switcher in the composer and
persistent CLI sessions. (#15065, #15179, #15022)
- **iMessage on Desktop** β€” New iMessage setup and bridge on desktop,
plus bot attachments across every platform. (#15228, #15227, #15029)
- **Skills in the Composer** β€” Drag skill chips into chat, trigger
installed skills from the slash menu mid-line, and surface project-level
skills in the homogeneous agent runtime. (#15095, #15061, #15110)
- **New Models** β€” DeepSeek V4 Flash/Pro and Gemini 3.5 Flash across
providers, with thinking params for structured output and chat cost
estimates. (#15031, #15001, #15051, #14876)
- **Agent Runtime Observability** β€” OpenTelemetry GenAI semantic
conventions plus per-call generation tracing. (#15123, #15124)

---

## πŸ€– Agents & Heterogeneous Runtime

- **Platform agent creation** β€” OpenClaw/Hermes creation UI, device
guard, and remote dispatch backend. (#15065)
- **Execution-target switcher** β€” Pick local vs remote execution
directly in the composer; device-selection UX with actionable guidance.
(#15179, #15111)
- **CLI hetero dispatch** β€” OpenClaw/Hermes dispatch with persistent
sessions and a notify protocol. (#15022)
- **Gateway snapshot as source of truth** β€” Consume the gateway
`uiMessages` snapshot at step boundaries to keep chat state consistent.
(#15153, #15152)
- **Client sub-agent as a normal tool call** β€” Simplifies the sub-agent
execution path. (#15281)
- **Hermes agent chain** β€” Implements the Hermes agent chain logic.
(#15189)
- **Device registry** β€” TRPC endpoints to register, list, update, and
remove devices. (#15299)
- **Desktop device routing** β€” Route gateway agent runs through `lh
hetero exec`; restore `userId` in gateway dispatch and gate local-system
by execution target. (#15132, #15232)
- **Agent signals** β€” Anchor agent-signal receipts to messages and
isolate memory-agent messages into a child thread. (#14969, #14921)

---

## πŸš€ Onboarding

- **Simplified first screen** β€” Defer topic creation to first send.
(#15090)
- **Market Agent Picker** β€” Added as a classic onboarding step, with
template prefetch. (#14980, #15041)
- **Welcome guidance** β€” Show agent welcome guidance on first run.
(#15098)
- **Mobile** β€” Adapt agent onboarding UI and restore Classic-step
padding on mobile. (#15019, #15032)
- **Discovery** β€” Streamline discovery to a single profession question.
(#14987)
- **Analytics** β€” Track onboarding step events and create-agent modal
source. (#15133, #15028)

---

## πŸ“„ Documents, Pages & Knowledge

- **Thread chat in preview** β€” Embed thread chat in the document preview
portal. (#15216)
- **Non-markdown rendering** β€” Render non-markdown docs as a read-only
highlight. (#15272)
- **Multi-select** β€” Multi-select delete in the document tree. (#15125)
- **Page-agent streaming** β€” Preview `initPage` streaming arguments.
(#15039)
- **Per-agent topics** β€” Per-agent topic management page. (#15207)
- **Server-side category** β€” Derive document category server-side and
drop frontend predicates. (#15076)

---

## 🧩 Skills & Tools

- **Drag skill chips** β€” Drag skills into chat input and register
agent-document skills. (#15095)
- **Slash menu** β€” Installed skills appear in the slash menu with a
mid-line trigger. (#15061)
- **Project skills** β€” Recognize project-level skills in the homogeneous
agent runtime and surface them regardless of active device. (#15110,
#15177)
- **VFS archiving** β€” Archive oversized tool results to VFS instead of
truncating. (#15074)
- **@localfile mentions** β€” Drag folders into chat input as `@localFile`
mentions on desktop. (#15071)

---

## 🧠 Model Runtime & Providers

- **Error spec registry** β€” Unify error codes into a spec + pattern
registry, split `ProviderBizError` into finer codes, classify Cloud-only
codes via a tier digit, and add `DatabasePersistError`. (#15262, #15286,
#15278, #15279)
- **New models** β€” DeepSeek V4 Flash/Pro (opencode-go) and Gemini 3.5
Flash; DeepSeek V4 Pro on SiliconCloud. (#15031, #15001, #15017, #15267)
- **Structured output** β€” Thinking params for structured output, Bedrock
structured generation, and DeepSeek `generateObject` tool choice.
(#15051, #15174, #15054)
- **Cost** β€” Chat cost estimate support; preserve usage cost in custom
streams. (#14876, #15218)

---

## πŸ’¬ Chat & User Experience

- **Follow-up chips** β€” Extend follow-up chip suggestions to general
chat with scene-specific model config. (#15101, #14797)
- **Input drafts** β€” Persist unsent input drafts across tab switches and
prevent repeated draft restore. (#14992, #15024)
- **Command menu** β€” Order topic/message search by recency and promote
inline type filters. (#15094, #14986)
- **Zoom HUD** β€” Show a zoom-level HUD on Cmd +/βˆ’ and Cmd 0. (#15294)
- **Copy** β€” Unescape markdown escapes when copying user messages.
(#15253)

---

## πŸ–₯️ Desktop

- **App Nap fix** β€” Prevent App Nap from dropping the gateway WebSocket
during display sleep. (#14994)
- **File preview** β€” Preview `.cjs`/`.mjs`/no-extension files instead of
binary fallback and expand `~` when opening local files. (#15168,
#15284)
- **Cross-platform settings** β€” Open settings via main-window navigation
on Windows/Linux and restore the route after an update restart. (#15036,
#14922)
- **Token refresh** β€” Prevent frequent logout from token-refresh
retries. (#14928)

---

## πŸ“Š Observability

- **OTel GenAI** β€” Instrument Agent Runtime with OpenTelemetry GenAI
semantic conventions. (#15123)
- **Generation tracing** β€” Per-call `llm_generation_tracing` with a
pre-allocated tracingId and recordFeedback router. (#15124, #15146)
- **Error classification** β€” Persist `ERROR_CODE_SPECS` classification
on operation errors. (#15273)

---

## πŸ—ƒοΈ Database Migrations

- **Batch migrations** β€” Topic usage stats, push tokens,
`tasks.editor_data`, and document shares. (#15280)
- **Tracing & eval tables** β€” Add `llm_generation_tracing` and agent
eval experiment tables. (#15126)

> Self-hosted operators should run the database migration (`pnpm
db:migrate`, or restart with auto-migrate enabled) after upgrading. The
changes are additive and backwards-compatible.

---

## πŸ”’ Security & Reliability

- **Security:** Remove the `getPlaintextCred` tool to prevent plaintext
credential exposure. (#14998)
- **Security:** Prompt account selection for Google OAuth and add
`prompt=consent` to the OIDC authorization URL to fix missing refresh
tokens. (#15234, #15010)
- **Reliability:** Preserve streamed content across a mid-stream cancel.
(#15173)
- **Reliability:** Bound the Redis command timeout and configure the
Anthropic client timeout. (#15091, #15042)
- **Reliability:** Prevent infinite recursion in the assistant chain.
(#15288)

---

## πŸ‘₯ Contributors

Huge thanks to **15 contributors** who shipped **220 merged PRs** this
cycle.

@AnotiaWang Β· @sxjeru Β· @algojogacor Β· @hardy-one Β· @arvinxx Β· @Innei Β·
@tjx666 Β· @lijian Β· @AmAzing129 Β· @rdmclin2 Β· @neko Β· @cy948 Β·
@CanisMinor Β· @sudongyuer Β· @rivertwilight

Plus @lobehubbot and renovate[bot] for maintenance.

---

**Full Changelog**: v2.2.0...release/weekly-20260528
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature:tool Tool calling and function execution size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant