Skip to content

🐛 fix(heterogeneous-agents): emit per-turn usage for batch-mode Claude Code#15577

Merged
arvinxx merged 3 commits into
canaryfrom
fix/cc-batch-mode-usage
Jun 9, 2026
Merged

🐛 fix(heterogeneous-agents): emit per-turn usage for batch-mode Claude Code#15577
arvinxx merged 3 commits into
canaryfrom
fix/cc-batch-mode-usage

Conversation

@arvinxx

@arvinxx arvinxx commented Jun 9, 2026

Copy link
Copy Markdown
Member

💻 Change Type

  • 🐛 fix

🔀 Description of Change

Follow-up to the CC remote-spawn model/provider fix. Two symptoms on remotely-spawned (device / sandbox) Claude Code runs:

  1. No token total in the model tag.
  2. Model showed claude-opus-4-8[1m] instead of the canonical claude-opus-4-8.

Root cause: device + sandbox runs spawn CC through the lh hetero exec CLI, which defaults to batch mode (includePartialMessages: false, no --include-partial-messages) — unlike the desktop driver, which always streams partial messages. In batch mode:

  • CC emits no stream_event / message_delta, so the partial-mode turn_metadata (which carries authoritative per-turn usage) never fires.
  • handleAssistant deliberately skips usage on assistant events, assuming the stale message_start echo that only exists in partial mode.
  • The grand-total result_usage is intentionally ignored (avoids double-counting).

Net result for batch runs: no usage persisted at all → no token count. And the only model source was system init, whose model field carries CC's [1m] 1M-context beta marker (assistant/message_delta events report the clean claude-opus-4-8).

Fix: the adapter now tracks whether any stream_event was seen (= partial mode). When none has (batch mode), it emits per-turn usage from the assistant event as turn_metadata — the assistant usage is authoritative there, not a stale echo. This:

  • persists usage → the token total renders, and
  • carries the clean claude-opus-4-8 model id, which supersedes the [1m] init-captured value (and matches what ModelIcon / pricing lookups expect).

Partial mode (desktop / local) is unchanged — message_delta still owns usage, and the assistant-event path stays skipped via the sawStreamEvent guard, so no double-counting.

🧪 How to Test

  • Added/updated tests

Updated claudeCode.test.ts: the existing "no turn_metadata on assistant" case now primes partial mode with a stream_event first; a new case asserts batch mode (no stream_event) emits turn_metadata with the clean model + authoritative usage. Full adapter suite (91 tests) passes; type-check clean on touched files.

Manual: run a Claude Code agent on a remote device, send a short prompt — the model tag should show claude-opus-4-8 with the token count, not claude-opus-4-8[1m] with no tokens.

📝 Additional Information

Shared-package change (@lobechat/heterogeneous-agents); affects all batch-mode CC runs (device, sandbox, CLI). No server or UI changes needed — the existing turn_metadata persistence path handles it.

…e Code

Device + sandbox runs spawn Claude Code via the `lh hetero exec` CLI in BATCH
mode (no `--include-partial-messages`), unlike the desktop driver which always
streams partial messages. In batch mode CC emits no `message_delta`, and the
adapter deliberately skipped usage on `assistant` events (assuming the stale
`message_start` echo that only exists in partial mode). The grand-total
`result_usage` is intentionally ignored to avoid double-counting, so batch runs
ended up persisting NO usage at all — the model tag showed no token count.

Track whether any `stream_event` was seen (partial mode); when none has been
(batch mode), emit per-turn usage from the `assistant` event as turn_metadata.
The assistant event's usage is authoritative in batch mode, not a stale echo.

This also fixes the model tag showing `claude-opus-4-8[1m]`: the `[1m]` 1M-context
beta marker only appears in the `system init` model field, while `assistant`
events report the canonical `claude-opus-4-8`. The new turn_metadata carries the
clean id, which supersedes the init-captured one (and matches the id ModelIcon /
pricing lookups expect).

Partial mode (desktop/local) is unchanged — `message_delta` still owns usage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @arvinxx, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@vercel

vercel Bot commented Jun 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
lobehub Ready Ready Preview, Comment Jun 9, 2026 6:57am

Request Review

@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jun 9, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f268d21c0e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/heterogeneous-agents/src/adapters/claudeCode.ts
@codecov

codecov Bot commented Jun 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.49%. Comparing base (0a6b02c) to head (b57bce7).
⚠️ Report is 2 commits behind head on canary.

Additional details and impacted files
@@            Coverage Diff             @@
##           canary   #15577      +/-   ##
==========================================
- Coverage   70.50%   70.49%   -0.01%     
==========================================
  Files        3312     3312              
  Lines      327060   327060              
  Branches    34721    29894    -4827     
==========================================
- Hits       230582   230574       -8     
- Misses      96296    96304       +8     
  Partials      182      182              
Flag Coverage Δ
app 61.23% <ø> (-0.01%) ⬇️
database 92.24% <ø> (ø)
packages/agent-manager-runtime 49.69% <ø> (ø)
packages/agent-runtime 81.06% <ø> (ø)
packages/builtin-tool-lobe-agent 18.52% <ø> (ø)
packages/context-engine 84.12% <ø> (ø)
packages/conversation-flow 91.29% <ø> (ø)
packages/device-gateway-client 90.18% <ø> (ø)
packages/eval-dataset-parser 95.15% <ø> (ø)
packages/eval-rubric 76.11% <ø> (ø)
packages/fetch-sse 85.57% <ø> (ø)
packages/file-loaders 87.89% <ø> (ø)
packages/memory-user-memory 74.99% <ø> (ø)
packages/model-bank 99.99% <ø> (ø)
packages/model-runtime 84.22% <ø> (ø)
packages/prompts 72.51% <ø> (ø)
packages/python-interpreter 92.90% <ø> (ø)
packages/ssrf-safe-fetch 0.00% <ø> (ø)
packages/types 35.25% <ø> (ø)
packages/utils 85.05% <ø> (ø)
packages/web-crawler 88.08% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Store 68.23% <ø> (ø)
Services 54.21% <ø> (ø)
Server 71.30% <ø> (-0.01%) ⬇️
Libs 55.82% <ø> (-0.14%) ⬇️
Utils 81.93% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

arvinxx and others added 2 commits June 9, 2026 14:23
The multi-step E2E fixture has no `stream_event` records (batch mode) and 5
assistant events with `message.usage`, so the new batch-mode path now emits 5
turn_metadata events. Update the expectation from 0 — this validates the fix on
a realistic device/sandbox session: per-turn usage lands with the canonical
model id.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…pawned CLI

The local CLI spawn forwarded the entire `process.env` to `claude`, so a
developer with `ANTHROPIC_API_KEY` / `ANTHROPIC_AUTH_TOKEN` / `ANTHROPIC_BASE_URL`
exported in their shell had it inherited by the CLI — overriding its own
subscription login and surfacing as a baffling "Invalid API key" + non-zero
exit on every message.

Strip those three vars from the inherited env via `buildInheritedSpawnEnv`.
`session.env` is still spread last, so an agent that explicitly configures an
API key continues to win. Adds regression tests for both the strip and the
override.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@arvinxx arvinxx merged commit 434532c into canary Jun 9, 2026
35 checks passed
@arvinxx arvinxx deleted the fix/cc-batch-mode-usage branch June 9, 2026 08:51
arvinxx added a commit that referenced this pull request Jun 10, 2026
# 🚀 LobeHub Release (20260610)

**Release Date:** June 10, 2026  
**Since v2.2.2:** 131 merged PRs · 13 contributors

> This weekly release strengthens agent collaboration across cloud,
desktop, CLI, and workspace flows, with steadier runtime behavior and a
broader foundation for workspace-scoped data.

---

## ✨ Highlights

- **Agent execution across devices** — Unifies per-device working
directories, project skill discovery, and sub-agent suspend/resume
behavior across server, QStash, and device RPC flows. (#15543, #15566,
#15481, #15620, #15591)
- **Connector and sandbox platform** — Expands connector permissions,
custom OAuth MCP connector onboarding, sandbox provider support, and
user-uploaded file sync into cloud sandbox runs. (#15463, #15546,
#15184, #15550)
- **Desktop and CLI reliability** — Fixes desktop cold-start,
auto-update, Windows build, CLI skill discovery, and `lh connect` agent
dispatch paths. (#15547, #15525, #15527, #15562, #15632, #15634)
- **Pages and sharing** — Refreshes topic sharing, improves Page Editor
layout behavior, and routes Page Agent tool execution through the
server-side editor path. (#15581, #15556, #15588, #15023, #15610)
- **Model availability and provider updates** — Adds user-scoped LobeHub
model availability, Claude Fable 5, Qwen thinking preservation, and
MiniMax M3 updates. (#15590, #15639, #13494, #15376)

---

## 🏗️ Core Product & Architecture

### Agent Runtime & Heterogeneous Agents

- Improves sub-agent lifecycle handling, including async suspend/resume,
queue-mode QStash resume delivery, and blocking nested sub-agent calls.
(#15481, #15620, #15575)
- Stabilizes heterogeneous agent ingestion and streaming with raw stream
dumps, per-turn usage, image forwarding on regenerate, and
duplicate-text fixes. (#15602, #15577, #15592, #15585)
- Adds execution-device and working-directory controls across device
RPC, legacy defaults, and remote-spawned Claude Code sessions. (#15543,
#15566, #15591, #15572)
- Improves runtime diagnostics and compatibility, including Gemini
multimodal output capture, abort stream semantics, and trace quality
analysis. (#15535, #13677, #15508)

---

## 📱 Platforms, Integrations & UX

### Connectors, Sandbox & Tools

- Ships API-level connector tool permissions, custom OAuth MCP connector
onboarding, and connector-first runtime execution. (#15463, #15546)
- Adds sandbox provider support, cloud sandbox file sync, and safer
external URL file input handling with SSRF validation. (#15184, #15550,
#12657)
- Improves tool visibility and execution with pinned app-fixed tools,
ANSI output rendering, gateway-tunneled MCP calls, and automatic
headless tool runs. (#15509, #15516, #15469, #15492)

### Desktop, CLI & Web UX

- Restores desktop startup and reload behavior, preserves IPC error
causes, and keeps the tab bar new-tab action visible across routes.
(#15547, #15597, #15638)
- Fixes desktop update and build stability for browser quit guards,
macOS update signing, and Windows Visual Studio detection. (#15525,
#15527, #15562)
- Shows the plan-limit upgrade UI on desktop builds. (#15628)
- Adds the Agent Run delivery checker and fixes CLI device dispatch plus
skill list/search output. (#15489, #15634, #15632)
- Refreshes onboarding, auth source preservation, topic UI states,
referral/Fable campaign copy, and chat-input control bar behavior.
(#15629, #15544, #15573, #15614, #15616, #15617, #15622, #15643)

---

## 🔒 Security, Reliability & Rollout Notes

- External URL file input now includes SSRF validation for safer Google
file handling. (#12657)
- Database workspace-scope migrations are part of this release;
self-hosted operators should run the normal migration path before
serving the updated app. (#15446, #15465, #15468, #15472)
- The release branch was re-cut from `canary` and includes the latest
`main` release-version commit so `v2.2.2` is the verified compare base.

---

## 👥 Contributors

@ONLY-yours, @sxjeru, @hardy-one, @xujingli, @hezhijie0327, @Coooolfan,
@arvinxx, @tjx666, @Innei, @rivertwilight, @rdmclin2, @cy948,
@AmAzing129

**Full Changelog**:
v2.2.2...release/weekly-20260610-recut-3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant