Skip to content

✨ feat(stats): add daily token-usage mode to activity heatmap#15417

Merged
arvinxx merged 4 commits into
canaryfrom
feat/stats-token-heatmaps
Jun 2, 2026
Merged

✨ feat(stats): add daily token-usage mode to activity heatmap#15417
arvinxx merged 4 commits into
canaryfrom
feat/stats-token-heatmaps

Conversation

@arvinxx

@arvinxx arvinxx commented Jun 2, 2026

Copy link
Copy Markdown
Member

💻 Change Type

  • ✨ feat

🔗 Related Issue

🔀 Description of Change

The stats activity heatmap previously only visualized daily message count. This PR adds a Messages / Tokens toggle so users can also see their daily token usage over the past year.

The token mode sums assistant messages' metadata.usage.totalTokens (the source of truth for usage) bucketed by the day each message was created — so tokens land on the day they were actually consumed, rather than piling onto a topic's creation date (a long-running topic spreads its tokens across the real days). Heatmap level (color intensity) is scaled relative to the busiest day so the visualization stays readable regardless of absolute token volume.

Changes:

  • packages/databaseMessageModel.getTokenHeatmaps(): SQL aggregation SUM((metadata->'usage'->>'totalTokens')::numeric) over role = 'assistant' messages, grouped by DATE(created_at) for the last year, with relative 0–4 level scaling. Aggregation is done in SQL (not pulled into JS).
  • src/server/routers/lambda/message.ts: new getTokenHeatmaps TRPC query.
  • src/services/message: client getTokenHeatmaps() wrapper.
  • AiHeatmaps.tsx: a Segmented toggle (Messages / Tokens) that swaps the data source and tooltip/total labels. The toggle is hidden in the share/screenshot view.
  • i18n: added heatmaps.tooltipTokens, heatmaps.totalCountTokens, stats.tokens to default + en-US / zh-CN.

🧪 How to Test

Open Settings → Statistics → "Activity in the past year" and switch the segmented control to Tokens — the heatmap re-renders with per-day summed token usage.

  • Tested locally
  • Added/updated tests
  • No tests needed

MessageModel.getTokenHeatmaps is covered by new cases in message.stats.test.ts (per-day SUM, level scaling, role='assistant' filter excludes user/tool messages, cross-user isolation, missing-usage → 0, empty state).

⚡️ Performance

Token mode aggregates over the messages table instead of the small topics table. Benchmarked on PGlite (500k messages, target user 100k / 25k assistant): the query uses the existing messages_user_id_idx (bitmap index scan) with role/date applied as a residual filter, ~19ms on WASM (≈single-digit ms on native PG with warm cache) — fine for an infrequently-loaded stats page. No new index added: the only worthwhile one would be a partial (user_id, created_at) WHERE role='assistant' index, but the few-ms gain isn't worth the write amplification on the write-heavy messages table at current scale.

📸 Screenshots / Videos

Before After
Heatmap shows daily message count only Heatmap has a Messages/Tokens toggle; Tokens shows daily token usage

📝 Additional Information

Token mode is purely additive (default remains Messages) and introduces no schema/migration changes — it reads the existing metadata.usage on assistant messages.

🤖 Generated with Claude Code

@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jun 2, 2026
@vercel

vercel Bot commented Jun 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
lobehub Ready Ready Preview, Comment Jun 2, 2026 5:41pm

Request Review

@dosubot dosubot Bot added feature:settings Settings and configuration issues feature:topic Topic/Conversation/Session management i18n labels Jun 2, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've reviewed this pull request using the Sourcery rules engine

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7d4d1d344a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/database/src/models/topic.ts Outdated
@codecov

codecov Bot commented Jun 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 45.41485% with 125 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.00%. Comparing base (8dee729) to head (202c25f).
⚠️ Report is 2 commits behind head on canary.

Additional details and impacted files
@@             Coverage Diff             @@
##           canary   #15417       +/-   ##
===========================================
- Coverage   89.74%   71.00%   -18.75%     
===========================================
  Files         862     3214     +2352     
  Lines      105577   321437   +215860     
  Branches    10216    28295    +18079     
===========================================
+ Hits        94746   228221   +133475     
- Misses      10656    93041    +82385     
  Partials      175      175               
Flag Coverage Δ
app 61.82% <18.83%> (?)
database 92.49% <100.00%> (+0.03%) ⬆️
packages/agent-runtime 80.48% <ø> (ø)
packages/builtin-tool-lobe-agent 18.52% <ø> (ø)
packages/context-engine 84.17% <ø> (ø)
packages/conversation-flow 91.29% <ø> (ø)
packages/file-loaders 87.89% <ø> (ø)
packages/memory-user-memory 74.99% <ø> (ø)
packages/model-bank 99.99% <ø> (ø)
packages/model-runtime 84.72% <ø> (ø)
packages/prompts 72.49% <ø> (ø)
packages/python-interpreter 92.90% <ø> (ø)
packages/ssrf-safe-fetch 0.00% <ø> (ø)
packages/types 35.36% <ø> (ø)
packages/utils 88.69% <ø> (ø)
packages/web-crawler 88.08% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Store 68.43% <ø> (∅)
Services 54.58% <33.33%> (∅)
Server 72.32% <75.00%> (∅)
Libs 57.01% <ø> (∅)
Utils 81.44% <ø> (-18.56%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Add a Messages/Tokens toggle to the stats activity heatmap. The token
mode sums assistant messages' `metadata.usage.totalTokens` (the source of
truth for usage) bucketed by the day each message was created, so tokens
land on the day they were actually consumed rather than on a topic's
creation date. Aggregation runs in SQL (SUM over the jsonb path, GROUP BY
date) and levels are scaled relative to the busiest day.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
arvinxx and others added 2 commits June 3, 2026 01:12
- Format tooltip token counts compactly (e.g. 44.2K, 12.5M) via the chart's
  customTooltip; message counts get thousand separators.
- Add a token-dimension summary row (cumulative / peak daily / current streak
  / longest streak) shown in token mode, derived client-side from the heatmap
  data over the past year.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the "longest task" figure to the token-mode stats row, computed from
the longest wall-clock agent operation (completedAt - startedAt) over the
past year — MAX in SQL on the agent_operations table, scoped by user and
using the (user_id, created_at) index. Rendered as a compact 1h 15m / 45s
duration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…itle

- Token is now the first/default segmented option (Messages second); the
  share card keeps Messages as its default.
- Move the Messages/Tokens toggle next to the section title (left) via a new
  StatsFormGroup `afterTitle` slot; day tags stay on the right.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@arvinxx arvinxx merged commit 2eb9e34 into canary Jun 2, 2026
35 checks passed
@arvinxx arvinxx deleted the feat/stats-token-heatmaps branch June 2, 2026 17:54
@arvinxx arvinxx mentioned this pull request Jun 3, 2026
arvinxx added a commit that referenced this pull request Jun 4, 2026
# 🚀 LobeHub Release (20260604)

**Release Date:** June 4, 2026  
**Since v2.2.1:** 88 merged PRs · 11 contributors

> This week brings Execution Devices out of the lab — run agents and
Claude Code on any configured local or remote machine — alongside Claude
Opus 4.8, token-usage analytics, and Page sharing.

---

## ✨ Highlights

- **Execution Devices** — Pick where an agent runs. Desktop and CLI
devices auto-register with a stable machine ID, route through the
gateway by channel, and surface a device switcher in the chat input. Run
remote Claude Code on a configured device, with a recent-directory
picker you can drag to reorder. (#15300, #15315, #15322, #15343, #15351,
#15371)
- **Claude Opus 4.8** — Day-one support for Anthropic's latest model.
(#15314)
- **Token-usage analytics** — A new token-usage mode on the activity
heatmap, backed by a denormalized topic usage/cost rollup so totals stay
accurate without recomputing from messages. (#15365, #15417, #15425)
- **Page sharing** — Share a Page through a dedicated document share
flow, plus new Workspace and Agent share tables. (#15309, #15439)
- **Self-iteration agents** — Agent Signal's execAgent migration lands a
server-runtime bridge, async memory writer, and a registered
self-iteration tool package, with a CLI trigger command for testing.
(#15360, #15364, #15392)
- **Knowledge search** — BM25 search now extends to file-backed
documents, and the portal ships an editable CodeMirror viewer for local
files with document highlighting. (#15247, #15298)

---

## 🏗️ Core Agent & Architecture

### Agent Signal & Runtime

- **execAgent migration** — Server-runtime bridge, completion
projection, async memory writer, and removal of the legacy
`executeSelfIteration` path. (#15392)
- Registered the self-iteration builtin tool package and restored the
three mode-specific self-iteration agent slugs. (#15202, #15364)
- Added a CLI trigger command with a golden-snapshot fixture for Agent
Signal. (#15360)
- **Skill priority** — Agent Builder now emits a skill-priority
instruction with matching server runtime. (#15409)
- Retry empty LLM completions instead of silently finishing the turn.
(#15355)
- Classify topic/agent/session foreign-key violations as
`ConversationParentMissing` for clearer recovery. (#15408)
- Persist canonical nested usage/performance on assistant messages, and
re-link orphan tool messages at the raw bucket write boundary. (#15359,
#15438)
- Guard `createAgent` against LLM double-encoded array fields. (#15381)

---

## 🖥️ Execution Devices & Gateway

- Auto-register desktop and CLI devices with a stable machine ID, and
add the `@lobechat/device-identity` package. (#15300, #15321)
- New Devices settings page behind the Execution Device Switcher lab,
with a device switcher shown for all agents in the chat input. (#15315,
#15371)
- `connectionId` + channel routing across the gateway client and device
list; preset the local device on the first LLM request for the 本机
target. (#15322, #15435)
- Run remote Claude Code on a configured device, with drag-to-reorder
recent-directory management and client renders for device tool results.
(#15343, #15351, #15437)
- Preserve content and state across gateway tool calls, and prevent
duplicate streaming from stale reconnects. (#15114, #15354)

---

## 🖥️ CLI & Desktop

- Preserve content/state for connect local file and shell tools; render
the `runCommand` tool result card. (#15441, #15442)
- New `lh topic view` command; CLI now auto-registers its device on
login, matching desktop. (#15340, #15377)
- Resolve CLI tools from the shell `PATH`, and clarify local command
session handling. (#15368, #15389)
- Relocate visual-ref helpers to `@lobechat/const` to fix a renderer
crash; upload `.blockmap` files to S3 for differential updates. (#15326,
#15369)
- Fix a market OAuth expiry that triggered the wrong re-login modal, and
kill dev child processes on parent shutdown. (#15246, #15290)

---

## 🗂️ Pages, Library & Knowledge

- Document share flow with business slot stubs, plus Workspace and Agent
share tables. (#15309, #15439)
- Export Agent profiles as Markdown, preserving an empty agent prompt on
export. (#15312, #15316)
- Editable CodeMirror viewer for local files with document highlighting;
BM25 search extended to file-backed documents. (#15247, #15298)
- Default new Agent-doc files to `.md` and preserve IME composition;
refresh folder data on slug switch and dedupe breadcrumb fetches.
(#15335, #15427)

---

## 💬 Chat & User Experience

- Group-by-status mode for the Topic sidebar; dropped the legacy
session→agentId compatibility path from Topic queries. (#15366, #15378)
- Restore editor focus after the file picker closes, and close the skill
dropdown before navigating to settings. (#15391, #15394)
- Strip markdown tokens from fallback Topic titles; keep an open
ActionBar popup when hovering another message. (#15303, #15372)
- Stabilize home starter loading and stop transliterating model names in
the home starter; show artifact source while streaming. (#15310, #15324,
#15386)
- Group the sidebar spacer with recents and agents. (#15373)

---

## 📊 Analytics, Tasks & Notifications

- Token-usage mode on the activity heatmap, backed by a denormalized
topic usage/cost rollup. (#15365, #15417, #15425)
- Push: new `PushChannel`, receipt cron, and `pushToken` tRPC API.
(#15233)
- Tasks now support file and image attachments. (#15141)

---

## 🧩 Models & Providers

- Support Claude Opus 4.8 and configurable model routing with starters.
(#15314, #15384)
- MiniMax M3: new model entry and an Anthropic video runtime. (#15380,
#15403)
- Add `intern-s2-preview` with `thinking_mode`, and `step-3.7-flash`
support. (#15308, #15317)
- Block disabling the official provider; fix default provider setup in
business mode. (#15379, #15382)

---

## 🎨 UI & Modals

- Migrate modals to `@lobehub/ui/base-ui` (LOBE-9711 + eval batch),
including the create-custom-model and feedback/changelog modals.
(#15401, #15416)
- Restructure confirmModal title and content across deletion flows;
polish the service-model form and migrate its Switch to base-ui.
(#15426, #15440)
- Wrap the BlueBubbles bridge config into a connection card; update
`@lobehub/ui` to v5.15.5. (#15325, #15342)

---

## 🔒 Reliability

- Replace hardcoded `session_context` values with template variables in
credentials. (#15352)
- Point `CHANGELOG_URL` to `/changelog`. (#15428)

---

## 👥 Contributors

Huge thanks to **11 contributors** who shipped **88 merged PRs** this
cycle.

@hezhijie0327 · @qybaihe · @sxjeru · @arvinxx · @Innei · @tjx666 ·
@lijian · @sudongyuer · @cy948 · @rivertwilight · @AmAzing129

Plus @lobehubbot and renovate[bot] for maintenance.

---

**Full Changelog**: v2.2.1...release/weekly-20260604
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature:settings Settings and configuration issues feature:topic Topic/Conversation/Session management i18n size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant