Skip to content

feat(status): add API call count to session status and usage footer#48851

Open
Qixingchen wants to merge 3 commits intoopenclaw:mainfrom
Qixingchen:feat/api-call-count
Open

feat(status): add API call count to session status and usage footer#48851
Qixingchen wants to merge 3 commits intoopenclaw:mainfrom
Qixingchen:feat/api-call-count

Conversation

@Qixingchen
Copy link
Copy Markdown

@Qixingchen Qixingchen commented Mar 17, 2026

  • track API call count per run in embedded runner
  • persist per-run call count on session entries
  • show current-turn call count in /status output
  • show current-turn call count in response usage footer
  • keep calls semantics aligned with current-turn token usage
  • add/update unit tests for API call count display

Summary

  • Problem: Users cannot see how many API calls were made per turn, only token usage
  • Why it matters: Some providers charge per call; helps debug usage patterns and understand tool-call loops
  • What changed: Added callCount tracking in the embedded runner, persisted to session entry, displayed in both /status and response usage footer
  • What did NOT change: Token counting, cost calculation, existing status fields, cumulative session statistics

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs
  • Security hardening
  • Chore/infra

Scope

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

User-visible / Behavior Changes

  • /status output now includes 📞 Calls: N in the usage line (current turn only)
  • Response usage footer now includes 📞 Calls: N when responseUsage is enabled (current turn only)
  • Call count reflects actual API calls in the current turn (including tool-use loops)
  • No config changes; no new settings

Security Impact

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No

Repro + Verification

Environment

  • OS: Linux (Ubuntu 22.04)
  • Runtime/container: Node.js v22
  • Model/provider: Any
  • Integration/channel: Any

Steps

  1. Run /status command in any session with prior activity
  2. Observe the usage line includes 📞 Calls: N
  3. Enable responseUsage: full in config
  4. Send a message and observe usage footer includes call count
  5. Trigger tool calls and verify count increases per API call

Expected

🧮 Tokens: 154k in / 656 out · 💵 Cost: $0.40 · 📞 Calls: 3

Actual

Works as expected.

Evidence

  • Passing unit tests
    • src/auto-reply/status.test.ts - "shows API call count from the latest run"
    • src/auto-reply/reply/agent-runner-utils.test.ts - "formats response usage line with API call count"

Human Verification

  • Verified scenarios: Unit tests pass; code review completed
  • Edge cases checked: Zero calls, single call, multiple tool calls
  • What you did not verify: Live integration test (requires local build and gateway restart)

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No

Failure Recovery

  • How to disable/revert this change quickly: Revert the commit
  • Files/config to restore: None
  • Known bad symptoms reviewers should watch for: None

Risks and Mitigations

None. This is a display-only feature with no side effects.

- track API call count per run in embedded runner
- persist per-run call count on session entries
- show current-turn call count in /status output
- show current-turn call count in response usage footer
- keep calls semantics aligned with current-turn token usage
- add/update unit tests for API call count display
Copilot AI review requested due to automatic review settings March 17, 2026 08:18
@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S labels Mar 17, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Mar 17, 2026

Greptile Summary

This PR adds API call count tracking (callCount) per turn through the entire stack: embedded runner → session entry → /status display and response usage footer. The implementation is clean and well-tested, with one meaningful gap in followup-runner.ts where the new field is not forwarded to persistRunSessionUsage, causing followup-turn call counts to be silently dropped.

  • callCount is correctly incremented in UsageAccumulator and propagated through EmbeddedPiAgentMeta on both success and error paths.
  • agent-runner.ts correctly passes callCount to both persistRunSessionUsage and formatResponseUsageLine.
  • followup-runner.ts omits callCount from its persistRunSessionUsage call, so after a followup turn the session entry retains the stale callCount from the prior main turn instead of the followup's actual count.
  • The refactored usageCostLine builder in status.ts using filter(Boolean).join(" · ") is a clean improvement and remains functionally equivalent for the existing cases.
  • Test coverage for both the formatter and the status message is solid.

Confidence Score: 3/5

  • Safe to merge with one caveat: followup turns silently drop the call count, leading to stale display in /status.
  • All display and persistence logic in the main turn path is correct. The one gap is followup-runner.ts not forwarding callCount to persistRunSessionUsage, which produces a misleading call count in /status after any followup turn. It's a one-line fix but does affect correctness of the feature for a common code path.
  • src/auto-reply/reply/followup-runner.ts — missing callCount in the persistRunSessionUsage call at line 268.

Comments Outside Diff (1)

  1. src/auto-reply/reply/followup-runner.ts, line 268-279 (link)

    P1 callCount not persisted for followup turns

    persistRunSessionUsage is called here without callCount, so when a followup turn runs (e.g., triggered by a tool-call loop), the API call count for that turn is silently dropped. Because session-usage.ts only writes callCount when the param is present and > 0, the session entry will retain the stale callCount from the previous main turn instead of reflecting the followup turn's actual call count. This means /status will show a misleading (too-low or too-high) call count after any followup turn.

    The same field is correctly passed in agent-runner.ts at line 479.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: src/auto-reply/reply/followup-runner.ts
    Line: 268-279
    
    Comment:
    **`callCount` not persisted for followup turns**
    
    `persistRunSessionUsage` is called here without `callCount`, so when a followup turn runs (e.g., triggered by a tool-call loop), the API call count for that turn is silently dropped. Because `session-usage.ts` only writes `callCount` when the param is present and `> 0`, the session entry will retain the stale `callCount` from the previous main turn instead of reflecting the followup turn's actual call count. This means `/status` will show a misleading (too-low or too-high) call count after any followup turn.
    
    The same field is correctly passed in `agent-runner.ts` at line 479.
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/auto-reply/reply/followup-runner.ts
Line: 268-279

Comment:
**`callCount` not persisted for followup turns**

`persistRunSessionUsage` is called here without `callCount`, so when a followup turn runs (e.g., triggered by a tool-call loop), the API call count for that turn is silently dropped. Because `session-usage.ts` only writes `callCount` when the param is present and `> 0`, the session entry will retain the stale `callCount` from the previous main turn instead of reflecting the followup turn's actual call count. This means `/status` will show a misleading (too-low or too-high) call count after any followup turn.

The same field is correctly passed in `agent-runner.ts` at line 479.

```suggestion
        await persistRunSessionUsage({
          storePath,
          sessionKey,
          usage,
          lastCallUsage: runResult.meta?.agentMeta?.lastCallUsage,
          promptTokens,
          modelUsed,
          providerUsed: fallbackProvider,
          contextTokensUsed,
          systemPromptReport: runResult.meta?.systemPromptReport,
          callCount: runResult.meta?.agentMeta?.callCount,
          logLabel: "followup",
        });
```

How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: 0436875

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds per-turn API call count tracking to the embedded agent runner and surfaces it in user-facing status/usage output, complementing existing per-turn token usage reporting.

Changes:

  • Introduces callCount on embedded runner metadata and persists it onto SessionEntry.
  • Displays 📞 Calls: N in /status and in the response usage footer.
  • Adds/updates unit tests covering call count formatting in both outputs.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/config/sessions/types.ts Adds SessionEntry.callCount to persist last-run (current turn) API call count.
src/auto-reply/status.ts Includes 📞 Calls: N in the /status usage line when available.
src/auto-reply/status.test.ts Tests that /status includes call count from the latest run.
src/auto-reply/reply/session-usage.ts Persists callCount onto session store entries.
src/auto-reply/reply/agent-runner.ts Plumbs runner callCount into session persistence and usage-footer formatting.
src/auto-reply/reply/agent-runner-utils.ts Extends usage footer formatting to include call count.
src/auto-reply/reply/agent-runner-utils.test.ts Tests usage footer formatting with call count.
src/agents/pi-embedded-runner/types.ts Adds callCount to embedded runner agent meta type.
src/agents/pi-embedded-runner/run.ts Tracks and reports per-run API call count via the usage accumulator.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/auto-reply/reply/session-usage.ts
Comment thread src/agents/pi-embedded-runner/run.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0436875ad3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run.ts Outdated
Comment thread src/auto-reply/reply/session-usage.ts
hoshi_lan added 2 commits March 17, 2026 16:37
Bot review feedback: callCount was missing in followup-runner.ts
persistRunSessionUsage call, causing stale count after followup turns.
This aligns with how usage/lastCallUsage are already handled there.
- Add callCount tracking in subscribeEmbeddedPiSession (incremented on each
  recordAssistantUsage call, i.e., each LLM API response with usage data)
- Add attemptCallCount to EmbeddedRunAttemptResult
- Pass callCount from attempt to run.ts and accumulate properly
- Add tests for callCount accumulation scenarios

This fixes the issue where callCount only counted attempts, not individual
API calls within tool-call loops.
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 50b4858035

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/agents/pi-embedded-subscribe.ts
@clawsweeper
Copy link
Copy Markdown
Contributor

clawsweeper Bot commented Apr 28, 2026

Codex review: needs changes before merge.

Summary
The PR adds per-turn API call count plumbing from embedded runner metadata into session persistence, /status, response usage footer display, and related tests.

Reproducibility: yes. for review purposes. Source inspection shows current main has no call-count field/display, and the PR diff exposes the undercount and stale-persistence paths without needing a live provider run.

Next step before merge
Queue a narrow repair or replacement because the remaining blockers are concrete code/test/changelog fixes that do not require a product-policy decision.

Security
Cleared: The PR changes TypeScript accounting/display code and tests only, with no dependency, workflow, package, secret, permission, or code-execution surface changes.

Review findings

  • [P2] Count model requests without requiring usage tokens — src/agents/pi-embedded-subscribe.ts:275
  • [P2] Clear stale call counts on every persisted run — src/auto-reply/reply/session-usage.ts:101-103
  • [P3] Add release notes for the visible usage change — src/auto-reply/status.ts:664-666
Review details

Best possible solution:

Land a rebased implementation on current main that counts per-run model requests independent of token metadata, overwrites the latest stored count deterministically, shows it in status/footer, and documents the visible change.

Do we have a high-confidence way to reproduce the issue?

Yes for review purposes. Source inspection shows current main has no call-count field/display, and the PR diff exposes the undercount and stale-persistence paths without needing a live provider run.

Is this the best way to solve the issue?

No. The product direction is reasonable, but this patch should count actual model requests outside the nonzero-usage path, clear or overwrite the latest stored count, adapt to current main's formatter location, and add release notes before merge.

Full review comments:

  • [P2] Count model requests without requiring usage tokens — src/agents/pi-embedded-subscribe.ts:275
    callCount is incremented only after usage metadata is accepted, so providers or error paths that return an assistant response without nonzero usage are excluded. That breaks the advertised API-call accounting for providers where token usage is unavailable; count at the model request or assistant-response boundary and keep token aggregation conditional.
    Confidence: 0.86
  • [P2] Clear stale call counts on every persisted run — src/auto-reply/reply/session-usage.ts:101-103
    callCount is meant to describe the latest run, but this only writes it when the new value is greater than zero. If a later run has zero counted calls or a path omits the field, the merged session entry keeps the previous value and /status can report stale data.
    Confidence: 0.9
  • [P3] Add release notes for the visible usage change — src/auto-reply/status.ts:664-666
    This feature changes user-visible /status and response usage footer output, but the branch does not update CHANGELOG.md. Repo policy requires a changelog entry for user-facing feat changes.
    Confidence: 0.83

Overall correctness: patch is incorrect
Overall confidence: 0.86

Acceptance criteria:

  • pnpm test src/auto-reply/status.test.ts src/auto-reply/reply/agent-runner.misc.runreplyagent.test.ts src/auto-reply/reply/session.test.ts src/auto-reply/reply/followup-runner.test.ts src/agents/pi-embedded-runner/usage-reporting.test.ts
  • pnpm exec oxfmt --check --threads=1 src/agents/pi-embedded-subscribe.ts src/agents/pi-embedded-runner/run.ts src/agents/pi-embedded-runner/run/attempt.ts src/agents/pi-embedded-runner/run/types.ts src/agents/pi-embedded-runner/types.ts src/auto-reply/reply/session-usage.ts src/auto-reply/reply/session-run-accounting.ts src/auto-reply/reply/agent-runner.ts src/auto-reply/reply/followup-runner.ts src/auto-reply/reply/agent-runner-usage-line.ts src/status/status-message.ts src/config/sessions/types.ts CHANGELOG.md
  • pnpm check:changed

What I checked:

  • Current main status output has no call count: buildStatusMessage composes the usage line from token usage and estimated cost only; there is no callCount read or Calls suffix on current main. (src/status/status-message.ts:953, d5edeae6ee9d)
  • Current main response usage footer has no call count: formatResponseUsageLine accepts usage/cost inputs only and returns tokens, cache, and estimated cost text without an API call count parameter. (src/auto-reply/reply/agent-runner-usage-line.ts:9, d5edeae6ee9d)
  • Current main session persistence has no call-count field: persistSessionUsageUpdate has no callCount parameter or patch field, and SessionEntry has no stored call-count property. (src/auto-reply/reply/session-usage.ts:71, d5edeae6ee9d)
  • PR diff undercounts usage-less model calls: The PR increments callCount inside recordAssistantUsage only after the nonzero-usage guard, so model responses with omitted or zero usage are excluded despite the advertised API-call semantics. (src/agents/pi-embedded-subscribe.ts:275, 50b48580358c)
  • PR diff can retain stale latest-run call counts: The PR only patches callCount when the value is greater than zero, so a later run with zero or omitted call count can leave the previous run's count visible in /status. (src/auto-reply/reply/session-usage.ts:101, 50b48580358c)
  • Changelog entry is still missing: The current changelog contains no API call count or Calls: entry for this user-visible feature. (CHANGELOG.md:1, d5edeae6ee9d)

Likely related people:

  • @pgondhi987: Local blame and git show --stat tie the central current-main status, session usage, response footer, and embedded subscription files to commit 1f724bc; the changelog credits the same work to this handle. (role: current-main area owner in local history; confidence: medium; commits: 1f724bc50b4d; files: src/status/status-message.ts, src/auto-reply/reply/agent-runner-usage-line.ts, src/auto-reply/reply/session-usage.ts)

Remaining risk / open question:

  • The PR branch predates current main's response-usage formatter split into src/auto-reply/reply/agent-runner-usage-line.ts, so the repair likely needs a rebase or replacement branch rather than a direct line-for-line update.

Codex review notes: model gpt-5.5, reasoning high; reviewed against d5edeae6ee9d.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Add API call count to status/usage output

2 participants