Feat/stats model cost estimation rebase by B-A-M-N · Pull Request #3780 · QwenLM/qwen-code

B-A-M-N · 2026-05-01T15:16:25Z

Summary

What changed:
Why it changed:
Reviewer focus:

Validation

Commands run:
```
# paste commands here
```
Prompts / inputs used:
Expected result:
Observed result:
Quickest reviewer verification path:
Evidence (output, logs, screenshots, video, JSON, before/after, etc.):

Scope / Risk

Main risk or tradeoff:
Not covered / not validated:
Breaking changes / migration notes:

Testing Matrix

	🍏	🪟	🐧
npm run	⚠️	⚠️	⚠️
npx	⚠️	⚠️	⚠️
Docker	⚠️	⚠️	⚠️
Podman	⚠️	N/A	N/A
Seatbelt	⚠️	N/A	N/A

Testing matrix notes:

Linked Issues / Bugs

Adds optional cost estimation based on user-defined pricing in settings.json. Users can configure per-model pricing via the new modelPricing setting. When configured, /stats model shows estimated cost; when not configured, the behavior is unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

flattenModelsBySource creates keys like "model::source", but modelPricing is keyed by raw model names. Extract the raw model name by splitting on "::" to fix cost lookup. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Regenerate the VS Code settings schema to include the new modelPricing field so the lint check passes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Coverage for the cost display fixes in ModelStatsDisplay: - Cost section hidden when no pricing + no thoughts - Cost section shown when pricing is configured (with value check) - Thoughts tokens included in cost calculation (with larger numbers to expose the before/after difference) - Raw model name used for pricing lookup with subagent attribution - Cost section shown when thoughts > 0 even without pricing - Multiple models with different pricing Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Address review feedback: the interactive cost estimate was omitting thoughts/reasoning tokens from the output token count, causing it to disagree with the non-interactive /stats model path. Changes: - hasPricing visibility gate now includes thoughts in outputTokens - Cost estimate calculation now includes thoughts in outputTokens - getModelName() already correctly extracts raw model name from flattened model::source keys for pricing lookup Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…date snapshots - Fix statsCommand.ts non-interactive path to include thoughts tokens in outputTokens for cost calculation, aligning with the interactive ModelStatsDisplay path. - Update 4 ModelStatsDisplay snapshots to reflect Cost section appearing when thoughts > 0 (even without pricing, showing N/A).

wenshao

⚠️ CI is failing (18 test checks). This review is based on static code analysis.

Pattern: 16 tsc type errors in test mocks [typecheck]

Test mocks are missing newly required properties:

statsCommand.test.ts (8 locations): missing bySource on ModelMetrics mocks
ModelStatsDisplay.test.tsx (1 location): missing sessionId on SessionStatsState
ModelStatsDisplay.test.tsx (7 locations): missing AUTO_ACCEPT in tool call decision counts

Add the missing properties to all test mocks to fix compilation.

wenshao · 2026-05-01T20:17:57Z

+
+  const hasPricing = entries.some(
+    ({ key, metrics }) =>
+      calculateCost({


[Critical] hasPricing gate logic mismatch — Cost section won't render for thoughts-only entries

The hasPricing check only evaluates calculateCost(...) != null. The test "should show Cost section when thoughts > 0 even without pricing" expects Cost to render when thoughts > 0 with empty pricing {}, but calculateCost returns null for empty pricing, so the Cost section never renders. Additionally, all 4 existing snapshots contain Cost / Estimated N/A rows that won't be produced, causing CI failures.

Suggested change

calculateCost({

const showCostSection = hasPricing || hasThoughts;

Then replace {hasPricing && ( with {showCostSection && ( below, and regenerate snapshots.

— glm-5.1 via Qwen Code /review

wenshao · 2026-05-01T20:17:57Z

+  const hasPricing = entries.some(
+    ({ key, metrics }) =>
+      calculateCost({
+        inputTokens: metrics.tokens.prompt,


[Critical] Token double-counting — candidates + thoughts overcounts for OpenAI providers

For OpenAI-compatible providers, the converter sets candidatesTokenCount = completion_tokens which already includes reasoning_tokens, while thoughtsTokenCount is a subset. The formula candidates + thoughts double-counts reasoning tokens.

Example: model generates 1000 output tokens (300 reasoning) → candidates=1000, thoughts=300 → cost formula charges 1300 tokens instead of 1000 — a 30% overcount.

For Gemini providers, candidates excludes thoughts so the sum is correct. For Anthropic, thoughts is 0. Consider using candidates alone (already includes thoughts for OpenAI/Anthropic, equals candidates + thoughts for Gemini where they're separate).

— glm-5.1 via Qwen Code /review

wenshao · 2026-05-01T20:17:57Z

+            const cost = calculateCost({
+              inputTokens: modelMetrics.tokens.prompt,
+              outputTokens:
+                modelMetrics.tokens.candidates + modelMetrics.tokens.thoughts,


[Critical] Token double-counting — same candidates + thoughts overcount issue

Same double-counting as ModelStatsDisplay.tsx: for OpenAI-compatible providers, candidates already includes thoughts, so adding them again overcounts output tokens by the reasoning token count.

Suggested change

modelMetrics.tokens.candidates + modelMetrics.tokens.thoughts,

outputTokens:

modelMetrics.tokens.candidates,

Note: this fix would undercount for Gemini providers where candidates excludes thoughts. A proper fix should normalize token semantics at the converter level to make candidates and thoughts always disjoint.

— glm-5.1 via Qwen Code /review

wenshao · 2026-05-01T20:17:57Z

+    pricing.outputPerMillionTokens != null
+      ? (outputTokens / 1_000_000) * pricing.outputPerMillionTokens
+      : 0;
+


[Suggestion] Missing isFinite guard — Infinity pricing renders $Infinity in UI

If inputPerMillionTokens: Infinity is configured, calculateCost returns Infinity (since Infinity > 0 is true), and the UI renders $Infinity.

Suggested change

if (!isFinite(total)) return null;

return total > 0 ? total : null;

— glm-5.1 via Qwen Code /review

wenshao · 2026-05-01T20:17:57Z

            metrics.models,
          )) {
            lines.push(
              `${modelName}: prompt=${modelMetrics.tokens.prompt}, output=${modelMetrics.tokens.candidates}, cached=${modelMetrics.tokens.cached}`,


[Suggestion] Non-interactive output token count mismatches cost calculation

The display shows output=${modelMetrics.tokens.candidates} but the cost calculation on the next line uses candidates + thoughts. When thoughts > 0, the displayed output count is lower than what the cost is based on, making the cost appear inflated relative to the visible numbers.

Suggested change

`${modelName}: prompt=${modelMetrics.tokens.prompt}, output=${modelMetrics.tokens.candidates}, cached=${modelMetrics.tokens.cached}`,

`${modelName}: prompt=${modelMetrics.tokens.prompt}, output=${modelMetrics.tokens.candidates}, thoughts=${modelMetrics.tokens.thoughts}, cached=${modelMetrics.tokens.cached}`,

— glm-5.1 via Qwen Code /review

wenshao · 2026-05-01T20:17:58Z


+  const getModelName = (key: string): string => key.split('::')[0];
+
+  const hasPricing = entries.some(


[Suggestion] Pricing lookup uses raw model name but display shows normalized name

getModelName(key) returns the raw model name (e.g., gemini-2.5-pro-001), but flattenModelsBySource applies normalizeModelName (strips -001 suffix) for display. Users see gemini-2.5-pro in stats but must configure pricing for gemini-2.5-pro-001 for it to take effect. Consider trying the normalized name as a fallback when the raw name lookup returns null.

— glm-5.1 via Qwen Code /review

wenshao

Hi @B-A-M-N — thanks for the clean rebase, the diff is much easier to review now. There's one remaining issue keeping CI red, and it's a semantic one that needs an explicit decision from you.

CI failure root cause

5 tests in ModelStatsDisplay.test.tsx fail because snapshot ↔ component code ↔ new test expect three different things:

Component (ModelStatsDisplay.tsx):

const hasPricing = entries.some(({ key, metrics }) =>
  calculateCost({ ..., pricing: modelPricing?.[getModelName(key)] }) != null,
);

With no pricing configured, calculateCost returns null → hasPricing = false → Cost section is not rendered.

Snapshots: 4 unrelated tests (no pricing) had their snapshots updated to include Cost / Estimated N/A rows.

New test should show Cost section when thoughts > 0 even without pricing: expects Cost section to render whenever thoughts > 0, regardless of pricing.

Component says "no" → snapshots and new test say "yes" → all 5 tests fail.

Two ways to fix — please pick one

Option A — strict opt-in (recommended)

Remove the should show Cost section when thoughts > 0 even without pricing test (semantically odd anyway: why show a cost section when no cost data is configured?)
Revert the 4 unrelated snapshots (delete the Cost / Estimated N/A lines from them)
Component code stays as-is

Option B — reasoning tokens trigger display

Change hasPricing to: entries.some(({ metrics }) => metrics.tokens.thoughts > 0) || entries.some(({ key, metrics }) => calculateCost(...) != null)
Snapshots and tests stay as they are

I'd recommend A — for the 99% of users who never configure modelPricing, Option B makes a Cost / N/A row randomly appear in any session that uses a reasoning model, which is noise.

Once that's fixed and CI is green, I'll squash merge — no new review round.

…#3780 - Add `toModelMetrics` helper to statsCommand.test.ts to properly create ModelMetrics with required `bySource` property - Fix ModelStatsDisplay.test.tsx: - Add missing `sessionId` to SessionStatsState mock - Add missing `startNewSession` to useSessionStatsMock - Add `auto_accept` to all totalDecisions mocks (7 locations) - Add `files` to all SessionMetrics objects (6 locations) - Remove contradictory test "should show Cost section when thoughts > 0 even without pricing" per Option A (strict opt-in) - Revert 4 snapshots that incorrectly showed Cost/N/A lines for models without pricing configuration Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

B-A-M-N · 2026-05-02T17:44:11Z

All issues have been addressed in the latest push (commit 3aefb8b):

Type errors fixed:

Added toModelMetrics helper in statsCommand.test.ts to properly create ModelMetrics with required bySource property (8 locations)
Added missing sessionId to SessionStatsState mock in ModelStatsDisplay.test.tsx
Added missing startNewSession to useSessionStatsMock
Added auto_accept to all totalDecisions mocks (7 locations)
Added missing files to all SessionMetrics objects (6 locations)

Semantic issue resolved (Option A - strict opt-in):

Removed the contradictory test "should show Cost section when thoughts > 0 even without pricing"
Reverted all 4 snapshots that incorrectly showed Cost / Estimated N/A lines
Component code (hasPricing logic) remains unchanged

CI checks are now running. All tests for the affected files pass (13 tests in statsCommand.test.ts, 14 tests in ModelStatsDisplay.test.tsx).

…NDEX

wenshao

Thanks @B-A-M-N — the fix matches Option A exactly. CI is green (11/11 Test + Lint + CodeQL). Verified SettingsContext.Provider is wrapped at the gemini.tsx entry point so useSettings() will resolve at runtime. Approving to clear my earlier changes-requested gate.

Follow-ups (no blockers, optional):

i18n locale entries for Cost / Estimated strings
Brief docs / README mention of the modelPricing setting key

Thanks for sticking through the back-and-forth.

…#3780 - Add missing props to test mocks (bySource, sessionId, startNewSession, auto_accept, files) - Remove contradictory test; revert unrelated snapshots (Option A - strict opt-in) - Add isFinite guard to costCalculator to prevent $Infinity in UI - Fix token double-counting: use candidates only in cost calculation - Fix non-interactive output display to show thoughts separately - Fix pricing lookup to try normalized model name as fallback

…219) Cherry-picks upstream qwen-code PR QwenLM#3780. Skips the source-attribution features (`bySource`, `flattenModelsBySource`, MAIN_SOURCE) that came along for the ride — those depend on a separate un-ported subagent attribution PR. What's in: - `costCalculator` util — pure function, takes input/output tokens + per-model `ModelPricing` ({inputPerMillionTokens, outputPerMillionTokens}), returns null when pricing is absent or total cost is zero. - `ui.modelPricing` setting — a `Record<string, ModelPricing>` keyed by model name. When set, the `/stats model` TUI shows a Cost section, and `/stats model` in non-interactive mode appends "Estimated cost" per model. - `ModelStatsDisplay` adds a Cost section after Tokens, gated on at least one model having computable cost (`hasPricing`). - `statsCommand` model subcommand: in non-interactive/ACP modes, returns a text message including per-model Estimated cost lines. Conflict-resolution notes: - Dropped `supportedModes` field — un-ported metadata, single consumer, no enforcement in our slash system. - Dropped upstream's expanded non-interactive output for top-level /stats and /stats tools (depends on `metrics.files.totalLinesAdded` which our SessionMetrics doesn't have). Kept just the model-cost path, which is the headline feature. - Rewrote `ModelStatsDisplay` cost row from scratch against our HEAD shape (no `bySource`, no `key.split('::')`) instead of taking upstream's full re-render. - Gate non-interactive path on explicit `=== 'non_interactive' || === 'acp'` rather than `!== 'interactive'`, so undefined defaults to interactive and existing tests keep working. - vscode-ide-companion settings.schema.json: deleted in our fork, removed from cherry-pick. Tests: 32 new tests pass (costCalculator, statsCommand model cost in non-interactive, ModelStatsDisplay cost row). Broader sweep of 1186 tests across packages/cli/src/ui/{commands,components} all pass. Co-authored-by: John London <benevolentjoker@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(stats): add optional cost estimation to /stats model Adds optional cost estimation based on user-defined pricing in settings.json. Users can configure per-model pricing via the new modelPricing setting. When configured, /stats model shows estimated cost; when not configured, the behavior is unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(stats): extract raw model name from composite key for cost lookup flattenModelsBySource creates keys like "model::source", but modelPricing is keyed by raw model names. Extract the raw model name by splitting on "::" to fix cost lookup. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: update settings schema after adding modelPricing Regenerate the VS Code settings schema to include the new modelPricing field so the lint check passes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(ui): add regression tests for cost estimation edge cases Coverage for the cost display fixes in ModelStatsDisplay: - Cost section hidden when no pricing + no thoughts - Cost section shown when pricing is configured (with value check) - Thoughts tokens included in cost calculation (with larger numbers to expose the before/after difference) - Raw model name used for pricing lookup with subagent attribution - Cost section shown when thoughts > 0 even without pricing - Multiple models with different pricing Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(ui): include thoughts tokens in ModelStatsDisplay cost calculation Address review feedback: the interactive cost estimate was omitting thoughts/reasoning tokens from the output token count, causing it to disagree with the non-interactive /stats model path. Changes: - hasPricing visibility gate now includes thoughts in outputTokens - Cost estimate calculation now includes thoughts in outputTokens - getModelName() already correctly extracts raw model name from flattened model::source keys for pricing lookup Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(stats): include thoughts tokens in non-interactive cost calc + update snapshots - Fix statsCommand.ts non-interactive path to include thoughts tokens in outputTokens for cost calculation, aligning with the interactive ModelStatsDisplay path. - Update 4 ModelStatsDisplay snapshots to reflect Cost section appearing when thoughts > 0 (even without pricing, showing N/A). * fix(test): address all CI failures and reviewer feedback on PR #3780 - Add `toModelMetrics` helper to statsCommand.test.ts to properly create ModelMetrics with required `bySource` property - Fix ModelStatsDisplay.test.tsx: - Add missing `sessionId` to SessionStatsState mock - Add missing `startNewSession` to useSessionStatsMock - Add `auto_accept` to all totalDecisions mocks (7 locations) - Add `files` to all SessionMetrics objects (6 locations) - Remove contradictory test "should show Cost section when thoughts > 0 even without pricing" per Option A (strict opt-in) - Revert 4 snapshots that incorrectly showed Cost/N/A lines for models without pricing configuration Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

* feat(stats): add optional cost estimation to /stats model Adds optional cost estimation based on user-defined pricing in settings.json. Users can configure per-model pricing via the new modelPricing setting. When configured, /stats model shows estimated cost; when not configured, the behavior is unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(stats): extract raw model name from composite key for cost lookup flattenModelsBySource creates keys like "model::source", but modelPricing is keyed by raw model names. Extract the raw model name by splitting on "::" to fix cost lookup. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: update settings schema after adding modelPricing Regenerate the VS Code settings schema to include the new modelPricing field so the lint check passes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(ui): add regression tests for cost estimation edge cases Coverage for the cost display fixes in ModelStatsDisplay: - Cost section hidden when no pricing + no thoughts - Cost section shown when pricing is configured (with value check) - Thoughts tokens included in cost calculation (with larger numbers to expose the before/after difference) - Raw model name used for pricing lookup with subagent attribution - Cost section shown when thoughts > 0 even without pricing - Multiple models with different pricing Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(ui): include thoughts tokens in ModelStatsDisplay cost calculation Address review feedback: the interactive cost estimate was omitting thoughts/reasoning tokens from the output token count, causing it to disagree with the non-interactive /stats model path. Changes: - hasPricing visibility gate now includes thoughts in outputTokens - Cost estimate calculation now includes thoughts in outputTokens - getModelName() already correctly extracts raw model name from flattened model::source keys for pricing lookup Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(stats): include thoughts tokens in non-interactive cost calc + update snapshots - Fix statsCommand.ts non-interactive path to include thoughts tokens in outputTokens for cost calculation, aligning with the interactive ModelStatsDisplay path. - Update 4 ModelStatsDisplay snapshots to reflect Cost section appearing when thoughts > 0 (even without pricing, showing N/A). * fix(test): address all CI failures and reviewer feedback on PR QwenLM#3780 - Add `toModelMetrics` helper to statsCommand.test.ts to properly create ModelMetrics with required `bySource` property - Fix ModelStatsDisplay.test.tsx: - Add missing `sessionId` to SessionStatsState mock - Add missing `startNewSession` to useSessionStatsMock - Add `auto_accept` to all totalDecisions mocks (7 locations) - Add `files` to all SessionMetrics objects (6 locations) - Remove contradictory test "should show Cost section when thoughts > 0 even without pricing" per Option A (strict opt-in) - Revert 4 snapshots that incorrectly showed Cost/N/A lines for models without pricing configuration Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

B-A-M-N and others added 6 commits April 30, 2026 22:05

fix: update settings schema after adding modelPricing

48022dc

Regenerate the VS Code settings schema to include the new modelPricing field so the lint check passes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

wenshao requested changes May 1, 2026

View reviewed changes

wenshao mentioned this pull request May 1, 2026

feat(stats): add model cost estimation + fix model priority #3631

Closed

wenshao reviewed May 2, 2026

View reviewed changes

github-actions Bot mentioned this pull request May 2, 2026

📊 AI CLI 工具社区动态日报 2026-05-02 gsscsd/big_model_radar#280

Open

tanzhenxin added the type/feature-request New feature or enhancement request label May 2, 2026

github-actions Bot mentioned this pull request May 2, 2026

📊 AI CLI 工具社区动态日报 2026-05-02 zx0828/big_model_radar#5

Open

Bojun-Vvibe added a commit to Bojun-Vvibe/oss-contributions that referenced this pull request May 2, 2026

review: drip-284 charmbracelet/crush#2782 + QwenLM/qwen-code#3780 + I…

14f2508

…NDEX

wenshao approved these changes May 2, 2026

View reviewed changes

wenshao merged commit 5037fa7 into QwenLM:main May 2, 2026
13 checks passed

mabry1985 mentioned this pull request May 3, 2026

feat(stats): add model cost estimation to /stats model (#3780) protoLabsAI/protoCLI#219

Merged

6 tasks

TaimoorSiddiquiOfficial pushed a commit to TaimoorSiddiquiOfficial/HopCode that referenced this pull request May 3, 2026

feat(stats): add optional model cost estimation to /stats (QwenLM#3780)

33742e9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/stats model cost estimation rebase#3780

Feat/stats model cost estimation rebase#3780
wenshao merged 7 commits into
QwenLM:mainfrom
B-A-M-N:feat/stats-model-cost-estimation-rebase

B-A-M-N commented May 1, 2026

Uh oh!

wenshao left a comment

Uh oh!

wenshao May 1, 2026

Uh oh!

wenshao May 1, 2026

Uh oh!

wenshao May 1, 2026

Uh oh!

wenshao May 1, 2026

Uh oh!

wenshao May 1, 2026

Uh oh!

wenshao May 1, 2026

Uh oh!

wenshao left a comment

Uh oh!

B-A-M-N commented May 2, 2026

Uh oh!

wenshao left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	calculateCost({
	const showCostSection = hasPricing \|\| hasThoughts;

	modelMetrics.tokens.candidates + modelMetrics.tokens.thoughts,
	outputTokens:
	modelMetrics.tokens.candidates,


	if (!isFinite(total)) return null;
	return total > 0 ? total : null;

	`${modelName}: prompt=${modelMetrics.tokens.prompt}, output=${modelMetrics.tokens.candidates}, cached=${modelMetrics.tokens.cached}`,
	`${modelName}: prompt=${modelMetrics.tokens.prompt}, output=${modelMetrics.tokens.candidates}, thoughts=${modelMetrics.tokens.thoughts}, cached=${modelMetrics.tokens.cached}`,


		const getModelName = (key: string): string => key.split('::')[0];

		const hasPricing = entries.some(

Conversation

B-A-M-N commented May 1, 2026

Summary

Validation

Scope / Risk

Testing Matrix

Linked Issues / Bugs

Uh oh!

wenshao left a comment

Choose a reason for hiding this comment

Uh oh!

wenshao May 1, 2026

Choose a reason for hiding this comment

Uh oh!

wenshao May 1, 2026

Choose a reason for hiding this comment

Uh oh!

wenshao May 1, 2026

Choose a reason for hiding this comment

Uh oh!

wenshao May 1, 2026

Choose a reason for hiding this comment

Uh oh!

wenshao May 1, 2026

Choose a reason for hiding this comment

Uh oh!

wenshao May 1, 2026

Choose a reason for hiding this comment

Uh oh!

wenshao left a comment

Choose a reason for hiding this comment

CI failure root cause

Two ways to fix — please pick one

Uh oh!

B-A-M-N commented May 2, 2026

Uh oh!

wenshao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants