feat: add Daily Claude Token Usage Analyzer workflow by lpcox · Pull Request #1605 · github/gh-aw-firewall

lpcox · 2026-04-02T16:38:55Z

Summary

Adds a Claude-specific token usage analyzer workflow that mirrors the existing Copilot Token Usage Analyzer. Runs daily on a schedule, analyzes Claude/Anthropic-engine workflow runs, and creates a summary issue.

Target Workflows

smoke-claude
secret-digger-claude
security-review, security-guard
Any other Claude-engine workflow with agent-artifacts

Differences from Copilot Analyzer

Feature	Copilot Analyzer	Claude Analyzer
Provider filter	`copilot`	`anthropic`
Cache write tracking	Not available (OpenAI limitation)	✅ Full cache_write_tokens
Cache write rate metric	N/A	✅ New metric
Pricing tiers	OpenAI/Copilot rates	Anthropic Sonnet/Haiku/Opus rates
Skip label	`token-usage-report`	`claude-token-usage-report`
Issue prefix	📊 Copilot Token Usage Report	📊 Claude Token Usage Report

Key Design Choices

Separate workflow (not combined) — keeps each focused and within the 15-minute timeout
Provider-filtered — only analyzes records with provider: "anthropic" from token-usage.jsonl
Cache write analysis — Anthropic uniquely exposes cache_creation_input_tokens, enabling cache write rate analysis that the Copilot analyzer cannot do
Anthropic pricing — uses Anthropic's published per-model rates including cache read/write discounts

Mirrors the existing Copilot Token Usage Analyzer but focused on Claude/Anthropic-engine workflows (smoke-claude, secret-digger-claude, security-review, security-guard). Key differences from the Copilot analyzer: - Filters for provider=anthropic in token-usage.jsonl - Includes cache_write_tokens in analysis (Anthropic exposes this) - Adds cache write rate metric - Uses Anthropic pricing tiers (Sonnet/Haiku/Opus) - Separate skip-if-match label (claude-token-usage-report) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-04-02T16:39:59Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	85.86%	85.96%	📈 +0.10%
Statements	85.75%	85.84%	📈 +0.09%
Functions	86.66%	86.66%	➡️ +0.00%
Branches	78.49%	78.55%	📈 +0.06%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/docker-manager.ts`	86.1% → 86.5% (+0.40%)	85.6% → 86.0% (+0.39%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

Adds a new daily, provider-filtered token usage reporting workflow focused on Claude/Anthropic usage, mirroring the existing Copilot token usage analyzer pattern.

Changes:

Introduces a Claude-specific token usage analyzer prompt (.md) that downloads agent-artifacts, parses token-usage.jsonl, filters provider: "anthropic", and instructs the agent to open a summary issue.
Adds the compiled gh-aw workflow (.lock.yml) that runs on a daily schedule and produces the issue via safe-outputs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
.github/workflows/claude-token-usage-analyzer.md	New analyzer prompt/instructions for collecting Anthropic-only token logs and producing a daily issue report.
.github/workflows/claude-token-usage-analyzer.lock.yml	Compiled gh-aw workflow implementing the daily scheduled run + safe-outputs issue creation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-02T16:42:08Z

.github/workflows/claude-token-usage-analyzer.md

+1. **Total tokens**: `input_tokens + output_tokens + cache_read_tokens + cache_write_tokens`
+2. **Billable tokens**: `input_tokens + output_tokens + cache_write_tokens` (cache reads are discounted)
+3. **Input/output ratio**: `(input_tokens + cache_read_tokens) / output_tokens` (if `output_tokens == 0`, treat the ratio as `∞`/`N/A` and exclude that request from ratio averages to avoid division by zero)
+4. **Cache hit rate**: `cache_read_tokens / (cache_read_tokens + input_tokens) * 100`
+5. **Cache write rate**: `cache_write_tokens / (cache_read_tokens + input_tokens + cache_write_tokens) * 100`
+6. **Request count**: Number of records in the JSONL
+7. **Average latency**: Mean `duration_ms` per request
+8. **Model distribution**: Count of requests per model
+9. **Estimated cost** (use approximate rates):
+   - Anthropic Sonnet: input $3/M, output $15/M, cache_read $0.30/M, cache_write $3.75/M
+   - Anthropic Haiku: input $0.80/M, output $4/M, cache_read $0.08/M, cache_write $1/M
+   - Anthropic Opus: input $15/M, output $75/M, cache_read $1.50/M, cache_write $18.75/M


The prompt defines Billable tokens as input_tokens + output_tokens + cache_write_tokens, but later the estimated cost section assigns a non-zero rate to cache_read_tokens (discounted, not free). As written, the report will undercount billable usage for runs with cache hits. Consider either (a) including cache_read_tokens in the billable token definition (possibly with a discount factor), or (b) renaming this metric to something like “full-price tokens” and clearly separating it from discounted cache-read billing.

github-actions · 2026-04-02T16:43:54Z

Smoke Test Results — Copilot Engine

Test	Result
GitHub MCP (last 2 merged PRs)	✅ #1603: fix: extract OpenAI/Copilot cached_tokens · #1593: fix: capture full session state
Playwright (github.com title)	✅ "GitHub · Change is constant…"
File write	✅ `/tmp/gh-aw/agent/smoke-test-copilot-23911242767.txt` created
Bash verify	✅ File read back successfully

Overall: PASS · PR by @lpcox · No assignees

📰 BREAKING: Report filed by Smoke Copilot for issue #1605

github-actions · 2026-04-02T16:44:00Z

Smoke Test Results — Run 23911242765

✅ GitHub MCP — Last 2 merged PRs: #1603 fix: extract OpenAI/Copilot cached_tokens, #1593 fix: capture full session state
✅ Playwright — github.com title contains "GitHub"
✅ File Write — /tmp/gh-aw/agent/smoke-test-claude-23911242765.txt created
✅ Bash — File contents verified

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude for issue #1605

lpcox · 2026-04-02T16:46:31Z

@copilot apply changes based on the comments in this thread

… section Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/1ea1df07-9ddd-4b4d-a83e-db6ac24214bd Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot · 2026-04-02T16:48:10Z

@copilot apply changes based on the comments in this thread

Done in 76241ed. Renamed "Billable tokens" → "Full-price tokens" (with a note pointing to the Estimated cost section for the true billed amount), and updated the Estimated cost description to explicitly say "cache reads are discounted, not free" — so both sections are now consistent and no cache-read usage is silently dropped.

github-actions · 2026-04-02T17:01:28Z

Smoke test report (run 23911616574)
PR titles: "fix: extract OpenAI/Copilot cached_tokens from prompt_tokens_details"; "fix: capture full session state — replace blanket ~/.copilot mount, add --session-state-dir"
GitHub MCP last 2 merged PRs: ✅
safeinputs-gh pr list query: ❌ (tool unavailable)
Playwright title contains "GitHub": ❌ (Playwright MCP EACCES)
Tavily web search returned results: ❌ (tool unavailable)
File write/read test (/tmp/gh-aw/agent/smoke-test-codex-23911616574.txt): ✅
Discussion oracle comment: ❌ (discussion query tool unavailable)
Build (npm ci && npm run build): ✅
Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-04-02T17:01:35Z

Smoke Test: GitHub Actions Services Connectivity

Check	Status
Redis PING (`host.docker.internal:6379`)	✅ PONG
PostgreSQL `pg_isready` (`host.docker.internal:5432`)	✅ accepting connections
PostgreSQL `SELECT 1` (db: `smoketest`, user: `postgres`)	✅ returned `1`

All 3 checks passed. (redis-cli was not installed; Redis was verified via raw TCP socket — port open, PING→PONG confirmed.)

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-04-02T17:01:36Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #1605 · ◷

lpcox requested a review from Mossaka as a code owner April 2, 2026 16:38

Copilot AI review requested due to automatic review settings April 2, 2026 16:38

Copilot started reviewing on behalf of lpcox April 2, 2026 16:39 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

github-actions bot added the smoke-copilot label Apr 2, 2026

github-actions bot added the smoke-claude label Apr 2, 2026

This comment has been minimized.

Sign in to view

github-actions bot added the smoke-services label Apr 2, 2026

This comment has been minimized.

Sign in to view

github-actions bot added the build-test label Apr 2, 2026

github-actions bot mentioned this pull request Apr 2, 2026

[agentics] No-Op Runs #1580

Closed

Copilot started work on behalf of lpcox April 2, 2026 16:46 View session

fix: rename 'Billable tokens' to 'Full-price tokens' and clarify cost…

76241ed

… section Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/1ea1df07-9ddd-4b4d-a83e-db6ac24214bd Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot finished work on behalf of lpcox April 2, 2026 16:48

lpcox merged commit 5ed84e4 into main Apr 2, 2026
59 of 61 checks passed

lpcox deleted the feat/claude-token-usage-analyzer branch April 2, 2026 17:17

This was referenced Apr 2, 2026

perf: reduce smoke-copilot token usage with pre-steps and tool trimming #1613

Merged

fix: fast-kill agent container on SIGTERM/SIGINT before full cleanup #1623

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add Daily Claude Token Usage Analyzer workflow#1605

feat: add Daily Claude Token Usage Analyzer workflow#1605
lpcox merged 2 commits intomainfrom
feat/claude-token-usage-analyzer

lpcox commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

lpcox commented Apr 2, 2026

Uh oh!

Copilot AI commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

lpcox commented Apr 2, 2026

Summary

Target Workflows

Differences from Copilot Analyzer

Key Design Choices

Uh oh!

github-actions bot commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 2, 2026

Smoke Test Results — Copilot Engine

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

lpcox commented Apr 2, 2026

Uh oh!

Copilot AI commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Smoke Test: GitHub Actions Services Connectivity

Uh oh!

github-actions bot commented Apr 2, 2026

🏗️ Build Test Suite Results

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Apr 2, 2026 •

edited

Loading