feat: add daily token usage analyzer workflow by lpcox · Pull Request #1557 · github/gh-aw-firewall

lpcox · 2026-04-01T19:29:01Z

Summary

Adds a daily agentic workflow that mines token-usage.jsonl from recent workflow runs, computes per-workflow statistics, identifies optimization opportunities, and creates a summary issue.

What it does

Discovers completed workflow runs from the past 24 hours
Downloads agent-artifacts and extracts token-usage.jsonl
Computes per-workflow stats: total tokens, cache hit rate, I/O ratio, cost estimate, model mix
Flags inefficiencies: zero cache hits, high I/O ratios, increasing costs
Compares with previous reports for historical trend analysis
Creates a structured issue with progressive disclosure (<details> blocks)

Configuration

Setting	Value
Schedule	Daily
Engine	Copilot (default)
Timeout	15 minutes
Safe outputs	`create-issue` with `📊 Token Usage Report` prefix
Skip logic	Skips if an open token usage report issue exists
Tools	GitHub MCP (default + actions), bash

Graceful Degradation

Token tracking is new (PR #1539), so the workflow:

Skips runs without agent-artifacts
Skips runs without token-usage.jsonl
Reports which workflows are not yet instrumented
Creates a minimal report if no data is available

Files

.github/workflows/token-usage-analyzer.md — Workflow definition
.github/workflows/token-usage-analyzer.lock.yml — Compiled lock file
.github/aw/actions-lock.json — Updated action SHA lock

Example Output

The issue will contain a summary table like:

Workflow	Runs	Total Tokens	Cost	Cache Rate	I/O Ratio
smoke-claude	2	395K	$0.46	99.5%	0.6:1
smoke-copilot	2	603K	$1.20	0%	184:1

Plus optimization recommendations and per-workflow detail sections.

Closes #1551

Adds an agentic workflow that runs daily to mine token-usage.jsonl from recent workflow runs, compute per-workflow statistics (tokens, cache rates, costs, model mix), identify optimization opportunities, and create a summary issue. The workflow: - Discovers completed runs from the past 24 hours - Downloads agent-artifacts and extracts token-usage.jsonl - Computes per-workflow stats (cache hit rate, I/O ratio, cost) - Flags inefficiencies (zero cache hits, high I/O ratios) - Compares with previous reports for historical trends - Creates a structured issue with progressive disclosure Gracefully handles missing data since token tracking is a new feature. Closes #1551 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-04-01T19:30:03Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	82.67%	82.77%	📈 +0.10%
Statements	82.34%	82.43%	📈 +0.09%
Functions	81.22%	81.22%	➡️ +0.00%
Branches	75.94%	76.00%	📈 +0.06%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/docker-manager.ts`	85.8% → 86.2% (+0.41%)	85.3% → 85.7% (+0.40%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

Adds a new scheduled agentic workflow to analyze token-usage.jsonl artifacts from recent GitHub Actions runs, compute per-workflow token/cost/cache metrics, and publish a daily “Token Usage Report” issue.

Changes:

Introduces .github/workflows/token-usage-analyzer.md defining the analyzer agent prompt, safe outputs, and skip logic.
Adds the compiled workflow .github/workflows/token-usage-analyzer.lock.yml.
Updates .github/aw/actions-lock.json to lock github/gh-aw-actions/setup@v0.65.3.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
.github/workflows/token-usage-analyzer.md	New workflow manifest/prompt describing discovery, artifact download, aggregation, and report issue structure.
.github/workflows/token-usage-analyzer.lock.yml	Generated compiled workflow to execute the agent and safe-outputs issue creation.
.github/aw/actions-lock.json	Adds the pinned SHA entry for the newer `gh-aw-actions/setup` version used by the compiled workflow.

Comments suppressed due to low confidence (1)

.github/workflows/token-usage-analyzer.md:163

The example Workflow Summary table inside the suggested issue body also uses || at the start of each row, which will render with a blank first column. Use standard GFM table syntax (| Workflow | ... |) so the report issue displays as intended.

### Workflow Summary

| Workflow | Runs | Total Tokens | Cost | Cache Rate | I/O Ratio | Top Model |
|----------|------|-------------|------|------------|-----------|-----------|
| smoke-claude | 2 | 395K | $0.46 | 99.5% | 0.6:1 | sonnet-4.6 |
| smoke-copilot | 2 | 603K | $1.20 | 0% | 184:1 | gpt-4o |
| ... | | | | | | |

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.github/workflows/token-usage-analyzer.md

github-actions · 2026-04-01T19:33:57Z

Smoke Test Results — Run 23866856394

✅ GitHub MCP — #1553 [WIP] Create daily token usage analysis workflow, #1552 [WIP] Fix NODE_EXTRA_CA_CERTS path issue after chroot for SSL bump
✅ Playwright — github.com title contains "GitHub"
✅ File Write — /tmp/gh-aw/agent/smoke-test-claude-23866856394.txt created and verified
✅ Bash — file contents confirmed

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude for issue #1557

github-actions · 2026-04-01T19:34:27Z

Smoke Test Results — @lpcox

✅ GitHub MCP: Last 2 merged PRs fetched (#1553 "[WIP] Create daily token usage analysis workflow", #1552 "[WIP] Fix NODE_EXTRA_CA_CERTS path issue after chroot for SSL bump")
✅ Playwright: github.com title contains "GitHub"
✅ File Write: /tmp/gh-aw/agent/smoke-test-copilot-23866856379.txt created
✅ Bash: File verified via cat

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot for issue #1557

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

github-actions · 2026-04-01T20:01:05Z

🤖 Smoke test results for Copilot engine — PR #1557 (@lpcox, no assignees)

Test	Result
GitHub MCP (last 2 merged PRs)	✅ #1555 "fix: copy AWF CA cert to chroot-accessible path for ssl-bump", #1550 "fix: decompress gzip responses for Anthropic token extraction"
Playwright (github.com title)	✅ Title contains "GitHub"
File write	✅ `/tmp/gh-aw/agent/smoke-test-copilot-23867957427.txt` created
Bash verify (`cat`)	✅ File contents confirmed

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot for issue #1557

github-actions · 2026-04-01T20:01:25Z

Smoke Test Results — PASS

✅ GitHub MCP: fix: copy AWF CA cert to chroot-accessible path for ssl-bump #1555 fix: copy AWF CA cert to chroot-accessible path for ssl-bump, fix: decompress gzip responses for Anthropic token extraction #1550 fix: decompress gzip responses for Anthropic token extraction
✅ Playwright: github.com title contains "GitHub"
✅ File write: /tmp/gh-aw/agent/smoke-test-claude-23867957467.txt created
✅ Bash verify: file contents confirmed

💥 [THE END] — Illustrated by Smoke Claude for issue #1557

github-actions · 2026-04-01T20:03:44Z

Smoke Test: GitHub Actions Services Connectivity ✅

All checks passed:

Check	Result
Redis PING `host.docker.internal:6379`	✅ `PONG`
PostgreSQL `pg_isready` `host.docker.internal:5432`	✅ accepting connections
`psql SELECT 1` on `smoketest` db	✅ returned `1`

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-04-01T20:04:06Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	passed	✅ PASS
Go	env	✅	passed	✅ PASS
Go	uuid	✅	passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	passed	✅ PASS
Node.js	execa	✅	passed	✅ PASS
Node.js	p-limit	✅	passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #1557 · ◷

github-actions · 2026-04-01T20:04:39Z

PR titles:

fix: copy AWF CA cert to chroot-accessible path for ssl-bump
fix: decompress gzip responses for Anthropic token extraction
feat: add daily token usage analyzer workflow
fix: add node ecosystem to smoke-codex network allow-list

🔮 The oracle has spoken through Smoke Codex

lpcox requested a review from Mossaka as a code owner April 1, 2026 19:29

Copilot AI review requested due to automatic review settings April 1, 2026 19:29

Copilot started reviewing on behalf of lpcox April 1, 2026 19:29 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

github-actions bot added the smoke-claude label Apr 1, 2026

github-actions bot added the smoke-copilot label Apr 1, 2026

This comment has been minimized.

Sign in to view

github-actions bot added the smoke-services label Apr 1, 2026

github-actions bot mentioned this pull request Apr 1, 2026

[agentics] No-Op Runs #1556

Closed

This comment has been minimized.

Sign in to view

github-actions bot added the build-test label Apr 1, 2026

lpcox and others added 4 commits April 1, 2026 12:55

Update .github/workflows/token-usage-analyzer.md

ad8f657

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update .github/workflows/token-usage-analyzer.md

b7d42a7

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update .github/workflows/token-usage-analyzer.md

ebe0348

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update .github/workflows/token-usage-analyzer.md

cac6402

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

lpcox merged commit 792505c into main Apr 1, 2026
60 of 61 checks passed

lpcox deleted the feat/token-usage-analyzer-workflow branch April 1, 2026 21:32

github-actions bot mentioned this pull request Apr 1, 2026

feat: add esbuild single-file bundle as lightweight distribution #1581

Merged

7 tasks

lpcox mentioned this pull request Apr 1, 2026

fix: recompile token-usage-analyzer lock file #1586

Merged

github-actions bot mentioned this pull request Apr 1, 2026

fix: rename and scope token analyzer to Copilot workflows #1588

Merged

Conversation

lpcox commented Apr 1, 2026

Summary

What it does

Configuration

Graceful Degradation

Files

Example Output

Uh oh!

github-actions bot commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Apr 1, 2026

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

github-actions bot commented Apr 1, 2026

Smoke Test: GitHub Actions Services Connectivity ✅

Uh oh!

github-actions bot commented Apr 1, 2026

🏗️ Build Test Suite Results

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Apr 1, 2026 •

edited

Loading