Conversation
Adds an agentic workflow that runs daily to mine token-usage.jsonl from recent workflow runs, compute per-workflow statistics (tokens, cache rates, costs, model mix), identify optimization opportunities, and create a summary issue. The workflow: - Discovers completed runs from the past 24 hours - Downloads agent-artifacts and extracts token-usage.jsonl - Computes per-workflow stats (cache hit rate, I/O ratio, cost) - Flags inefficiencies (zero cache hits, high I/O ratios) - Compares with previous reports for historical trends - Creates a structured issue with progressive disclosure Gracefully handles missing data since token tracking is a new feature. Closes #1551 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
The .md file was updated via GitHub web UI but the lock file was not recompiled, causing frontmatter hash mismatch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a new scheduled Agentic Workflow (gh-aw) to analyze AWF api-proxy token usage logs across recent workflow runs and publish a daily “Token Usage Report” issue.
Changes:
- Introduces a new gh-aw workflow manifest (
token-usage-analyzer.md) defining the token usage analysis steps and report format. - Adds the compiled GitHub Actions workflow (
token-usage-analyzer.lock.yml) for execution on a daily schedule. - Updates
.github/aw/actions-lock.jsonto include the pinnedgithub/gh-aw-actions/setup@v0.65.3entry used by the compiled workflow.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| .github/workflows/token-usage-analyzer.md | New workflow prompt/manifest for discovering runs, parsing token-usage.jsonl, computing stats, and creating a report issue. |
| .github/workflows/token-usage-analyzer.lock.yml | Compiled workflow implementing the manifest with AWF + Copilot execution and Safe Outputs issue creation. |
| .github/aw/actions-lock.json | Locks the gh-aw-actions/setup action version used by the compiled workflow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| skip-if-match: | ||
| query: 'is:issue is:open label:token-usage-report' | ||
| max: 1 |
There was a problem hiding this comment.
skip-if-match is configured to skip whenever any open issue has the token-usage-report label. Since this workflow creates an issue with that label and doesn't instruct the agent to close it, the next scheduled run will be skipped indefinitely after the first report. Consider removing skip-if-match, or narrowing the query so it only matches “today’s” report (e.g., by title/date) or a specific tracking issue you intend to keep open.
| skip-if-match: | |
| query: 'is:issue is:open label:token-usage-report' | |
| max: 1 |
| "timestamp": "2026-04-01T17:38:12.486Z", | ||
| "request_id": "uuid", | ||
| "provider": "anthropic", | ||
| "model": "claude-sonnet-4-6", |
There was a problem hiding this comment.
The example token-usage record uses "model": "claude-sonnet-4-6", but the api-proxy logs/tests in this repo use values like claude-sonnet-4.6 or date-suffixed variants (e.g., claude-sonnet-4-20250514). Using a non-existent/incorrect model string here may cause downstream parsing/mapping (e.g., cost tables by model) to misclassify requests. Align the example with the actual model values emitted by the proxy.
| "model": "claude-sonnet-4-6", | |
| "model": "claude-sonnet-4.6", |
| 1. **Total tokens**: `input_tokens + output_tokens + cache_read_tokens + cache_write_tokens` | ||
| 2. **Billable tokens**: `input_tokens + output_tokens + cache_write_tokens` (cache reads are discounted) | ||
| 3. **Input/output ratio**: `(input_tokens + cache_read_tokens) / output_tokens` (if `output_tokens == 0`, treat the ratio as `∞`/`N/A` and exclude that request from ratio averages to avoid division by zero) | ||
| 4. **Cache hit rate**: `cache_read_tokens / (cache_read_tokens + input_tokens) * 100` |
There was a problem hiding this comment.
Cache hit rate is defined as cache_read_tokens / (cache_read_tokens + input_tokens) * 100, but there’s no guidance for the edge case where (cache_read_tokens + input_tokens) == 0 (possible if usage extraction yields only output tokens). To avoid division-by-zero in the analyzer script, specify how to handle a zero denominator (e.g., treat as 0% or exclude from averages).
| 4. **Cache hit rate**: `cache_read_tokens / (cache_read_tokens + input_tokens) * 100` | |
| 4. **Cache hit rate**: `cache_read_tokens / (cache_read_tokens + input_tokens) * 100` (if `cache_read_tokens + input_tokens == 0`, treat the hit rate as `0%` and exclude that request from hit-rate averages to avoid division by zero) |
| - name: Check skip-if-match query | ||
| id: check_skip_if_match | ||
| uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 | ||
| env: | ||
| GH_AW_SKIP_QUERY: "is:issue is:open label:token-usage-report" | ||
| GH_AW_WORKFLOW_NAME: "Daily Token Usage Analyzer" | ||
| GH_AW_SKIP_MAX_MATCHES: "1" |
There was a problem hiding this comment.
The compiled workflow’s pre-activation skip query will block all future scheduled runs after the first report if the created report issue remains open (is:issue is:open label:token-usage-report). If the intent is a daily series of reports, this skip condition needs to be removed or made date-specific in the source manifest and recompiled.
No description provided.