Skip to content

Feat/token usage analyzer workflow#1585

Closed
lpcox wants to merge 6 commits intomainfrom
feat/token-usage-analyzer-workflow
Closed

Feat/token usage analyzer workflow#1585
lpcox wants to merge 6 commits intomainfrom
feat/token-usage-analyzer-workflow

Conversation

@lpcox
Copy link
Copy Markdown
Collaborator

@lpcox lpcox commented Apr 1, 2026

No description provided.

lpcox and others added 6 commits April 1, 2026 12:28
Adds an agentic workflow that runs daily to mine token-usage.jsonl
from recent workflow runs, compute per-workflow statistics (tokens,
cache rates, costs, model mix), identify optimization opportunities,
and create a summary issue.

The workflow:
- Discovers completed runs from the past 24 hours
- Downloads agent-artifacts and extracts token-usage.jsonl
- Computes per-workflow stats (cache hit rate, I/O ratio, cost)
- Flags inefficiencies (zero cache hits, high I/O ratios)
- Compares with previous reports for historical trends
- Creates a structured issue with progressive disclosure

Gracefully handles missing data since token tracking is a new feature.

Closes #1551

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
The .md file was updated via GitHub web UI but the lock file
was not recompiled, causing frontmatter hash mismatch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lpcox lpcox requested a review from Mossaka as a code owner April 1, 2026 22:34
Copilot AI review requested due to automatic review settings April 1, 2026 22:34
@lpcox lpcox closed this Apr 1, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new scheduled Agentic Workflow (gh-aw) to analyze AWF api-proxy token usage logs across recent workflow runs and publish a daily “Token Usage Report” issue.

Changes:

  • Introduces a new gh-aw workflow manifest (token-usage-analyzer.md) defining the token usage analysis steps and report format.
  • Adds the compiled GitHub Actions workflow (token-usage-analyzer.lock.yml) for execution on a daily schedule.
  • Updates .github/aw/actions-lock.json to include the pinned github/gh-aw-actions/setup@v0.65.3 entry used by the compiled workflow.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
.github/workflows/token-usage-analyzer.md New workflow prompt/manifest for discovering runs, parsing token-usage.jsonl, computing stats, and creating a report issue.
.github/workflows/token-usage-analyzer.lock.yml Compiled workflow implementing the manifest with AWF + Copilot execution and Safe Outputs issue creation.
.github/aw/actions-lock.json Locks the gh-aw-actions/setup action version used by the compiled workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +6 to +8
skip-if-match:
query: 'is:issue is:open label:token-usage-report'
max: 1
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skip-if-match is configured to skip whenever any open issue has the token-usage-report label. Since this workflow creates an issue with that label and doesn't instruct the agent to close it, the next scheduled run will be skipped indefinitely after the first report. Consider removing skip-if-match, or narrowing the query so it only matches “today’s” report (e.g., by title/date) or a specific tracking issue you intend to keep open.

Suggested change
skip-if-match:
query: 'is:issue is:open label:token-usage-report'
max: 1

Copilot uses AI. Check for mistakes.
"timestamp": "2026-04-01T17:38:12.486Z",
"request_id": "uuid",
"provider": "anthropic",
"model": "claude-sonnet-4-6",
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example token-usage record uses "model": "claude-sonnet-4-6", but the api-proxy logs/tests in this repo use values like claude-sonnet-4.6 or date-suffixed variants (e.g., claude-sonnet-4-20250514). Using a non-existent/incorrect model string here may cause downstream parsing/mapping (e.g., cost tables by model) to misclassify requests. Align the example with the actual model values emitted by the proxy.

Suggested change
"model": "claude-sonnet-4-6",
"model": "claude-sonnet-4.6",

Copilot uses AI. Check for mistakes.
1. **Total tokens**: `input_tokens + output_tokens + cache_read_tokens + cache_write_tokens`
2. **Billable tokens**: `input_tokens + output_tokens + cache_write_tokens` (cache reads are discounted)
3. **Input/output ratio**: `(input_tokens + cache_read_tokens) / output_tokens` (if `output_tokens == 0`, treat the ratio as `∞`/`N/A` and exclude that request from ratio averages to avoid division by zero)
4. **Cache hit rate**: `cache_read_tokens / (cache_read_tokens + input_tokens) * 100`
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cache hit rate is defined as cache_read_tokens / (cache_read_tokens + input_tokens) * 100, but there’s no guidance for the edge case where (cache_read_tokens + input_tokens) == 0 (possible if usage extraction yields only output tokens). To avoid division-by-zero in the analyzer script, specify how to handle a zero denominator (e.g., treat as 0% or exclude from averages).

Suggested change
4. **Cache hit rate**: `cache_read_tokens / (cache_read_tokens + input_tokens) * 100`
4. **Cache hit rate**: `cache_read_tokens / (cache_read_tokens + input_tokens) * 100` (if `cache_read_tokens + input_tokens == 0`, treat the hit rate as `0%` and exclude that request from hit-rate averages to avoid division by zero)

Copilot uses AI. Check for mistakes.
Comment on lines +1030 to +1036
- name: Check skip-if-match query
id: check_skip_if_match
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
env:
GH_AW_SKIP_QUERY: "is:issue is:open label:token-usage-report"
GH_AW_WORKFLOW_NAME: "Daily Token Usage Analyzer"
GH_AW_SKIP_MAX_MATCHES: "1"
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compiled workflow’s pre-activation skip query will block all future scheduled runs after the first report if the created report issue remains open (is:issue is:open label:token-usage-report). If the intent is a daily series of reports, this skip condition needs to be removed or made date-specific in the source manifest and recompiled.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants