Skip to content

feat: Token budget diff CLI command + optional GitHub Action wrapper #81

Description

@spboyer

Summary

Create a GitHub Action that runs waza tokens check on PRs and posts a formatted comment showing token budget changes (delta) compared to the base branch.

Updated approach (2026-03-05): Implement this as a CLI-first feature (waza tokens diff) that ships through the existing distribution channels (binary, azd extension), with an optional thin GitHub Action wrapper for PR comment automation.

Motivation

Token budgets are critical for skill quality. When a PR modifies a skill's SKILL.md or related files, reviewers need to see the token impact at a glance — not dig through CI logs. A PR comment with a clear before/after diff makes token regressions immediately visible.

Revised Implementation

Phase 1: CLI Command — waza tokens diff [base-ref]

Add a new CLI command that compares token budgets between the current working tree (or HEAD) and a git ref (branch, tag, commit SHA).

Usage:
`�ash

Compare current branch vs main

waza tokens diff main

Compare two refs

waza tokens diff main..feature-branch

JSON output for CI integration

waza tokens diff main --format json

With threshold (exit code 1 if any skill exceeds threshold)

waza tokens diff main --threshold 10
`

Output (human-readable):
`
🔢 Token Budget Diff (vs main)

Skill Before After Delta Status
azure-deploy 3,200 3,450 +250 ⚠️ +7.8%
azure-search 2,100 2,100 0 ✅ No change
`

Output (JSON, --format json):
json { "base_ref": "main", "skills": [ { "name": "azure-deploy", "before": 3200, "after": 3450, "delta": 250, "percent": 7.8 } ], "threshold": 10, "passed": true }

This ships automatically via:

  • waza tokens diff (binary install)
  • �zd waza tokens diff (azd extension — no extra install for existing users)

Phase 2: Optional GitHub Action Wrapper (~20 lines)

A thin action at .github/actions/token-diff/action.yml that:

  1. Installs waza (or uses azd extension)
  2. Runs waza tokens diff --format json
  3. Posts/updates a PR comment with the formatted table
  4. Sets exit code based on threshold

`yaml

  • uses: microsoft/waza/.github/actions/token-diff@main
    with:
    skills-dir: skills/
    threshold: 10
    `

Why CLI-first?

  • Works everywhere — GitHub, GitLab, Azure DevOps, local dev
  • No vendor lock-in — token logic lives in the CLI, not a GitHub-specific action
  • Free distribution — azd extension users get it automatically, no extra install
  • Single source of truth — comparison logic in Go, action is just a wrapper
  • Matches existing patterns — Copilot for Azure team already uses �zd waza run in CI

Acceptance Criteria

  • waza tokens diff [ref] CLI command with human-readable and JSON output
  • Configurable failure threshold (e.g., fail if >10% increase)
  • Works with both skills/ and .github/skills/ directories
  • Respects .waza.yaml and .token-limits.json configs
  • Optional: thin GitHub Action wrapper that posts PR comment with delta table

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestpriority:p1This sprintsquad:linusAssigned to Linus (Backend Developer)

Fields

No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions