Target Workflow: smoke-copilot.md

Source report: #1604
Estimated cost per run: $0.70
Total tokens per run: ~374K (211K billable)
Cache hit rate: 43.7%
LLM turns: 5
Model: claude-sonnet-4.6 via Copilot provider

Current Configuration

Per-Turn Breakdown

Root Cause: 21 Unused Playwright/Browser Tools

The Copilot CLI v0.65.5 includes built-in Playwright/browser tools. When compiled with --allow-all-tools (required for non-interactive mode), all 21 browser tools are sent to the model on every turn — even though smoke-copilot never uses any of them:

browser_close  browser_resize     browser_console_messages  browser_handle_dialog
browser_evaluate  browser_file_upload  browser_fill_form    browser_press_key
browser_type   browser_navigate   browser_navigate_back     browser_network_requests
browser_run_code  browser_take_screenshot  browser_snapshot  browser_click
browser_drag   browser_hover      browser_select_option     browser_tabs
browser_wait_for

At ~500 tokens/schema × 21 tools = ~10,500 tokens of dead weight on every turn.

Recommendations

1. Exclude Built-in Browser/Playwright Tools

Estimated savings: ~40K tokens/run (10K billable) → **$0.09/run (~13%)**

The Copilot CLI supports --excluded-tools to filter which tools are visible to the model. Since smoke-copilot only needs bash, the GitHub MCP tools, and safe-output tools, all 21 browser tools can be excluded.

Implementation — modify postprocess-smoke-workflows.ts to add --excluded-tools after --allow-all-tools in the copilot command for smoke-copilot.lock.yml:

- -- /bin/bash -c '/usr/local/bin/copilot ... --allow-all-tools --allow-all-paths ...
+ -- /bin/bash -c '/usr/local/bin/copilot ... --allow-all-tools \
+     --excluded-tools=browser_close,browser_resize,browser_console_messages,browser_handle_dialog,browser_evaluate,browser_file_upload,browser_fill_form,browser_press_key,browser_type,browser_navigate,browser_navigate_back,browser_network_requests,browser_run_code,browser_take_screenshot,browser_snapshot,browser_click,browser_drag,browser_hover,browser_select_option,browser_tabs,browser_wait_for \
+     --allow-all-paths ...

Alternative (preferred long-term): Add tools.excluded frontmatter support to the gh-aw compiler so this can be expressed in smoke-copilot.md:

tools:
  bash:
    - "*"
  github:
    toolsets: [pull_requests]
  excluded:
    - browser_*

This would set GH_AW_INFO_EXCLUDED_TOOLS in the lock file and allow the runner to pass --excluded-tools.

2. Remove Redundant "Verify MCP" Tool Call (Save 1 Turn)

Estimated savings: 42K tokens/run (24K billable) → **$0.07/run (~10%)**

The prompt's Section 1 (GitHub MCP Testing) instructs the agent to call github-list_pull_requests to "verify MCP connectivity." But the pre-step already fetched 2 merged PRs via SMOKE_PR_DATA. This forces Turn 1 to make a redundant network call that adds a full turn's worth of context growth.

Change in smoke-copilot.md — replace:

### 1. GitHub MCP Testing
The last 2 merged pull requests have been fetched. Verify MCP connectivity by calling `github-list_pull_requests` for ${{ github.repository }} (limit 1, state merged) and confirm data is returned.

with:

### 1. GitHub MCP Testing
Pre-step result: MCP connectivity confirmed — 2 merged PRs were pre-fetched successfully. No additional MCP call required; use the pre-fetched data below.

This converts Turn 1 from a tool-calling turn (40K input + MCP round-trip) to a reasoning-only turn and eliminates the need for a dedicated MCP verification turn, reducing from 5 → 4 turns.

Additional micro-optimization: The bash echo "bash works" test in Section 4 is redundant since bash functionality is already proven by the pre-step (which ran shell commands) and by the cat file test in Section 3. Removing it saves another turn (5 → 3 turns total):

- ### 4. Bash Tool Testing
- Run a simple bash command (e.g., `echo "bash works"`) to verify the bash tool is functional.

3. Drop the repos GitHub Toolset

Estimated savings: ~12K tokens/run (5K billable) → **$0.015/run (~2%)**

The repos toolset loads 4 tools (list_releases, list_tags, search_repositories, search_code) that are never called in smoke-copilot. Only list_pull_requests and pull_request_read (from pull_requests toolset) are relevant.

Change in smoke-copilot.md:

 tools:
   bash:
     - "*"
   github:
-    toolsets: [repos, pull_requests]
+    toolsets: [pull_requests]

This removes 4 tool schemas (~2,400 tokens/turn) from every request.

Expected Impact

Implementation Checklist

• Improve links in readme to AW project #1 (Tool exclusion): Add --excluded-tools=browser_close,... to postprocess-smoke-workflows.ts for smoke-copilot, OR implement tools.excluded in gh-aw frontmatter schema
• Secret proxying #2 (Remove redundant MCP call): Edit prompt in .github/workflows/smoke-copilot.md to use pre-fetched data instead of calling list_pull_requests
• #2b (Remove echo bash test): Remove Section 4 "Bash Tool Testing" from smoke-copilot.md (bash is proven by file write/read)
• feat: add integration test for rostbuness #3 (Toolset reduction): Change toolsets: [repos, pull_requests] → toolsets: [pull_requests] in smoke-copilot.md
• Recompile: gh-aw compile .github/workflows/smoke-copilot.md
• Post-process: npx tsx scripts/ci/postprocess-smoke-workflows.ts
• Verify CI passes on PR
• Compare token usage on new run vs baseline (target: 5 turns → 3, ~370K → ~280K)

Note: Recommendations #2 and #3 only require editing smoke-copilot.md and recompiling — no changes to post-processing or the gh-aw tool. Recommendation #1 requires either a postprocess patch or a gh-aw compiler feature request.

Generated by Daily Copilot Token Optimization Advisor · ◷