⚡ Token optimization for pelis-agent-factory-advisor (~151K tokens/run, −11%)#1718
⚡ Token optimization for pelis-agent-factory-advisor (~151K tokens/run, −11%)#1718
Conversation
…arallel hints, condense Phase 4 Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/88a7333b-1b4b-4b0f-ae2d-0bc182b4ad03 Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR reduces token usage in the pelis-agent-factory-advisor agentic workflow by shrinking the GitHub MCP tool prompt, encouraging batched tool calls, and shortening the Phase 4 output template to avoid repeated boilerplate across turns.
Changes:
- Restricts GitHub MCP toolsets to
contextonly to reduce prompt/tooling token overhead. - Updates the workflow guidance to batch file reads and combine
agentic-workflows status+auditin one turn. - Condenses the Phase 4 output template and recompiles the generated
.lock.yml.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/pelis-agent-factory-advisor.md | Narrows GitHub toolset + adds batching guidance + compacts output template. |
| .github/workflows/pelis-agent-factory-advisor.lock.yml | Regenerated compiled workflow; reflects toolset restriction and other compile-time deltas. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 2
| ## Phase 1: Learn Pelis Agent Factory Patterns | ||
|
|
||
| > **Efficiency note:** Read all required files in a **single parallel batch** — call `bash:cat` for `.content-hash.txt`, `.pelis-agent-factory-docs.txt`, `.agentics-patterns.txt`, and `.repo-structure.txt` simultaneously in your first turn. Do not read them one at a time. | ||
|
|
There was a problem hiding this comment.
The Phase 1 cache logic says to compare .content-hash.txt to pelis_docs_hash and skip reading docs/patterns when unchanged, but the new Efficiency note instructs reading .pelis-agent-factory-docs.txt and .agentics-patterns.txt in the first turn unconditionally. This can increase token usage on cache hits and contradicts the skip behavior. Consider batching only .content-hash.txt (and optionally .repo-structure.txt) first, then reading the doc/pattern files in a subsequent turn only when the hash differs.
See below for a potential fix:
> **Efficiency note:** Use **batched reads**, but preserve the cache gate. In your first turn, call `bash:cat` for `.content-hash.txt` and `.repo-structure.txt` together. Only if the hash is changed or missing should you make a second parallel batch to read `.pelis-agent-factory-docs.txt` and `.agentics-patterns.txt`. Do not read the doc/pattern files on cache hits.
Check cache-memory for `pelis_docs_hash`. Read the precomputed hash from
`.content-hash.txt` and compare it to the cached value.
If unchanged, skip reading `.pelis-agent-factory-docs.txt` and `.agentics-patterns.txt` and continue to Phase 2 using cached knowledge.
Otherwise read those files in a single parallel batch and update the hash in cache-memory.
| name: "Pelis Agent Factory Advisor" | ||
| "on": | ||
| schedule: | ||
| - cron: "26 21 * * *" | ||
| - cron: "29 10 * * *" | ||
| # Friendly format: daily (scattered) |
There was a problem hiding this comment.
The compiled workflow’s daily cron time changed ("26 21" -> "29 10"). The PR description doesn’t mention a schedule shift; if this is unintentional (e.g., a recompile side effect), it will change when the advisor runs each day. Please confirm whether the new cron is intended and, if not, adjust the source .md/compile settings to keep the prior run time.
|
@copilot apply changes based on the comments in this thread |
Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/82a91652-b50f-481b-a286-f32ed48069ba Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Applied in 899f815. The Phase 1 efficiency note now preserves the cache gate: the first turn batches only Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (1 files)
Coverage comparison generated by |
|
Smoke test results (run
Overall: PASS
|
🔥 Smoke Test Results
PR: ⚡ Token optimization for pelis-agent-factory-advisor (~151K tokens/run, −11%) Overall: PASS 🟢
|
Security Review: PR #1718 — Pelis Agent Factory Advisor Token OptimizationOverall: One security regression, two security improvements, no critical issues.
|
|
Smoke test results:
|
Smoke Test: GitHub Actions Services Connectivity
All checks failed.
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
|
The
pelis-agent-factory-advisorworkflow costs ~$2.02/run (1,399K tokens, 11 turns) due to oversized GitHub MCP toolset, sequential file reads on early turns, and a bloated Phase 4 template repeated across all turns.Changes
Restrict GitHub toolset to
contextonly — dropsGITHUB_TOOLSETSfromcontext,repos,issues,pull_requests(25 tools) tocontext(~2 tools); saves ~88K tokens/runParallel file read instruction — adds an efficiency note at the top of Phase 1 directing the agent to
catall four pre-fetched files in a single batched turn; collapses the turn-2 +19K token jump caused by sequential readsParallel status+audit instruction — top-level note + Phase 2.3 rewrite direct the agent to call
agentic-workflows statusandauditin the same turn instead of sequentiallyCondense Phase 4 output template — reduces from ~103 lines to ~50 lines (~14K tokens/run saved); replaces repeated placeholder blocks with a compact 7-item numbered list
Recompile — regenerated
pelis-agent-factory-advisor.lock.ymlviagh aw compile+postprocess-smoke-workflows.tsProjected outcome: ~1,248K tokens/run (−151K), ~$1.80/run (−$0.22), 8–9 LLM turns vs 11.