fix: extract OpenAI/Copilot cached_tokens from prompt_tokens_details by lpcox · Pull Request #1603 · github/gh-aw-firewall

lpcox · 2026-04-02T15:35:39Z

Problem

The api-proxy token tracker only extracted Anthropic-style cache fields (cache_read_input_tokens, cache_creation_input_tokens) but missed the OpenAI/Copilot format where cache info is nested under usage.prompt_tokens_details.cached_tokens.

This caused token-usage.jsonl to report cache_read_tokens: 0 for all Copilot API requests, even when the upstream API was returning significant cache hits.

Evidence from smoke-copilot run 23878258933

What token-usage.jsonl reported (api-proxy):

cache_read_tokens: 0   (all 10 requests)

What agent.log showed (from actual API responses):

Turn	prompt_tokens	cached_tokens	Net New
1	40,272	0	40,272
2	41,344	36,500	4,844
3	41,658	41,280	378
...	...	...	...
10	43,977	43,894	83

Prompt caching was saving ~88% of input tokens, but the tracker was blind to it.

Root Cause

The OpenAI/Copilot API returns cache info in a nested format:

{
  "usage": {
    "prompt_tokens": 41344,
    "completion_tokens": 256,
    "prompt_tokens_details": {
      "cached_tokens": 36500
    }
  }
}

Both extractUsageFromJson() and extractUsageFromSseLine() only read flat prompt_tokens/completion_tokens/total_tokens — they never looked inside prompt_tokens_details. The normalizeUsage() function then fell back to cache_read_input_tokens ?? 0 (the Anthropic field name) which was never set.

Fix

Extract usage.prompt_tokens_details.cached_tokens in both extractUsageFromJson() and extractUsageFromSseLine()
Map it to cache_read_input_tokens so normalizeUsage() picks it up correctly
Updated JSDoc comments to document the OpenAI cache format
Added 4 new tests covering the fix

Testing

All 192 api-proxy tests pass (cd containers/api-proxy && npm test).

New tests:

extractUsageFromJson → extracts prompt_tokens_details.cached_tokens
extractUsageFromJson → handles responses without prompt_tokens_details
extractUsageFromSseLine → extracts cache tokens from streaming final chunk
normalizeUsage → normalizes OpenAI cache tokens via cache_read_input_tokens mapping

The token tracker only extracted Anthropic-style cache fields (cache_read_input_tokens, cache_creation_input_tokens) but missed the OpenAI/Copilot format where cache info is nested under usage.prompt_tokens_details.cached_tokens. This caused token-usage.jsonl to report cache_read_tokens: 0 for all Copilot API requests, even when the API was returning significant cache hits (e.g., 43,894 of 43,977 prompt tokens cached). Fix both extractUsageFromJson() and extractUsageFromSseLine() to read prompt_tokens_details.cached_tokens and map it to the normalized cache_read_input_tokens field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-04-02T15:36:37Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	85.86%	85.96%	📈 +0.10%
Statements	85.75%	85.84%	📈 +0.09%
Functions	86.66%	86.66%	➡️ +0.00%
Branches	78.49%	78.55%	📈 +0.06%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/docker-manager.ts`	86.1% → 86.5% (+0.40%)	85.6% → 86.0% (+0.39%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

Fixes api-proxy token tracking for OpenAI/Copilot prompt caching by extracting usage.prompt_tokens_details.cached_tokens and surfacing it through the existing normalized cache fields so token-usage.jsonl reflects cache hits correctly.

Changes:

Extract usage.prompt_tokens_details.cached_tokens in both non-streaming JSON and streaming SSE usage parsing, mapping it to cache_read_input_tokens.
Update usage-format JSDoc examples to include the nested OpenAI/Copilot cache field.
Add tests covering JSON extraction, SSE final-chunk extraction, and normalization behavior for OpenAI/Copilot cached tokens.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
containers/api-proxy/token-tracker.js	Adds extraction of OpenAI/Copilot nested cached-token usage fields for both JSON and SSE parsing paths.
containers/api-proxy/token-tracker.test.js	Adds targeted regression tests ensuring cached tokens are extracted and normalized as expected.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-02T15:38:45Z

containers/api-proxy/token-tracker.js

+      // OpenAI/Copilot nested cache fields (prompt_tokens_details.cached_tokens)
+      if (json.usage.prompt_tokens_details && typeof json.usage.prompt_tokens_details.cached_tokens === 'number') {
+        usage.cache_read_input_tokens = json.usage.prompt_tokens_details.cached_tokens;
+        hasField = true;
+      }


normalizeUsage()’s JSDoc (later in this file) still states cache_read_tokens/cache_write_tokens are “Anthropic only, 0 for others”, but this change now populates cache_read_input_tokens from OpenAI/Copilot prompt_tokens_details.cached_tokens, which will result in non-zero cache_read_tokens for OpenAI/Copilot too. Please update the normalization doc comment to reflect the new supported source(s) so future readers don’t assume cache fields are Anthropic-only.

github-actions · 2026-04-02T15:40:39Z

Smoke Test Results — @lpcox

✅ GitHub MCP: PR #1593 "fix: capture full session state…" / PR #1588 "fix: rename and scope token analyzer…"
✅ Playwright: github.com title contains "GitHub"
✅ File write: /tmp/gh-aw/agent/smoke-test-copilot-23908548855.txt created and verified
✅ Bash: file read back successfully

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot for issue #1603

github-actions · 2026-04-02T15:40:50Z

Smoke test results (run 23908548862)

✅ GitHub MCP: #1593 fix: capture full session state, #1581 feat: add esbuild single-file bundle
✅ Playwright: github.com title contains "GitHub"
✅ File write: /tmp/gh-aw/agent/smoke-test-claude-23908548862.txt created and verified
✅ Bash: file contents confirmed

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude for issue #1603

github-actions · 2026-04-02T15:42:32Z

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.3	❌ NO
Node.js	v24.14.1	v20.20.2	❌ NO
Go	go1.22.12	go1.22.12	✅ YES

Result: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot environments.

Tested by Smoke Chroot for issue #1603

github-actions · 2026-04-02T15:42:51Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #1603 · ◷

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-04-02T15:48:30Z

Smoke test results

✅ GitHub MCP: #1593 fix: capture full session state, #1581 feat: add esbuild bundle
✅ Playwright: github.com title contains "GitHub"
✅ File write: /tmp/gh-aw/agent/smoke-test-claude-23908898243.txt created
✅ Bash verify: file content confirmed

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude for issue #1603

github-actions · 2026-04-02T15:48:38Z

Smoke Test Results — Copilot Engine

Test	Result
GitHub MCP (last 2 merged PRs)	✅ #1593, #1588
Playwright (github.com title)	✅
File write	✅
Bash verify	✅

Overall: PASS

Author: @lpcox · No assignees

📰 BREAKING: Report filed by Smoke Copilot for issue #1603

github-actions · 2026-04-02T15:49:47Z

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	`Python 3.12.13`	`Python 3.12.3`	❌ NO
Node.js	`v24.14.1`	`v20.20.2`	❌ NO
Go	`go1.22.12`	`go1.22.12`	✅ YES

Overall: ❌ FAILED — Python and Node.js versions differ between host and chroot environments.

Tested by Smoke Chroot for issue #1603

github-actions · 2026-04-02T15:50:24Z

Smoke test results for workflow run 23908898315

GitHub MCP last 2 merged PRs: ✅ fix: capture full session state — replace blanket ~/.copilot mount, add --session-state-dir, feat: add esbuild single-file bundle as lightweight distribution
safeinputs-gh PR query: ❌ tool unavailable in this runtime
Playwright title contains "GitHub": ✅
Tavily search returned at least one item: ❌ tool unavailable in this runtime
File write/read /tmp/gh-aw/agent/smoke-test-codex-23908898315.txt: ✅
Build npm ci && npm run build: ✅
Discussion query + mystical discussion comment: ❌ no discussion result/tool path available

Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-04-02T15:50:44Z

Smoke Test: GitHub Actions Services Connectivity ✅

All 3 connectivity checks passed:

Check	Result
Redis `PING` → `host.docker.internal:6379`	✅ `PONG`
`pg_isready` → `host.docker.internal:5432`	✅ accepting connections
`psql SELECT 1` → `smoketest` db	✅ returned `1`

Note: redis-cli was not available in the environment; Redis was verified via a raw RESP protocol socket connection.

🔌 Service connectivity validated by Smoke Services

Copilot AI review requested due to automatic review settings April 2, 2026 15:35

lpcox requested a review from Mossaka as a code owner April 2, 2026 15:35

Copilot started reviewing on behalf of lpcox April 2, 2026 15:36 View session

Copilot AI approved these changes Apr 2, 2026

View reviewed changes

github-actions bot added the smoke-copilot label Apr 2, 2026

github-actions bot added the smoke-claude label Apr 2, 2026

github-actions bot mentioned this pull request Apr 2, 2026

[agentics] No-Op Runs #1580

Closed

This comment has been minimized.

Sign in to view

github-actions bot added the smoke-services label Apr 2, 2026

github-actions bot added the build-test label Apr 2, 2026

docs: update normalizeUsage JSDoc to reflect OpenAI cache support

261d731

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

lpcox merged commit 88017ae into main Apr 2, 2026
63 of 65 checks passed

lpcox deleted the fix/openai-cache-token-tracking branch April 2, 2026 16:11

This was referenced Apr 2, 2026

feat: add Daily Claude Token Usage Analyzer workflow #1605

Merged

perf: reduce smoke-copilot token usage with pre-steps and tool trimming #1613

Merged

Conversation

lpcox commented Apr 2, 2026

Problem

Evidence from smoke-copilot run 23878258933

Root Cause

Fix

Testing

Uh oh!

github-actions bot commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

This comment has been minimized.

github-actions bot commented Apr 2, 2026

Chroot Version Comparison Results

Uh oh!

This comment has been minimized.

github-actions bot commented Apr 2, 2026

🏗️ Build Test Suite Results

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Smoke Test Results — Copilot Engine

Uh oh!

github-actions bot commented Apr 2, 2026

Chroot Version Comparison Results

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Smoke Test: GitHub Actions Services Connectivity ✅

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Apr 2, 2026 •

edited

Loading