fix(context): align fold summary prefix with main agent for cache reuse#1565
Merged
Conversation
added 4 commits
May 22, 2026 08:45
…ult, lifecycle plans Headline themes: - TUI: Static-history renderer is the only path; virtual-viewport layers removed (#1529 stages 1-4) - Chat: queued mid-turn steer handling so input mid-render doesn't drop or fight the live frame (#1501) - Web search: default switches to Bing; dashboard engine switcher; Mojeek dropped (#1558) - Plans: lifecycle evidence summaries surface why a plan is ready to accept (#1500) - Desktop: native OS notifications for approvals + completion (#1519) - i18n: CLI command output (/mcp /sessions /prune /theme) + approval-prompt labels translated (#1524, #1560) - Security: SSRF block in web_fetch (#1544), edit-snapshot path containment (#1454), shell redirect sandbox (#1457), Task integrity guardrail (#1516) - Tools: per-turn dispatch-rate limit (#1356); run_command discourages shell-based edits (#1514) - Client: DeepSeek 429 → concurrency-limit hint (#1526); timeoutMs honored with AbortSignal (#1535); --no-proxy opt-out for direct route (#1507) - Files: read/edit/restore preserves source encoding (GB18030 / UTF-8 BOM) (#1518) - Context: pinned constraints survive folds + full tail capture (#1515, #1552) - Refactor: lifecycle risk policy extracted into its own module (#1557) See CHANGELOG for the full list.
The summarizer call was sending a bespoke "You compress conversation history" system prompt and no tools, guaranteeing a 0% cache hit against the main agent's just-cached prefix. Reshape the request so system + tools + head bytes mirror the live agent's last call — the only novel bytes are the trailing summarize instruction. Skill-pin handling now collects bodies read-only instead of stubbing mid-head, so the cache prefix stays unbroken. The summarize instruction names pinned skills so the model knows not to paraphrase their bodies (which we append verbatim regardless). Measured on a real session at 48.7K prompt tokens: OLD shape: 0.0% cache hit → $0.145 per fold NEW shape: 99.6% cache hit → $0.015 per fold saving: 89.6% per fold
bench-fold-cache-shape.mjs replays real session jsonls, simulates OLD vs NEW summary-call shapes at the fold point, and reports byte-level shared-prefix with the main agent's preceding request. Pure local — no API required. bench-fold-cache-live.mjs sends one priming + two summary calls to DeepSeek and reports prompt_cache_hit_tokens / cost for each shape. Used to confirm the shape change actually translates to API-side cache hits.
…d-summary # Conflicts: # src/context-manager.ts # src/loop.ts
| const resp = await fetch("https://api.deepseek.com/chat/completions", { | ||
| method: "POST", | ||
| headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" }, | ||
| body: JSON.stringify(body), |
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The fold summary call was being sent with a bespoke
"You compress conversation history…"system prompt and no tools, which produces a 0% prefix-cache hit against the main agent's just-cached request. Every fold paid full input price on tens of thousands of tokens.This PR reshapes the summary request so its prefix mirrors the live agent's last call:
Skill-pin handling was split:
collectPinnedSkillsis read-only and leaves head bytes intact. The summarize instruction now names the pinned skills so the model doesn't paraphrase their bodies (we still append them verbatim regardless).Numbers
Measured on a real 60-message session at 48.7K prompt tokens (
bench-fold-cache-live.mjs, real DeepSeek API):~89.6% saving per fold. Across 6 real session replays the saving holds steady at 88-90%. The remaining ~0.4% miss is just the trailing summarize instruction (~185 tokens).
Test plan
tests/context-manager-cache-aligned-fold.test.ts(system / tools / head bytes / trailing instruction / model pin / skill-pin verbatim)context-manager-skill-pin,context-manager-thinking-mode,context-manager-fold-timeout)npm run verify: build + lint + typecheck + 3561 tests greentools/bench-fold-cache-live.mjsconfirms 99.6% cache hit on real DeepSeek