Summary

Refactor Anthropic prompt caching to use a structured multi-block system prompt with per-block cache_control markers instead of a single monolithic system message. This maximizes cache hits by isolating volatile content (timestamps, platform hints) from stable content (identity, skills, memory).

Architecture

The system prompt is now assembled as three SystemPromptBlock instances with different cache TTLs:

At API call time, blocks are converted to Anthropic content block format (`[{type: text, text: ..., cache_control: ...}, ...]`) and sent as the system message. Non-caching models fall through to the flat-string path unchanged.

New public API in `agent/prompt_caching.py`

• `SystemPromptBlock`, `CacheMetrics`, `AggregatedCacheMetrics` dataclasses
• `build_system_content_blocks(blocks)` — convert blocks to Anthropic format
• `apply_anthropic_cache_control_v2(messages, tools, cache_ttl, native_anthropic)` — multi-block + tool caching with budget management (max 4 breakpoints across tools + system + messages)
• `extract_cache_metrics(usage, api_mode)` — per-call cache extraction supporting both native Anthropic (`cache_read_input_tokens`, `cache_creation_input_tokens`) and OpenRouter (`prompt_tokens_details.cached_tokens`) response formats
• `aggregate_cache_metrics(metrics_list)` — cross-turn aggregation

The v1 `apply_anthropic_cache_control` function and `_apply_cache_marker` helper are preserved unchanged for backward compatibility.

Integration in `run_agent.py`

• New `_build_system_prompt_blocks()` method assembles the three tiered blocks and caches them on `self._cached_system_blocks`
• The existing `_build_system_prompt()` method still returns a flat string (for backward compatibility with code paths that expect one) but now delegates to the block builder
• Cached blocks are invalidated on context compression (`_cached_system_blocks = None` alongside `_cached_system_prompt = None`)
• At API call time, when `_use_prompt_caching` is enabled and `_cached_system_blocks` is populated, a multi-block path builds `{role: system, content: [...]}` with cache_control markers already set per block
• Plugin turn context (`_plugin_turn_context`) remains reserved for future system-level plugin instructions; plugin context from pre_llm_call hooks still goes into user messages (unchanged)
• Fallback flat-string path handles non-caching models and pre-structured content correctly

Test coverage

• `tests/agent/test_prompt_caching.py` — 46 unit tests covering v1 (preserved) and v2 functions: data structures, cache markers, content block conversion, pre-structured detection, breakpoint budgeting, metrics extraction and aggregation
• `tests/agent/test_prompt_caching_v2.py` — 38 additional integration tests for v2 behavior (tool caching interaction with system blocks, budget with pre-structured content, backward compatibility with v1 code paths)
• `tests/test_prompt_caching_integration.py` — 10 integration tests against `run_agent.py` block assembly (three-block structure, tier TTLs, timestamp in ephemeral block only, cache invalidation, backward-compat string return, non-caching models unaffected)

Verified: 317 tests passing (all of the above plus `tests/test_run_agent.py` regression suite).

Test plan

• All new v2 unit tests pass (`pytest tests/agent/test_prompt_caching.py tests/agent/test_prompt_caching_v2.py`)
• Integration tests against `run_agent.py` block assembly pass (`pytest tests/test_prompt_caching_integration.py`)
• Full run_agent.py regression suite passes (`pytest tests/test_run_agent.py`)
• `run_agent` imports cleanly
• Manual: verify cache hit rate improves on a multi-turn conversation with stable context files (reviewer action)
• Manual: verify non-caching models (e.g. local Ollama) still work via flat-string fallback (reviewer action)

Platforms tested

Linux (WSL2, Ubuntu 22.04), Python 3.11

🤖 Generated with Claude Code