Skip to content

feat: add Anthropic Context Editing API support#1147

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-6ec3b1a9
Mar 13, 2026
Merged

feat: add Anthropic Context Editing API support#1147
teknium1 merged 1 commit into
mainfrom
hermes/hermes-6ec3b1a9

Conversation

@teknium1

@teknium1 teknium1 commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Refs #526, supersedes #528.

Integrates Anthropic's server-side context management (beta: context-management-2025-06-27). When enabled, the API automatically clears old tool use/result pairs and thinking blocks after prompt cache lookup but before token counting — preserving cache prefixes while freeing context space.

How it works

The client sends the full unmodified message array + a context_management parameter:

  1. Anthropic checks prompt cache → cache hit (messages unchanged)
  2. Applies edits server-side (removes old thinking/tool results)
  3. Counts tokens on edited version (now smaller)
  4. Runs inference on edited version

This is impossible with client-side stripping, which modifies the message array and breaks cache prefixes.

Implementation

  • anthropic_adapter.py: Added context-management-2025-06-27 beta header. build_anthropic_kwargs() builds context_management edits via extra_body. clear_thinking only included when reasoning is enabled (API requirement).
  • run_agent.py: Pipes context_editing config through AIAgent to the adapter
  • cli.py / gateway/run.py: Loads context_editing config from config.yaml
  • hermes_cli/config.py: context_editing section in DEFAULT_CONFIG
  • tests/test_context_editing.py: 8 tests covering all scenarios

Config (opt-in)

context_editing:
  enabled: true
  trigger_tokens: null          # auto: 60% of context window
  keep_tool_uses: 5             # recent tool pairs to preserve
  keep_thinking_turns: 2        # recent thinking turns to preserve
  exclude_tools:                # never cleared
    - memory
    - skill_manage
    - todo
  clear_tool_inputs: false
  clear_at_least_tokens: null   # auto: 10% of context window

Live tested with Anthropic API

  • ✅ Single turn with context_management — accepted, response normal
  • ✅ Multi-turn with tool calls + thinking + context editing — works end-to-end
  • clear_thinking correctly omitted when reasoning is disabled
  • ✅ Config plumbing verified through AIAgent._build_api_kwargs()
  • ✅ 3437 tests pass (6 pre-existing failures)

Notes

  • Direct Anthropic only — works with provider: anthropic. OpenRouter passthrough is unverified.
  • Disabled by default — opt-in via context_editing.enabled: true
  • Works alongside compression — context editing is first line of defense (cheap, cache-friendly), compression kicks in later if needed (expensive, cache-breaking)

Persist OAuth/setup tokens in ANTHROPIC_TOKEN instead of ANTHROPIC_API_KEY.
Reserve ANTHROPIC_API_KEY for regular Console API keys.

Changes:
- anthropic_adapter: reorder resolve_anthropic_token() priority —
  ANTHROPIC_TOKEN first, ANTHROPIC_API_KEY as legacy fallback
- config: add save_anthropic_oauth_token() / save_anthropic_api_key() helpers
  that clear the opposing slot to prevent priority conflicts
- config: show_config() prefers ANTHROPIC_TOKEN for display
- setup: OAuth login and pasted setup-tokens write to ANTHROPIC_TOKEN
- setup: API key entry writes to ANTHROPIC_API_KEY and clears ANTHROPIC_TOKEN
- main: same fixes in _run_anthropic_oauth_flow() and _model_flow_anthropic()
- main: _has_any_provider_configured() checks ANTHROPIC_TOKEN
- doctor: use _is_oauth_token() for correct auth method validation
- runtime_provider: updated error message
- run_agent: simplified client init to use resolve_anthropic_token()
- run_agent: updated 401 troubleshooting messages
- status: prefer ANTHROPIC_TOKEN in status display
- tests: updated priority test, added persistence helper tests

Cherry-picked from PR #1141 by kshitijk4poor, rebased onto current main
with unrelated changes (web_policy config, blocklist CLI) removed.

Co-authored-by: kshitijk4poor <kshitijk4poor@users.noreply.github.com>
@teknium1 teknium1 force-pushed the hermes/hermes-6ec3b1a9 branch from 048c0a9 to bb3f5ed Compare March 13, 2026 09:09
@teknium1 teknium1 changed the title fix: save Anthropic OAuth tokens to correct env var (CLAUDE_CODE_OAUTH_TOKEN) fix: separate Anthropic OAuth tokens from API keys Mar 13, 2026
@teknium1 teknium1 merged commit 34c8a5f into main Mar 13, 2026
1 check failed
@teknium1 teknium1 changed the title fix: separate Anthropic OAuth tokens from API keys feat: Anthropic Context Editing API + OAuth token fix Mar 13, 2026
@teknium1 teknium1 changed the title feat: Anthropic Context Editing API + OAuth token fix feat: add Anthropic Context Editing API support Mar 13, 2026
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
…6ec3b1a9

fix: separate Anthropic OAuth tokens from API keys
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
…6ec3b1a9

fix: separate Anthropic OAuth tokens from API keys
olympus-terminal pushed a commit to olympus-terminal/hermes-agent that referenced this pull request May 16, 2026
…6ec3b1a9

fix: separate Anthropic OAuth tokens from API keys
Egavasyug pushed a commit to Egavasyug/hermes-agent that referenced this pull request Jun 10, 2026
…6ec3b1a9

fix: separate Anthropic OAuth tokens from API keys
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants