Skip to content

Add optional OpenRouter response caching headers#19112

Closed
Julientalbot wants to merge 1 commit into
NousResearch:mainfrom
Julientalbot:jt/openrouter-response-cache
Closed

Add optional OpenRouter response caching headers#19112
Julientalbot wants to merge 1 commit into
NousResearch:mainfrom
Julientalbot:jt/openrouter-response-cache

Conversation

@Julientalbot

@Julientalbot Julientalbot commented May 3, 2026

Copy link
Copy Markdown
Contributor

Context

OpenRouter announced beta support for response caching:
https://openrouter.ai/announcements/response-caching

This lets clients opt into caching completed responses for identical OpenRouter requests. Hermes already sends OpenRouter attribution headers; this PR adds a small, explicit opt-in path for the new cache headers without changing default behaviour.

The intent is to make response caching available for workflows where repeat requests are expected and safe, such as retries, test runs, cron/batch jobs, and stable auxiliary tasks. It stays disabled by default because live agent conversations and fresh tool-context workflows should not silently change transport behaviour.

Summary

  • add a shared OpenRouter header helper that preserves existing attribution headers
  • add optional OpenRouter response caching headers behind explicit config/env opt-in
  • apply the helper to the main agent OpenRouter client and auxiliary OpenRouter clients

Config

Default remains disabled:

openrouter:
  response_cache:
    enabled: false
    ttl_seconds: null

To enable:

openrouter:
  response_cache:
    enabled: true
    ttl_seconds: 600

Environment overrides:

HERMES_OPENROUTER_RESPONSE_CACHE=true
HERMES_OPENROUTER_RESPONSE_CACHE_TTL=600

Tests

  • python -m pytest tests/agent/test_openrouter_headers.py tests/agent/test_auxiliary_client.py tests/run_agent/test_run_agent.py -q
  • python -m pytest tests/hermes_cli/test_config.py tests/hermes_cli/test_config_validation.py tests/hermes_cli/test_config_drift.py -q
  • python -m py_compile agent/openrouter_headers.py agent/auxiliary_client.py run_agent.py hermes_cli/config.py
  • git diff --check

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/agent Core agent loop, run_agent.py, prompt builder provider/openrouter OpenRouter aggregator labels May 3, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Likely duplicate of #18921 — same feature (opt-in OpenRouter response caching via headers/env vars). Both add X-OpenRouter-Cache headers behind config/env opt-in.

@Julientalbot

Copy link
Copy Markdown
Contributor Author

@alt-glitch likely duplicate you are right, sorry !

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Merged via PR #19132 which consolidates all three OpenRouter response caching PRs (#18921, #19112, #19132). Your config.yaml integration approach was the foundation — the final implementation uses config.yaml as the primary surface with env var overrides on top, plus cache status logging. Thanks for the comprehensive work @Julientalbot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent loop, run_agent.py, prompt builder P3 Low — cosmetic, nice to have provider/openrouter OpenRouter aggregator type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants