Skip to content

[Bug]: openai-codex headers from PR #12664 are read off wrong SDK attribute (_default_headers vs _custom_headers), silently dropped before request #19981

@crmchattie

Description

@crmchattie

Summary

PR #12664 (merged 2026-04-19) added _codex_cloudflare_headers to agent/auxiliary_client.py and wired it into resolve_provider_client's raw_codex branch (line 2161 in current main). The headers are correctly attached to the OpenAI client returned by the resolver.

However the hand-off in run_agent.py reads them off the wrong attribute:

https://github.com/NousResearch/hermes-agent/blob/main/run_agent.py#L1459-L1460

if hasattr(_routed_client, '_default_headers') and _routed_client._default_headers:
    client_kwargs["default_headers"] = dict(_routed_client._default_headers)

OpenAI Python SDK ≥ 1.x stores user-supplied default_headers=... under _custom_headers, not _default_headers. Verified live with openai==2.32.0:

>>> from openai import OpenAI
>>> c = OpenAI(api_key="x", default_headers={"A": "b"})
>>> getattr(c, "_default_headers", "<missing>")
'<missing>'
>>> c._custom_headers
{'A': 'b'}

The same wrong attribute is used at run_agent.py:1504-1505 for the fallback path, while a sibling code path at line 7662 already uses _custom_headers correctly:

https://github.com/NousResearch/hermes-agent/blob/main/run_agent.py#L7662

So the codex headers are silently dropped between the resolver and the new client. Downstream the request goes out with only Authorization + Content-Type, the Codex backend can't determine the plan, and rejects every model with:

{"detail":"The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account."}

Reproduction

  • Hermes 0.12.0 (2026.4.30)
  • ChatGPT subscription via hermes auth add openai-codex --type oauth
  • Linux (Hetzner Ubuntu 24.04, x86_64), Python 3.11.15, OpenAI SDK 2.32.0
$ hermes --provider openai-codex -m gpt-5.5 -z "Reply with only the word PONG."
[no output]
$ jq '.error.message' ~/.hermes/sessions/request_dump_*.json | tail -1
"Error code: 400 - {'detail': \"The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account.\"}"
$ jq '.request.headers | keys' ~/.hermes/sessions/request_dump_*.json | tail -10
[
  "Authorization",
  "Content-Type"
]

The same account works fine with the official codex CLI on the same host (gpt-5.5 returns "PONG" immediately), so the account, network, and OAuth flow are not at fault.

After the patch below is applied, the request goes out with originator: codex_cli_rs, User-Agent: codex_cli_rs/0.0.0 (Hermes Agent), and ChatGPT-Account-ID: ... — and the same prompt returns "PONG" cleanly.

Suggested fix

One-line replacement at both call sites (preserving _default_headers as a fallback for OpenAI SDK versions where the attribute name differs):

- if hasattr(_routed_client, '_default_headers') and _routed_client._default_headers:
-     client_kwargs["default_headers"] = dict(_routed_client._default_headers)
+ _routed_headers = getattr(_routed_client, '_custom_headers', None) \
+     or getattr(_routed_client, '_default_headers', None)
+ if _routed_headers:
+     client_kwargs["default_headers"] = dict(_routed_headers)

Same shape for the fallback site at run_agent.py:1504-1505.

I'm happy to open a PR if that helps; otherwise the diff is above for any maintainer who'd like to land it.

Why this didn't surface in the test suite

tests/agent/test_codex_cloudflare_headers.py checks that _codex_cloudflare_headers() returns the right dict and that it's wired into resolve_provider_client. It doesn't assert that the kwargs passed to _create_openai_client actually carry those headers (i.e. it doesn't cover the cross-module hand-off). Adding an integration test that constructs _routed_client then asserts client_kwargs["default_headers"] contains originator would prevent regressions like this from being silent.

Side-effects I noticed while debugging

  • The _codex_cloudflare_headers helper exists in auxiliary_client.py only. Inlining or making it a public helper in a shared module would discourage future hand-off mismatches.
  • The cli flag --provider <p> requires --model <m> (or HERMES_INFERENCE_MODEL). If only --provider openai-codex is passed without a model, Hermes errors out before hitting any Codex code, so the bug is invisible from a one-line provider-only smoke test.

Environment

  • hermes --version: 0.12.0 (2026.4.30)
  • Project: /home/kabooy/.hermes/hermes-agent
  • Python: 3.11.15
  • OpenAI SDK: 2.32.0
  • OS: Ubuntu 24.04 (Hetzner VPS)
  • Auth: ChatGPT subscription OAuth via hermes auth add openai-codex --type oauth

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High — major feature broken, no workaroundcomp/agentCore agent loop, run_agent.py, prompt builderprovider/openaiOpenAI / Codex Responses APIsweeper:implemented-on-mainSweeper: behavior already present on current maintype/bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions