fix(anthropic): work around OAuth third-party billing-lane classifier (#15080)#24250
fix(anthropic): work around OAuth third-party billing-lane classifier (#15080)#24250thundron wants to merge 2 commits into
Conversation
|
One gap I noticed: the |
thank you for the heads up! I actually found some more things to adjust too thanks to that |
…h#15080) Anthropic's plan-vs-extra-usage classifier on OAuth (Pro/Max) requests returns HTTP 400 ("Third-party apps now draw from your extra usage…") when a request fingerprints as non-Claude-Code and the account has no overage credit. Bisection (NousResearch#15080) isolates two independent triggers: 1. Tool-name set matches hermes snake_case (terminal, read_file, session_search, …) rather than Claude Code's PascalCase canonicals (Bash, Read, Task, …). 2. Multi-block system prompt with hermes-flavored content (SOUL.md, AGENTS.md, memory). New agent/oauth_compat owns both mitigations behind a StealthMode enum (OFF | RENAME_ONLY | FULL_STEALTH) and a per-session ToolNameMap with forward+reverse mapping, idempotency, and collision handling. Default behavior is unchanged: mode starts OFF, error_classifier flags the specific 400 as FailoverReason.oauth_third_party_classifier, and run_agent escalates to FULL_STEALTH on first match and retries once (mirrors the existing oauth_long_context_beta_forbidden recovery). Accounts with overage credit never escalate and see zero behavior change. Config: agent.oauth_stealth: auto|on|off|rename_only|full_stealth in config.yaml (default: auto). 321 tests pass — 33 new in test_oauth_compat, 4 integration in test_anthropic_adapter, 5 classification in test_error_classifier. Signed-off-by: thundron <la@thundron.dev> --- Rebase note: ported onto upstream main after the run_agent.py refactor (~14k → ~4k lines). The original commit's seven run_agent.py hunks now live in: - agent/agent_init.py (init: oauth_stealth_mode, ToolNameMap) - agent/chat_completion_helpers.py (build_kwargs threading: main path, summary path, retry path) - agent/conversation_loop.py (retry-state flag, reactive recovery block, normalize_response oauth_tool_map threading on both main + length-truncation paths) An additional sibling site discovered during the port — the length- truncation recovery in conversation_loop.py that rebuilds the assistant message from the truncated response — also receives the oauth_tool_map so stealth-renamed tool names are reversed in the rebuilt continuation message. Without that, the next iteration's tool registry would not recognize the renamed tool. This site did not exist in the same form in the pre-refactor file; the fix is preserved by virtue of porting both normalize_response callsites. 321 of 321 tests in the directly-affected modules pass (test_oauth_compat, test_anthropic_adapter, test_error_classifier). Broader sweep: 0 regressions attributable to this port (10 of the 12 sweep failures pre-exist on origin/main; the remaining 2 pass in isolation and are pre-existing test-isolation issues).
…Research#15080) Empirical inspection of the Claude Code 2.1.143 Windows binary confirms CC does NOT send the fine-grained-tool-streaming-2025-05-14 beta header. Grepping the binary for interned beta strings shows every other beta CC uses as a plain string, but fine-grained-tool-streaming-2025-05-14 only appears inside a bundled documentation skill, which itself instructs: Remove the effort-2025-11-24 and fine-grained-tool-streaming-2025-05-14 beta headers (GA on 4.6) CC opts in to fine-grained tool streaming at the per-tool level via `eager_input_streaming: true` gated on CLAUDE_CODE_ENABLE_FINE_GRAINED_TOOL_STREAMING and the tengu_fgts feature flag, NOT via the global beta header. Sending the beta on OAuth requests is a fingerprint divergence from real CC and may contribute to the plan-vs-extra-usage classifier rejection already worked around in commit 041957564 (NousResearch#15080). Even if not an independent classifier trigger, exact CC beta-set parity is the whole point of the OAuth path. Scope: - Strip the beta only on the OAuth code path (is_oauth_request=True); x-api-key callers may still target older Claude (4.5/4.1) endpoints that benefit from the explicit opt-in. - Threaded through both build_anthropic_client and the fast-mode extra_headers override in build_anthropic_kwargs, otherwise the per-request fast-mode header would silently reintroduce the beta we stripped at client level. Tests (324 pass, +3 new): - test_oauth_strips_fine_grained_tool_streaming_beta (new) - test_api_key_still_sends_fine_grained_tool_streaming_beta (new) - test_fast_mode_oauth_strips_fine_grained_tool_streaming_beta (new) - Updated test_setup_token_uses_auth_token and test_oauth_drop_context_1m_beta_strips_only_1m to assert the beta is absent (was incorrectly asserted present). Credit: PR reviewer flagged this gap; this commit confirms the wider scope (not just stealth mode) using direct evidence from the CC binary.
0419575 to
168655a
Compare
note: I'm using this locally (both on Windows WSL2 + macOS); this is code generated by a Pi coding agent, that was fixed by Hermes initially, which then broke, and I made Pi fix-back Hermes. not sure how it happened, but it happened.
feel free to take inspiration, let me know things to change or just close it as noise
What does this PR do?
Adds a workaround for Anthropic's plan-vs-extra-usage classifier on OAuth (Pro/Max) requests. The classifier returns HTTP 400 ("Third-party apps now draw from your extra usage…") when a request fingerprints as a non-Claude-Code agent and the account has no overage credit configured. Issue #15080 has the original report and discussion; this PR addresses the two empirical triggers I bisected:
Renaming a single tool is sub-threshold; the whole set has to look canonical/neutral, AND the system prompt has to be the bare Claude Code identity line. With both mitigations the request passes the classifier on accounts that previously got 400 on every turn.
Default behavior is unchanged. Stealth mode starts off. The error classifier flags the specific 400 (FailoverReason.oauth_third_party_classifier), and run_agent escalates to full stealth on the first match and retries — same pattern as the existing
oauth_long_context_beta_forbidden recovery. Accounts with overage credit never escalate and see zero behavior change.
Related Issue
Fixes #15080
Type of Change
Changes Made
How to Test
On an affected account: first three return 200, the last returns 400 with the classifier error.
hermes -z "say hi in 3 words"Without this PR: API call failed after 3 retries: HTTP 400: Third-party apps…
With this PR: response text, plus a one-line stderr notice on first turn explaining the escalation to full stealth and the config key to skip the probe.
hermes -z "run uname and tell me the output" --yoloShould produce normal output, no tool not found errors.
pytest tests/agent/test_oauth_compat.py \ tests/agent/test_anthropic_adapter.py \ tests/agent/test_error_classifier.py -q321 tests, all pass on my setup.
Checklist
Code
Documentation & Housekeeping
────────────────────────────────────────────────────────────────────────────────
Disclosure
The bisection and patch were developed in a long debugging session assisted by an AI coding agent (Claude). I drove the investigation, made the architectural decisions (per-session map vs module-global, auto default with reactive recovery, narrow classifier rule), and own the code. Happy to discuss any line in review.