Skip to content

fix(chat): forward extra_headers to all LLM calls in agentic pipeline#436

Merged
pancacake merged 1 commit into
HKUDS:devfrom
zuoliangyu:fix/chat-extra-headers
May 4, 2026
Merged

fix(chat): forward extra_headers to all LLM calls in agentic pipeline#436
pancacake merged 1 commit into
HKUDS:devfrom
zuoliangyu:fix/chat-extra-headers

Conversation

@zuoliangyu

Copy link
Copy Markdown
Contributor

Summary

The chat capability silently dropped extra_headers from the active LLM profile, because three downstream call sites in AgenticChatPipeline did not forward them:

  • _build_openai_client() constructed AsyncOpenAI / AsyncAzureOpenAI without default_headers, so custom headers required by some gateways (e.g. a User-Agent override to bypass WAF allowlists that block OpenAI/Python <ver>) were lost on the native-tool-calling path.
  • _stream_messages and _run_react_fallback invoked llm_stream() with model, api_key, base_url, binding but no extra_headers. The factory's _resolve_call_config treats this set of caller-provided fields as an explicit override and rebuilds LLMConfig with empty extra_headers, bypassing the catalog value already loaded by get_llm_config().

The fix pulls extra_headers from llm_config once in __init__ and forwards it from all three call sites.

Reproduce

  1. Configure an OpenAI-compatible gateway whose WAF rejects the default OpenAI/Python <ver> User-Agent (e.g. some commercial proxies front-running OpenAI).
  2. In the active LLM profile in model_catalog.json, set:
    "extra_headers": { "User-Agent": "Mozilla/5.0 ..." }
  3. Start a chat.

Before this PR — chat fails with:

Capability chat failed: [openai] Error: Your request was blocked.

After this PR — the override reaches the gateway and chat succeeds.

I verified the SDK behaviour with an isolated probe against the same gateway:

client construction result
AsyncOpenAI(api_key, base_url) (no override) 403 Your request was blocked.
AsyncOpenAI(api_key, base_url, default_headers={'User-Agent': 'Mozilla/5.0 ...'}) 200 OK

Test plan

  • Manual: gateway with UA WAF — chat returns 403 before, succeeds after.
  • Manual: gateway without UA WAF — no behaviour change (extra_headers is empty dict, falls through as None).
  • Existing test suite passes (please run on CI; my local environment is not set up to run the full Python test matrix).

Notes / not in this PR

While debugging I noticed two adjacent issues that I did not include here, to keep the diff minimal and reviewable:

  1. factory._resolve_call_config could fall back to the resolver-loaded extra_headers when the caller supplies model + api_key + base_url + binding without extra_headers. Today this path silently substitutes {}, which is what made the chat-side bug fatal. Fixing it would prevent any future agent that forgets to forward extra_headers from regressing the same way. Happy to send as a follow-up if reviewers prefer.
  2. When chat fails (any reason — UA block, model timeout, bad key), the session's backend status is left at running. The next time the user opens that session, UnifiedChatContext reads status === "running"isStreaming = true → the send button stays permanently disabled. New chat works around it but the state never recovers. This deserves its own PR with proper error-path coverage.

The chat capability silently dropped `extra_headers` from the active
LLM profile because three downstream call sites in `AgenticChatPipeline`
did not forward them:

- `_build_openai_client()` constructed `AsyncOpenAI` /
  `AsyncAzureOpenAI` without `default_headers`, so custom headers
  required by some gateways (e.g. a `User-Agent` override to bypass
  WAF allowlists that block `OpenAI/Python <ver>`) were lost on the
  native-tool-calling path.
- `_stream_messages` and `_run_react_fallback` invoked `llm_stream()`
  with `model`, `api_key`, `base_url`, `binding` but no
  `extra_headers`. The factory's `_resolve_call_config` treats this
  set of caller-provided fields as an explicit override and rebuilds
  `LLMConfig` with empty `extra_headers`, bypassing the catalog value
  loaded by `get_llm_config()`.

Pulls `extra_headers` from `llm_config` once in `__init__` and
forwards it from all three call sites.

Reproduce: configure an OpenAI-compatible gateway whose WAF rejects
the default `OpenAI/Python <ver>` User-Agent; set
`extra_headers: { "User-Agent": "Mozilla/5.0 ..." }` on the active
LLM profile in `model_catalog.json`. Before this change, chat returns
HTTP 403 "Your request was blocked." After, the override reaches the
gateway and chat succeeds.
@pancacake pancacake changed the base branch from main to dev May 4, 2026 03:52
@pancacake pancacake merged commit 5520e21 into HKUDS:dev May 4, 2026
@pancacake

Copy link
Copy Markdown
Collaborator

Thanks for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants