Skip to content

fix: harden ChatGPT Responses missing content-type streams#90487

Open
anyech wants to merge 1 commit into
openclaw:mainfrom
anyech:fix-chatgpt-responses-missing-content-type-sse-sniff
Open

fix: harden ChatGPT Responses missing content-type streams#90487
anyech wants to merge 1 commit into
openclaw:mainfrom
anyech:fix-chatgpt-responses-missing-content-type-sse-sniff

Conversation

@anyech

@anyech anyech commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Builds on #90399 and narrows the missing-Content-Type workaround to the native ChatGPT/Codex Responses stream path.

  • send Accept: text/event-stream for native ChatGPT/Codex Responses SDK stream requests, including the transport-aware alias path
  • only synthesize text/event-stream when the response is 2xx, has a body, targets the expected chatgpt.com/backend-api or /codex backend, and the body prefix sniffs as SSE
  • fail closed for missing-header JSON/HTML/unknown bodies and for non-native OpenAI-compatible providers

Why

#90399 covers the direct openai-chatgpt-responses API value, but the transport-aware stream path can route through openclaw-openai-responses-transport. Matching only the direct API value leaves that path without the proactive SSE Accept header.

The response-side guard also should not treat every missing Content-Type as acceptable. A missing header is only safe to normalize for the native ChatGPT/Codex Responses backend after the body confirms it is SSE.

Related: #90083. Current main already contains the central body-sniffing normalization for this missing-Content-Type SSE failure mode; this PR keeps the native ChatGPT/Codex SSE path working while tightening the missing-header normalization boundary and adding the scoped SSE Accept header. It does not claim coverage for unrelated payload-shape/provider failures.

Tests

  • pnpm exec vitest run --config test/vitest/vitest.agents.config.ts src/agents/provider-transport-fetch.test.ts src/agents/openai-transport-stream.test.ts
  • pnpm exec oxlint src/agents/openai-transport-stream.ts src/agents/openai-transport-stream.test.ts src/agents/provider-transport-fetch.ts src/agents/provider-transport-fetch.test.ts
  • pnpm tsgo:core
  • pnpm exec tsgo -p test/tsconfig/tsconfig.core.test.agents.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/core-test-agents.tsbuildinfo
  • pnpm exec oxfmt --check src/agents/openai-transport-stream.ts src/agents/openai-transport-stream.test.ts src/agents/provider-transport-fetch.ts src/agents/provider-transport-fetch.test.ts

Real behavior proof

  • Behavior addressed: Native ChatGPT/Codex Responses streaming through the transport-aware alias should send Accept: text/event-stream on the SDK stream request. When that native path receives an SSE body with a missing Content-Type, the SDK stream should still parse events successfully. Missing-header bodies that do not sniff as SSE should fail closed; this is intentional for non-SSE/native responses and for non-native provider routes.

  • Real setup tested: Source checkout at PR commit 90cbbeebc449 using this branch's actual OpenAI SDK path, buildGuardedModelFetch, and buildOpenAISdkRequestOptions output. Non-production local harness: a loopback CONNECT proxy routed https://chatgpt.com/backend-api/codex/responses to a local TLS server with a generated chatgpt.com certificate. This kept traffic off external endpoints and used no real provider credentials. The local harness disabled TLS verification only for the generated local certificate; this proof is about request headers, missing-header SSE normalization, and fail-closed body classification, not TLS trust behavior. Model route under test: provider=openai, api=openclaw-openai-responses-transport, baseUrl=https://chatgpt.com/backend-api/codex.

  • Exact steps or command run after this patch: Ran a temporary proof harness with node --import tsx from this PR branch. The harness generated a local certificate for chatgpt.com, started a loopback CONNECT proxy and local TLS server, created an OpenAI SDK client using this branch's guarded fetch, passed this branch's stream request options into client.responses.create({ stream: true }), and repeated the request with a non-SSE missing-header body to verify fail-closed behavior.

  • Evidence after fix: copied terminal output from the non-production proof harness:

    commit=90cbbeebc449
    real_setup=local CONNECT proxy + generated local TLS server for chatgpt.com SAN; TLS verification disabled only for this local proof harness
    model_api=openclaw-openai-responses-transport
    model_base_url=https://chatgpt.com/backend-api/codex
    sse_request=POST /backend-api/codex/responses HTTP/1.1
    sse_server_accept=text/event-stream
    sse_server_content_type=<missing>
    sse_sdk_result=ok
    sse_sdk_events=response.created | response.output_text.delta delta=proof-ok | response.completed
    unknown_request=POST /backend-api/codex/responses HTTP/1.1
    unknown_server_accept=text/event-stream
    unknown_server_content_type=<missing>
    unknown_sdk_result=name=Error causeName=ProviderHttpError causeCode=invalid_provider_content_type causeStatus=200 message=Connection error.
    
  • Observed result after fix: The transport-aware alias path sent Accept: text/event-stream on the actual SDK request. The SDK parsed a missing-Content-Type SSE response successfully after guarded-fetch normalization. A missing-Content-Type non-SSE body was rejected through ProviderHttpError with invalid_provider_content_type, confirming the stricter fail-closed behavior is intentional.

  • What was not tested: No live ChatGPT OAuth/provider endpoint was called. No production Gateway/runtime state, user session, or real credential was used. This proof does not claim coverage for arbitrary OpenAI-compatible providers; those routes intentionally remain fail-closed for missing-header non-native streams.

Send SSE Accept headers for native ChatGPT/Codex Responses stream
requests, including the transport-aware alias path.

Only normalize missing Content-Type responses for the native HTTPS
ChatGPT/Codex backend after the body sniffs as SSE. Keep JSON,
HTML, unknown bodies, and non-native providers fail-closed.
@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: M triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 5, 2026
@clawsweeper

clawsweeper Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 5, 2026, 6:03 AM ET / 10:03 UTC.

Summary
The PR adds stream-only SSE Accept headers for native ChatGPT/Codex Responses SDK calls and narrows missing Content-Type normalization to sniffed SSE from the native chatgpt.com backend.

PR surface: Source +45, Tests +124. Total +169 across 4 files.

Reproducibility: yes. The linked report gives concrete failing openclaw infer model run commands, and the PR body supplies an after-fix loopback SDK-stream harness showing the Accept header, missing-header SSE success, and non-SSE fail-closed result.

Review metrics: 1 noteworthy metric.

  • Stream content-type policy: 1 broad tolerance narrowed, 1 stream Accept header added. This is the maintainer-relevant behavior change that determines whether existing or near-term OpenAI-compatible stream routes remain tolerated.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Get explicit maintainer acceptance or rejection of the fail-closed missing-header policy before merge.

Risk before merge

  • [P1] The patch intentionally rejects missing-Content-Type SSE for non-native OpenAI-compatible providers and missing-header JSON-like native responses that current main would tolerate, so maintainers should explicitly accept that stricter provider-compatibility boundary before merge.

Maintainer options:

  1. Accept the stricter native-only boundary
    Maintainers can approve the fail-closed behavior because it prevents broad missing-header tolerance from shipping beyond native ChatGPT/Codex SSE streams.
  2. Preserve broader compatibility before merge
    If missing-header SSE from non-native OpenAI-compatible providers is an intended supported path, adjust the guard and tests to keep that behavior explicitly rather than rejecting it.
  3. Pause for live provider evidence
    If the native ChatGPT/Codex backend behavior is still uncertain, hold the PR for a redacted live OAuth/provider run before deciding the compatibility policy.

Next step before merge

  • [P1] The remaining action is maintainer compatibility judgment on the fail-closed policy, not a narrow automated repair.

Security
Cleared: No concrete security or supply-chain regression was found; the diff only changes provider stream request/response handling and colocated tests.

Review details

Best possible solution:

Land this shape only after maintainers explicitly accept the stricter missing-header compatibility boundary; otherwise keep native SSE support and choose a narrower owner-approved compatibility path.

Do we have a high-confidence way to reproduce the issue?

Yes. The linked report gives concrete failing openclaw infer model run commands, and the PR body supplies an after-fix loopback SDK-stream harness showing the Accept header, missing-header SSE success, and non-SSE fail-closed result.

Is this the best way to solve the issue?

Yes, with a maintainer-policy caveat. The request-options and guarded-fetch boundary is the right layer for this SDK stream issue, but maintainers still need to accept the stricter compatibility policy before merge.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 1a3ce7c2a8da.

Label changes

Label changes:

  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal output from a real SDK-path loopback CONNECT/TLS harness showing the new header, successful missing-header SSE parsing, and fail-closed non-SSE behavior.

Label justifications:

  • P1: The PR targets a broken OpenAI/ChatGPT Responses streaming path affecting real model runs for gpt-5.4/gpt-5.5 users.
  • merge-risk: 🚨 compatibility: Merging changes current-main missing-header stream tolerance and can make non-native or JSON-like missing-header provider responses fail closed.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes after-fix terminal output from a real SDK-path loopback CONNECT/TLS harness showing the new header, successful missing-header SSE parsing, and fail-closed non-SSE behavior.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal output from a real SDK-path loopback CONNECT/TLS harness showing the new header, successful missing-header SSE parsing, and fail-closed non-SSE behavior.
Evidence reviewed

PR surface:

Source +45, Tests +124. Total +169 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 53 8 +45
Tests 2 163 39 +124
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 216 47 +169

What I checked:

  • Repository policy read: The full root policy and scoped src/agents guide were read; the root policy makes provider routing/fallback and stricter fail-closed behavior compatibility-sensitive, and the scoped guide applies to agent runtime/test changes. (AGENTS.md:12, 1a3ce7c2a8da)
  • PR stream-header change: The PR passes { stream: true } into the SDK request options and adds Accept: text/event-stream only when the model is the native OpenAI/Codex Responses backend. (src/agents/openai-transport-stream.ts:1846, 90cbbeebc449)
  • PR missing-header boundary: The PR allows missing Content-Type normalization only for openai ChatGPT/Codex Responses models on https://chatgpt.com/backend-api* when the body sniffs as SSE, and otherwise throws invalid_provider_content_type. (src/agents/provider-transport-fetch.ts:237, 90cbbeebc449)
  • Regression coverage: The PR adds coverage for direct and transport-aware native ChatGPT/Codex SSE success, missing-header JSON-like rejection, non-native missing-header SSE rejection, wrong-base-url rejection, and insecure native URL rejection. (src/agents/provider-transport-fetch.test.ts:245, 90cbbeebc449)
  • Current main comparison: Current main already contains broad missing-header stream sniffing that synthesizes SSE for SSE bodies and JSON content type for JSON-like bodies before this PR narrows that current-main behavior. (src/agents/provider-transport-fetch.ts:314, 82710b4f1f10)
  • OpenAI SDK contract: The inspected OpenAI SDK 6.39.1 package accepts per-request headers, merges them after default headers, and sends streamed responses through Stream.fromSSEResponse, whose parser consumes SSE bytes rather than validating the response Content-Type itself. (openai@6.39.1/package/src/internal/request-options.ts:12)

Likely related people:

  • steipete: Git blame and log show Peter Steinberger authored the current provider transport files, including the missing-header stream normalization and OpenAI/Codex transport routing surfaces that this PR changes. (role: recent area contributor; confidence: high; commits: 82710b4f1f10; files: src/agents/provider-transport-fetch.ts, src/agents/openai-transport-stream.ts, src/agents/provider-transport-stream.ts)
  • Vincent Koc: The latest shipped release tag points at Vincent Koc's release commit, which is useful for deciding whether this current-main-only compatibility boundary has shipped. (role: release provenance contact; confidence: medium; commits: 2e08f0f4221f; files: CHANGELOG.md)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 5, 2026
@anyech

anyech commented Jun 5, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 5, 2026
@anyech

anyech commented Jun 5, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 5, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 5, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 5, 2026
@backthatup

Copy link
Copy Markdown

Adding a macOS repro that appears to match the invalid_provider_content_type / OpenClaw Default + OpenAI OAuth path.

Environment:

  • OpenClaw 2026.6.1 (2e08f0f)
  • macOS global install, local gateway reachable
  • OpenAI OAuth profile valid/not expired
  • No API-key fallback
  • No custom models.providers.openai.baseUrl
  • models.providers restored to {}
  • Primary model: openai/gpt-5.5

Failing case:

openclaw infer model run --gateway --model openai/gpt-5.5 --prompt 'Reply with exactly: OPENCLAW-55-OK' --json

Result:

GatewayClientRequestError: FailoverError: LLM request failed: network connection error.

Earlier probe/gateway logs for the same failure showed:

[openai-transport] [responses] error provider=openai api=openai-chatgpt-responses model=gpt-5.5
causeName=ProviderHttpError causeCode=invalid_provider_content_type message=Connection error.

The failed session resolves as:

selectedModel: openai/gpt-5.5
runtime: OpenClaw Default

Control case:

On the same host, same OAuth account, and same canonical openai/gpt-5.5 model ref, routing through agentRuntime.id="codex" succeeds.

So this does not appear to be expired OAuth, gateway offline, bad model ref, proxy/baseUrl override, or generic OpenAI OAuth failure. It looks specific to:

openai/gpt-5.5 + agentRuntime.id="openclaw" + OpenAI OAuth
-> openai-chatgpt-responses / OpenClaw Default transport
-> invalid_provider_content_type / network connection error

I also tested provider-scoped runtime policy:

"models": {
  "providers": {
    "openai": {
      "agentRuntime": { "id": "openclaw" }
    }
  }
}

openclaw config validate passed, gateway restart succeeded, but the same tiny model-run smoke still failed with the same network connection error. I rolled that config experiment back.

This looks aligned with the fix in PR #90205, where missing content-type on valid streamed ChatGPT/Codex responses is tolerated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants