Skip to content

fix: strip prompt_cache_key for providers that reject unknown fields#49727

Closed
ShaunTsai wants to merge 1 commit into
openclaw:mainfrom
ShaunTsai:fix/strip-prompt-cache-key-unsupported-providers
Closed

fix: strip prompt_cache_key for providers that reject unknown fields#49727
ShaunTsai wants to merge 1 commit into
openclaw:mainfrom
ShaunTsai:fix/strip-prompt-cache-key-unsupported-providers

Conversation

@ShaunTsai

@ShaunTsai ShaunTsai commented Mar 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: Volcano Engine DeepSeek (and other non-OpenAI providers using the openai-responses API) returns HTTP 400 unknown field "prompt_cache_key" because the pi-ai library unconditionally injects prompt_cache_key and prompt_cache_retention into OpenAI Responses request bodies.
  • Why it matters: users configuring Volcano Engine models cannot use them at all — every request fails with a 400.
  • What changed: added an onPayload wrapper that strips prompt_cache_key and prompt_cache_retention from the request body for providers not in the known-supported set (openai, openai-codex, opencode, azure-openai-responses, github-copilot).
  • What did NOT change: prompt caching behavior for OpenAI, Codex, Azure, and GitHub Copilot providers. Anthropic caching uses a different mechanism (cacheRetention option) and is unaffected. The existing createBedrockNoCacheWrapper for non-Anthropic Bedrock models is also unchanged.

Change Type

  • Bug fix

Scope

  • Gateway / orchestration

Linked Issue

User-visible / Behavior Changes

Volcano Engine DeepSeek (and other non-OpenAI providers using openai-responses API) will no longer fail with HTTP 400 on unknown prompt_cache_key field.

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No

Evidence

  • Code inspection: traced prompt_cache_key injection from pi-ai openai-responses.js:140 through to the HTTP request body; confirmed the field is only meaningful for OpenAI's own API

Human Verification (required)

  • Verified scenarios: traced the full code path from applyExtraParamsToAgent through streamWithPayloadPatch to confirm the wrapper correctly strips the fields before the request is sent
  • Edge cases checked: OpenAI/Codex/Azure providers are in the allowlist and keep prompt caching; Anthropic uses a separate caching mechanism; existing Bedrock no-cache wrapper is preserved
  • What I did not verify: live Volcano Engine API call (no credentials available); full pnpm check currently fails on existing main typing debt unrelated to this PR

AI Disclosure

  • AI-assisted (Kiro CLI)
  • I understand what the code does

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No

Failure Recovery (if this breaks)

  • How to disable/revert: revert this commit
  • Files/config to restore: src/agents/pi-embedded-runner/extra-params.ts
  • Known bad symptoms: if a provider that actually needs prompt_cache_key is missing from PROMPT_CACHE_KEY_PROVIDERS, prompt caching would silently stop working for that provider

Risks and Mitigations

  • Risk: a new provider that supports prompt_cache_key would need to be added to the allowlist.
    • Mitigation: the set is explicit and easy to extend; prompt caching is an optimization, not a correctness requirement, so missing it degrades performance but doesn't break functionality.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S labels Mar 18, 2026
@greptile-apps

greptile-apps Bot commented Mar 18, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a real HTTP 400 failure for Volcano Engine DeepSeek (and any other non-OpenAI provider using the openai-responses API) by stripping prompt_cache_key and prompt_cache_retention — fields unconditionally injected by pi-ai — from request payloads for providers not in a new explicit allowlist (openai, openai-codex, opencode, azure-openai-responses, github-copilot).

The implementation reuses the existing streamWithPayloadPatch / onPayload interception pattern correctly: delete on a missing key is a no-op, so Anthropic and other non-OpenAI API types are unaffected in practice, and the workspaceDir threading through createAnthropicToolPayloadCompatibilityWrapper (addressed in prior review threads) is preserved.

Key observations:

  • The new createStripPromptCacheFieldsWrapper does not include the model.api !== "openai-responses" short-circuit that sibling wrappers like createParallelToolCallsWrapper use. Since prompt_cache_key is only injected for openai-responses calls, this is harmless but inconsistent with the established pattern.
  • No unit tests were added for the new stripping behaviour. Given the existing per-feature test files in extra-params.*.test.ts, a focused test asserting stripping for non-listed providers and preservation for listed providers would improve confidence and catch future regressions.

Confidence Score: 4/5

  • Safe to merge — the fix is narrowly scoped, uses the established onPayload interception pattern, and is backward compatible for all existing providers.
  • The logic is correct and the regression surface is small: delete on a missing key is a no-op, so only providers that actually use openai-responses with an unlisted provider ID are affected. The two gaps (missing API-type guard and no tests) are style/coverage concerns, not functional bugs.
  • src/agents/pi-embedded-runner/extra-params.ts — verify the PROMPT_CACHE_KEY_PROVIDERS allowlist is complete if any new openai-responses-backed provider is added in the future.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/extra-params.ts
Line: 44-51

Comment:
**Missing API-type guard, unlike sibling wrappers**

`createParallelToolCallsWrapper` (line 172) short-circuits immediately when `model.api` is neither `"openai-completions"` nor `"openai-responses"`:

```ts
if (model.api !== "openai-completions" && model.api !== "openai-responses") {
  return underlying(model, context, options);
}
```

`createStripPromptCacheFieldsWrapper` has no such guard. `prompt_cache_key` / `prompt_cache_retention` are only injected by pi-ai for `openai-responses` payloads, so the `delete` calls are no-ops for every other API type — but the wrapper still installs an extra `onPayload` interceptor on every Anthropic, Google, Moonshot, etc. call made through providers not in the allowlist.

Adding the guard would be consistent with the existing pattern and make the intent explicit:

```suggestion
function createStripPromptCacheFieldsWrapper(baseStreamFn: StreamFn | undefined): StreamFn {
  const underlying = baseStreamFn ?? streamSimple;
  return (model, context, options) => {
    if (model.api !== "openai-responses") {
      return underlying(model, context, options);
    }
    return streamWithPayloadPatch(underlying, model, context, options, (payload) => {
      delete payload.prompt_cache_key;
      delete payload.prompt_cache_retention;
    });
  };
}
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/extra-params.ts
Line: 248-250

Comment:
**No test coverage for the new stripping logic**

The `extra-params.*.test.ts` suite is fairly comprehensive — there are dedicated files for OpenAI, Google, xAI, OpenRouter cache-control, and cache-retention defaults. No test file was added (or updated) to assert that `prompt_cache_key` and `prompt_cache_retention` are stripped for unlisted providers (e.g. a "volcano" or "custom-openai-responses" provider), and that they are _kept_ for providers in `PROMPT_CACHE_KEY_PROVIDERS`.

A failing assertion here would surface if the allowlist accidentally drops a supported provider or if `streamWithPayloadPatch` changes its interception order.

How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: "fix: strip prompt_ca..."

Comment thread src/agents/pi-embedded-runner/extra-params.ts
Comment thread src/agents/pi-embedded-runner/extra-params.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 752dc1fbe6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/extra-params.ts Outdated
@ShaunTsai ShaunTsai closed this Mar 18, 2026
@ShaunTsai ShaunTsai force-pushed the fix/strip-prompt-cache-key-unsupported-providers branch from 752dc1f to 4a44ca8 Compare March 18, 2026 13:36
The pi-ai library unconditionally adds prompt_cache_key and
prompt_cache_retention to OpenAI Responses API request bodies.
Providers like Volcano Engine DeepSeek reject these unknown fields
with HTTP 400.

Strip both fields via an onPayload wrapper for any provider not in
the known-supported set (openai, openai-codex, opencode,
azure-openai-responses, github-copilot).

Fixes openclaw#48155
@ShaunTsai ShaunTsai reopened this Mar 18, 2026
@ShaunTsai

Copy link
Copy Markdown
Contributor Author

Review Conversations Status

All bot review comments have been addressed:

greptile-apps[bot] — workspaceDir silently dropped (P1)

Resolved — The initial push was based on a stale copy of extra-params.ts. After rebasing on latest main in 672148d5, workspaceDir is preserved in the function signature (line 205) and correctly forwarded to createAnthropicToolPayloadCompatibilityWrapper (line 269).

greptile-apps[bot] — workspaceDir no longer forwarded to Anthropic tool-payload wrapper (P1)

Resolved — Same rebase fix. The call now correctly passes { config: cfg, workspaceDir }.

chatgpt-codex-connector[bot] — Restore resolver options for Anthropic payload compatibility (P1)

Resolved — Same root cause and fix as above.

Verification

  • Current diff is 1 commit ahead of main, 0 behind, 1 file changed
  • workspaceDir confirmed present at line 205 (signature) and line 269 (forwarded)
  • The only change is the addition of createStripPromptCacheFieldsWrapper + PROMPT_CACHE_KEY_PROVIDERS set

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 672148d51e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +37 to +41
"openai",
"openai-codex",
"opencode",
"azure-openai-responses",
"github-copilot",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add azure-openai to prompt cache allowlist

applyExtraParamsToAgent now strips prompt_cache_key/prompt_cache_retention for every provider not listed here, but azure-openai is a supported OpenAI Responses provider in this codebase (OPENAI_RESPONSES_PROVIDERS includes it in openai-stream-wrappers.ts). As written, Azure OpenAI models configured under provider azure-openai will silently lose prompt-cache fields on every request, which regresses caching behavior for that provider.

Useful? React with 👍 / 👎.

Comment on lines +44 to +51
function createStripPromptCacheFieldsWrapper(baseStreamFn: StreamFn | undefined): StreamFn {
const underlying = baseStreamFn ?? streamSimple;
return (model, context, options) =>
streamWithPayloadPatch(underlying, model, context, options, (payload) => {
delete payload.prompt_cache_key;
delete payload.prompt_cache_retention;
});
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing API-type guard, unlike sibling wrappers

createParallelToolCallsWrapper (line 172) short-circuits immediately when model.api is neither "openai-completions" nor "openai-responses":

if (model.api !== "openai-completions" && model.api !== "openai-responses") {
  return underlying(model, context, options);
}

createStripPromptCacheFieldsWrapper has no such guard. prompt_cache_key / prompt_cache_retention are only injected by pi-ai for openai-responses payloads, so the delete calls are no-ops for every other API type — but the wrapper still installs an extra onPayload interceptor on every Anthropic, Google, Moonshot, etc. call made through providers not in the allowlist.

Adding the guard would be consistent with the existing pattern and make the intent explicit:

Suggested change
function createStripPromptCacheFieldsWrapper(baseStreamFn: StreamFn | undefined): StreamFn {
const underlying = baseStreamFn ?? streamSimple;
return (model, context, options) =>
streamWithPayloadPatch(underlying, model, context, options, (payload) => {
delete payload.prompt_cache_key;
delete payload.prompt_cache_retention;
});
}
function createStripPromptCacheFieldsWrapper(baseStreamFn: StreamFn | undefined): StreamFn {
const underlying = baseStreamFn ?? streamSimple;
return (model, context, options) => {
if (model.api !== "openai-responses") {
return underlying(model, context, options);
}
return streamWithPayloadPatch(underlying, model, context, options, (payload) => {
delete payload.prompt_cache_key;
delete payload.prompt_cache_retention;
});
};
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/extra-params.ts
Line: 44-51

Comment:
**Missing API-type guard, unlike sibling wrappers**

`createParallelToolCallsWrapper` (line 172) short-circuits immediately when `model.api` is neither `"openai-completions"` nor `"openai-responses"`:

```ts
if (model.api !== "openai-completions" && model.api !== "openai-responses") {
  return underlying(model, context, options);
}
```

`createStripPromptCacheFieldsWrapper` has no such guard. `prompt_cache_key` / `prompt_cache_retention` are only injected by pi-ai for `openai-responses` payloads, so the `delete` calls are no-ops for every other API type — but the wrapper still installs an extra `onPayload` interceptor on every Anthropic, Google, Moonshot, etc. call made through providers not in the allowlist.

Adding the guard would be consistent with the existing pattern and make the intent explicit:

```suggestion
function createStripPromptCacheFieldsWrapper(baseStreamFn: StreamFn | undefined): StreamFn {
  const underlying = baseStreamFn ?? streamSimple;
  return (model, context, options) => {
    if (model.api !== "openai-responses") {
      return underlying(model, context, options);
    }
    return streamWithPayloadPatch(underlying, model, context, options, (payload) => {
      delete payload.prompt_cache_key;
      delete payload.prompt_cache_retention;
    });
  };
}
```

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +248 to +250
if (!PROMPT_CACHE_KEY_PROVIDERS.has(provider)) {
agent.streamFn = createStripPromptCacheFieldsWrapper(agent.streamFn);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 No test coverage for the new stripping logic

The extra-params.*.test.ts suite is fairly comprehensive — there are dedicated files for OpenAI, Google, xAI, OpenRouter cache-control, and cache-retention defaults. No test file was added (or updated) to assert that prompt_cache_key and prompt_cache_retention are stripped for unlisted providers (e.g. a "volcano" or "custom-openai-responses" provider), and that they are kept for providers in PROMPT_CACHE_KEY_PROVIDERS.

A failing assertion here would surface if the allowlist accidentally drops a supported provider or if streamWithPayloadPatch changes its interception order.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/extra-params.ts
Line: 248-250

Comment:
**No test coverage for the new stripping logic**

The `extra-params.*.test.ts` suite is fairly comprehensive — there are dedicated files for OpenAI, Google, xAI, OpenRouter cache-control, and cache-retention defaults. No test file was added (or updated) to assert that `prompt_cache_key` and `prompt_cache_retention` are stripped for unlisted providers (e.g. a "volcano" or "custom-openai-responses" provider), and that they are _kept_ for providers in `PROMPT_CACHE_KEY_PROVIDERS`.

A failing assertion here would surface if the allowlist accidentally drops a supported provider or if `streamWithPayloadPatch` changes its interception order.

How can I resolve this? If you propose a fix, please make it concise.

@ShaunTsai

Copy link
Copy Markdown
Contributor Author

Closing in favor of a cleaner approach. The allowlist-based fix duplicates provider knowledge already encoded in openai-stream-wrappers.ts. The replacement PR will fold the stripping logic into createOpenAIResponsesContextManagementWrapper using the existing isDirectOpenAIBaseUrl() check — no new provider lists, no new files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: prompt_cache_key not supported by Volcano Engine DeepSeek

1 participant