fix: sanitize thinking signatures on cross-provider model fallback#32407
fix: sanitize thinking signatures on cross-provider model fallback#32407tag-assistant wants to merge 1 commit intoopenclaw:mainfrom
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 17f35f61a8
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ? downgradeOpenAIFunctionCallReasoningPairs( | ||
| downgradeOpenAIReasoningBlocks(sanitizedCompactionUsage), | ||
| signatureIncompatible | ||
| ? downgradeOpenAIReasoningBlocks(sanitizedModelSwitch) | ||
| : sanitizedModelSwitch, |
There was a problem hiding this comment.
Run orphan-reasoning downgrade for all OpenAI responses runs
downgradeOpenAIReasoningBlocks is now gated on signatureIncompatible, so same-provider/same-API sessions skip this sanitizer entirely. That re-allows orphan OpenAI reasoning signatures (e.g. a trailing thinkingSignature JSON item without a following content item) to pass through unchanged, which can still trigger OpenAI Responses 400 transcript errors even when no cross-provider switch happened. This path was previously always sanitized for openai-responses/openai-codex-responses, so the new condition reintroduces a failure mode in normal same-provider continuations.
Useful? React with 👍 / 👎.
| sessionKey: sandboxSessionKey, | ||
| sessionId: params.sessionId, | ||
| runId: params.runId, | ||
| agentDir, |
There was a problem hiding this comment.
Pass runId into tool wrapper construction
This createOpenClawCodingTools call no longer forwards runId, so before_tool_call wrapping falls back to tool-call-id-only scoping (buildAdjustedParamsKey), which defeats per-run isolation when toolCallIds collide and drops run correlation data from hook payloads. The codebase already has explicit handling for runId-scoped collisions, so omitting it here can mix adjusted params across concurrent runs and make hook-driven auditing/debugging unreliable.
Useful? React with 👍 / 👎.
| sessionId: params.sessionId, | ||
| workspaceDir: resolvedWorkspace, | ||
| messageProvider: params.messageProvider ?? undefined, | ||
| trigger: params.trigger, | ||
| channelId: params.messageChannel ?? params.messageProvider ?? undefined, | ||
| }; |
There was a problem hiding this comment.
Preserve trigger and channelId in plugin hook context
The hook context built for model-resolution no longer includes trigger or channelId, even though PluginHookAgentContext defines them for hook consumers. Plugins that choose provider/model behavior based on run source (cron/heartbeat/user) or channel now always receive undefined in this phase, which silently regresses existing hook logic for embedded runs; the same omission also appears in the prompt-build hook context path.
Useful? React with 👍 / 👎.
Greptile SummaryThis PR adds cross-provider thinking/reasoning block sanitization to prevent 400 errors when model fallback causes provider-specific thinking signatures (Anthropic base64, OpenAI Key issues found:
Confidence Score: 2/5
Last reviewed commit: 17f35f6 |
| const sanitizedOpenAI = isOpenAIResponsesApi | ||
| ? downgradeOpenAIFunctionCallReasoningPairs( | ||
| downgradeOpenAIReasoningBlocks(sanitizedCompactionUsage), | ||
| signatureIncompatible | ||
| ? downgradeOpenAIReasoningBlocks(sanitizedModelSwitch) | ||
| : sanitizedModelSwitch, | ||
| ) | ||
| : sanitizedCompactionUsage; | ||
| : sanitizedModelSwitch; |
There was a problem hiding this comment.
downgradeOpenAIReasoningBlocks is now effectively unreachable
downgradeOpenAIReasoningBlocks is only called when signatureIncompatible is true. However, when signatureIncompatible is true, needsThinkingDowngrade is also true, so downgradeThinkingBlocksOnModelSwitch has already converted every signed thinking block (including OpenAI rs_*-signed ones) into plain text blocks. By the time downgradeOpenAIReasoningBlocks runs, there are no thinking blocks with OpenAI reasoning signatures left — making the call a no-op.
Before this PR, downgradeOpenAIReasoningBlocks was called unconditionally for every openai-responses session:
// Before (always ran for openai-responses):
const sanitizedOpenAI = isOpenAIResponsesApi
? downgradeOpenAIFunctionCallReasoningPairs(
downgradeOpenAIReasoningBlocks(sanitizedCompactionUsage),
)
: sanitizedCompactionUsage;Its purpose was to drop orphaned rs_*-signed reasoning blocks that trail an assistant turn with no following text item — a condition the OpenAI Responses API rejects with a 400 error. That cleanup now never happens for same-provider OpenAI sessions (the common case), re-introducing the exact class of 400 errors this PR sets out to fix.
The fix is to call downgradeOpenAIReasoningBlocks unconditionally for isOpenAIResponsesApi, independent of the cross-provider signatureIncompatible flag:
const sanitizedOpenAI = isOpenAIResponsesApi
? downgradeOpenAIFunctionCallReasoningPairs(
downgradeOpenAIReasoningBlocks(
signatureIncompatible
? sanitizedModelSwitch
: sanitizedModelSwitch, // <-- always pass through downgrade
),
)
: sanitizedModelSwitch;
Or more simply:
const sanitizedOpenAI = isOpenAIResponsesApi
? downgradeOpenAIFunctionCallReasoningPairs(
downgradeOpenAIReasoningBlocks(sanitizedModelSwitch),
)
: sanitizedModelSwitch;Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/google.ts
Line: 498-504
Comment:
**`downgradeOpenAIReasoningBlocks` is now effectively unreachable**
`downgradeOpenAIReasoningBlocks` is only called when `signatureIncompatible` is `true`. However, when `signatureIncompatible` is `true`, `needsThinkingDowngrade` is also `true`, so `downgradeThinkingBlocksOnModelSwitch` has *already* converted every signed thinking block (including OpenAI `rs_*`-signed ones) into plain text blocks. By the time `downgradeOpenAIReasoningBlocks` runs, there are no thinking blocks with OpenAI reasoning signatures left — making the call a no-op.
Before this PR, `downgradeOpenAIReasoningBlocks` was called **unconditionally** for every `openai-responses` session:
```typescript
// Before (always ran for openai-responses):
const sanitizedOpenAI = isOpenAIResponsesApi
? downgradeOpenAIFunctionCallReasoningPairs(
downgradeOpenAIReasoningBlocks(sanitizedCompactionUsage),
)
: sanitizedCompactionUsage;
```
Its purpose was to drop orphaned `rs_*`-signed reasoning blocks that trail an assistant turn with no following text item — a condition the OpenAI Responses API rejects with a 400 error. That cleanup now never happens for **same-provider OpenAI sessions** (the common case), re-introducing the exact class of 400 errors this PR sets out to fix.
The fix is to call `downgradeOpenAIReasoningBlocks` unconditionally for `isOpenAIResponsesApi`, independent of the cross-provider `signatureIncompatible` flag:
```
const sanitizedOpenAI = isOpenAIResponsesApi
? downgradeOpenAIFunctionCallReasoningPairs(
downgradeOpenAIReasoningBlocks(
signatureIncompatible
? sanitizedModelSwitch
: sanitizedModelSwitch, // <-- always pass through downgrade
),
)
: sanitizedModelSwitch;
```
Or more simply:
```typescript
const sanitizedOpenAI = isOpenAIResponsesApi
? downgradeOpenAIFunctionCallReasoningPairs(
downgradeOpenAIReasoningBlocks(sanitizedModelSwitch),
)
: sanitizedModelSwitch;
```
How can I resolve this? If you propose a fix, please make it concise.| if (strippedCount > 0) { | ||
| console.warn( | ||
| `[thinking-sig] Stripped ${strippedCount} non-round-trippable thinking signature(s) from session history`, | ||
| ); | ||
| } |
There was a problem hiding this comment.
Use project logger instead of console.warn
Both stripInvalidThinkingSignatures and downgradeThinkingBlocksOnModelSwitch use console.warn directly, while the rest of the codebase uses the structured log module (imported from ./logger.js). console.warn bypasses any log-level filtering, structured-logging sinks, or log routing the project relies on.
Consider importing and using log.warn(...) to match the surrounding code's logging conventions. The same applies to the identical console.warn call at line 406–410 in downgradeThinkingBlocksOnModelSwitch.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-helpers/openai.ts
Line: 322-326
Comment:
**Use project logger instead of `console.warn`**
Both `stripInvalidThinkingSignatures` and `downgradeThinkingBlocksOnModelSwitch` use `console.warn` directly, while the rest of the codebase uses the structured `log` module (imported from `./logger.js`). `console.warn` bypasses any log-level filtering, structured-logging sinks, or log routing the project relies on.
Consider importing and using `log.warn(...)` to match the surrounding code's logging conventions. The same applies to the identical `console.warn` call at line 406–410 in `downgradeThinkingBlocksOnModelSwitch`.
How can I resolve this? If you propose a fix, please make it concise.17f35f6 to
b9fe216
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b9fe216311
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ); | ||
| } | ||
| } | ||
| const compactResult = await compactEmbeddedPiSessionDirect({ |
There was a problem hiding this comment.
Pass observed overflow token count into compaction
Overflow recovery no longer extracts/passes the provider-reported token count into compaction; this call now omits currentTokenCount, so compaction loses the precise over-budget signal from errors like “277403 > 200000”. In that scenario compaction can under-trim and immediately re-overflow, turning a recoverable retry into a terminal context_overflow.
Useful? React with 👍 / 👎.
| : attempt.yieldDetected | ||
| ? "end_turn" | ||
| : (lastAssistant?.stopReason as string | undefined), | ||
| stopReason: attempt.clientToolCall ? "tool_calls" : undefined, |
There was a problem hiding this comment.
Emit end_turn stopReason for sessions_yield completions
This now reports stopReason only for clientToolCall and drops the previous yieldDetected -> "end_turn" mapping. sessions_yield sets yieldDetected (not clientToolCall), so successful yield exits now surface stopReason: undefined, which breaks lifecycle consumers that rely on end_turn to treat the parent run as cleanly finished and idle.
Useful? React with 👍 / 👎.
| } | ||
|
|
||
| const { model, error, authStorage, modelRegistry } = await resolveModelAsync( | ||
| const { model, error, authStorage, modelRegistry } = resolveModel( |
There was a problem hiding this comment.
Use async model resolution to run dynamic model prep hooks
Switching this call from resolveModelAsync to resolveModel skips provider prepareDynamicModel hooks (implemented in the async resolver path), so providers that require async priming before dynamic model resolution (for example OpenRouter capability preload) now run with stale/default model metadata. That can mis-size context budgets and feature flags on first use, leading to incorrect runtime behavior.
Useful? React with 👍 / 👎.
|
This pull request has been automatically marked as stale due to inactivity. |
|
Closing due to inactivity. |
Problem
When OpenClaw falls back between different model providers (e.g., Claude → GPT, or Claude → Gemini), the session history may contain thinking/reasoning blocks from the previous provider. These blocks use provider-specific formats:
thinkingcontent blocks withsignaturefieldsreasoningcontent typethoughtflag on partsWhen a session with Claude thinking blocks is sent to GPT or Gemini, the API rejects the request with a 400 error because it doesn't recognize the foreign thinking format. This causes production crashes during model fallback, which is a critical reliability issue since fallback is the primary resilience mechanism.
Solution
1. Cross-provider thinking block sanitization (
openai.ts)New
sanitizeOpenAISessionHistory()function that:thinkingblocks from assistant messages before sending to OpenAI2. Thinking block downgrade for Gemini (
google.ts)thought: trueparts when the target model doesn't support thinking3. Compact-time signature stripping (
compact.ts)signaturefields from thinking blocks4. Model capability detection (
run.ts,attempt.ts)supportsThinkingflag through the run pipelineTesting
sanitize-session-history.test.tscovering:Impact
Fixes silent failures during model fallback that could leave users with no response. This is a reliability fix — no new features, just making existing fallback work correctly.