Skip to content

fix: sanitize thinking signatures on cross-provider model fallback#32407

Closed
tag-assistant wants to merge 1 commit intoopenclaw:mainfrom
tag-assistant:fix/thinking-signature-sanitization
Closed

fix: sanitize thinking signatures on cross-provider model fallback#32407
tag-assistant wants to merge 1 commit intoopenclaw:mainfrom
tag-assistant:fix/thinking-signature-sanitization

Conversation

@tag-assistant
Copy link
Copy Markdown
Contributor

Problem

When OpenClaw falls back between different model providers (e.g., Claude → GPT, or Claude → Gemini), the session history may contain thinking/reasoning blocks from the previous provider. These blocks use provider-specific formats:

  • Claude: thinking content blocks with signature fields
  • OpenAI: reasoning content type
  • Google: thought flag on parts

When a session with Claude thinking blocks is sent to GPT or Gemini, the API rejects the request with a 400 error because it doesn't recognize the foreign thinking format. This causes production crashes during model fallback, which is a critical reliability issue since fallback is the primary resilience mechanism.

Solution

1. Cross-provider thinking block sanitization (openai.ts)

New sanitizeOpenAISessionHistory() function that:

  • Strips Claude-style thinking blocks from assistant messages before sending to OpenAI
  • Removes empty content arrays that result from stripping
  • Preserves all non-thinking content intact

2. Thinking block downgrade for Gemini (google.ts)

  • Strips thought: true parts when the target model doesn't support thinking
  • Converts thinking-annotated parts to plain text when possible

3. Compact-time signature stripping (compact.ts)

  • During session compaction, strips signature fields from thinking blocks
  • Signatures are ephemeral and become invalid after the response, but were being persisted in session history

4. Model capability detection (run.ts, attempt.ts)

  • Passes supportsThinking flag through the run pipeline
  • Uses it to conditionally sanitize history before API calls

Testing

  • 328+ test cases in sanitize-session-history.test.ts covering:
    • Claude → OpenAI fallback (thinking blocks stripped)
    • Claude → Gemini fallback (thought parts handled)
    • Signature stripping during compaction
    • Mixed content preservation (text + thinking → text only)
    • Edge cases: empty thinking blocks, nested content, multiple thinking blocks

Impact

Fixes silent failures during model fallback that could leave users with no response. This is a reliability fix — no new features, just making existing fallback work correctly.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 17f35f61a8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines 499 to +502
? downgradeOpenAIFunctionCallReasoningPairs(
downgradeOpenAIReasoningBlocks(sanitizedCompactionUsage),
signatureIncompatible
? downgradeOpenAIReasoningBlocks(sanitizedModelSwitch)
: sanitizedModelSwitch,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Run orphan-reasoning downgrade for all OpenAI responses runs

downgradeOpenAIReasoningBlocks is now gated on signatureIncompatible, so same-provider/same-API sessions skip this sanitizer entirely. That re-allows orphan OpenAI reasoning signatures (e.g. a trailing thinkingSignature JSON item without a following content item) to pass through unchanged, which can still trigger OpenAI Responses 400 transcript errors even when no cross-provider switch happened. This path was previously always sanitized for openai-responses/openai-codex-responses, so the new condition reintroduces a failure mode in normal same-provider continuations.

Useful? React with 👍 / 👎.

Comment on lines 646 to 648
sessionKey: sandboxSessionKey,
sessionId: params.sessionId,
runId: params.runId,
agentDir,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass runId into tool wrapper construction

This createOpenClawCodingTools call no longer forwards runId, so before_tool_call wrapping falls back to tool-call-id-only scoping (buildAdjustedParamsKey), which defeats per-run isolation when toolCallIds collide and drops run correlation data from hook payloads. The codebase already has explicit handling for runId-scoped collisions, so omitting it here can mix adjusted params across concurrent runs and make hook-driven auditing/debugging unreliable.

Useful? React with 👍 / 👎.

Comment on lines 263 to 266
sessionId: params.sessionId,
workspaceDir: resolvedWorkspace,
messageProvider: params.messageProvider ?? undefined,
trigger: params.trigger,
channelId: params.messageChannel ?? params.messageProvider ?? undefined,
};
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve trigger and channelId in plugin hook context

The hook context built for model-resolution no longer includes trigger or channelId, even though PluginHookAgentContext defines them for hook consumers. Plugins that choose provider/model behavior based on run source (cron/heartbeat/user) or channel now always receive undefined in this phase, which silently regresses existing hook logic for embedded runs; the same omission also appears in the prompt-build hook context path.

Useful? React with 👍 / 👎.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Mar 3, 2026

Greptile Summary

This PR adds cross-provider thinking/reasoning block sanitization to prevent 400 errors when model fallback causes provider-specific thinking signatures (Anthropic base64, OpenAI rs_* JSON, or field-name artifacts) to be replayed against a different provider. It introduces stripInvalidThinkingSignatures for field-name artifacts, downgradeThinkingBlocksOnModelSwitch for cross-provider history rewriting, and SDK_MANAGED_TOKEN support for GitHub Copilot auth.

Key issues found:

  • Critical regression in google.ts: downgradeOpenAIReasoningBlocks is now gated on signatureIncompatible, but it was previously called unconditionally for all openai-responses API sessions. When signatureIncompatible is true, downgradeThinkingBlocksOnModelSwitch already converts all signed thinking blocks to text, making the conditional downgradeOpenAIReasoningBlocks call a no-op. For the common case of staying on the same OpenAI provider (signatureIncompatible = false), orphaned trailing rs_* reasoning blocks are no longer cleaned up — reintroducing the 400 errors that downgradeOpenAIReasoningBlocks was originally added to prevent.
  • Minor style issue in openai.ts: New functions stripInvalidThinkingSignatures and downgradeThinkingBlocksOnModelSwitch use console.warn instead of the project's structured log module used throughout the rest of the codebase.

Confidence Score: 2/5

  • Not safe to merge as-is — the conditional downgradeOpenAIReasoningBlocks change regresses same-provider OpenAI Responses API reliability
  • The core cross-provider sanitization logic (new stripInvalidThinkingSignatures and downgradeThinkingBlocksOnModelSwitch functions) is sound and well-tested. However, the restructuring of the sanitizedOpenAI pipeline in google.ts inadvertently makes downgradeOpenAIReasoningBlocks unreachable in any meaningful scenario. That function existed to prevent 400 errors from orphaned OpenAI reasoning items in normal (non-provider-switching) sessions — skipping it regresses that protection for the common case.
  • src/agents/pi-embedded-runner/google.ts — the sanitizedOpenAI construction block (around line 498) requires the downgradeOpenAIReasoningBlocks call to be made unconditionally for isOpenAIResponsesApi, not gated on signatureIncompatible.

Last reviewed commit: 17f35f6

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8 files reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

Comment on lines 498 to +504
const sanitizedOpenAI = isOpenAIResponsesApi
? downgradeOpenAIFunctionCallReasoningPairs(
downgradeOpenAIReasoningBlocks(sanitizedCompactionUsage),
signatureIncompatible
? downgradeOpenAIReasoningBlocks(sanitizedModelSwitch)
: sanitizedModelSwitch,
)
: sanitizedCompactionUsage;
: sanitizedModelSwitch;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

downgradeOpenAIReasoningBlocks is now effectively unreachable

downgradeOpenAIReasoningBlocks is only called when signatureIncompatible is true. However, when signatureIncompatible is true, needsThinkingDowngrade is also true, so downgradeThinkingBlocksOnModelSwitch has already converted every signed thinking block (including OpenAI rs_*-signed ones) into plain text blocks. By the time downgradeOpenAIReasoningBlocks runs, there are no thinking blocks with OpenAI reasoning signatures left — making the call a no-op.

Before this PR, downgradeOpenAIReasoningBlocks was called unconditionally for every openai-responses session:

// Before (always ran for openai-responses):
const sanitizedOpenAI = isOpenAIResponsesApi
  ? downgradeOpenAIFunctionCallReasoningPairs(
      downgradeOpenAIReasoningBlocks(sanitizedCompactionUsage),
    )
  : sanitizedCompactionUsage;

Its purpose was to drop orphaned rs_*-signed reasoning blocks that trail an assistant turn with no following text item — a condition the OpenAI Responses API rejects with a 400 error. That cleanup now never happens for same-provider OpenAI sessions (the common case), re-introducing the exact class of 400 errors this PR sets out to fix.

The fix is to call downgradeOpenAIReasoningBlocks unconditionally for isOpenAIResponsesApi, independent of the cross-provider signatureIncompatible flag:

  const sanitizedOpenAI = isOpenAIResponsesApi
    ? downgradeOpenAIFunctionCallReasoningPairs(
        downgradeOpenAIReasoningBlocks(
          signatureIncompatible
            ? sanitizedModelSwitch
            : sanitizedModelSwitch,   // <-- always pass through downgrade
        ),
      )
    : sanitizedModelSwitch;

Or more simply:

  const sanitizedOpenAI = isOpenAIResponsesApi
    ? downgradeOpenAIFunctionCallReasoningPairs(
        downgradeOpenAIReasoningBlocks(sanitizedModelSwitch),
      )
    : sanitizedModelSwitch;
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/google.ts
Line: 498-504

Comment:
**`downgradeOpenAIReasoningBlocks` is now effectively unreachable**

`downgradeOpenAIReasoningBlocks` is only called when `signatureIncompatible` is `true`. However, when `signatureIncompatible` is `true`, `needsThinkingDowngrade` is also `true`, so `downgradeThinkingBlocksOnModelSwitch` has *already* converted every signed thinking block (including OpenAI `rs_*`-signed ones) into plain text blocks. By the time `downgradeOpenAIReasoningBlocks` runs, there are no thinking blocks with OpenAI reasoning signatures left — making the call a no-op.

Before this PR, `downgradeOpenAIReasoningBlocks` was called **unconditionally** for every `openai-responses` session:

```typescript
// Before (always ran for openai-responses):
const sanitizedOpenAI = isOpenAIResponsesApi
  ? downgradeOpenAIFunctionCallReasoningPairs(
      downgradeOpenAIReasoningBlocks(sanitizedCompactionUsage),
    )
  : sanitizedCompactionUsage;
```

Its purpose was to drop orphaned `rs_*`-signed reasoning blocks that trail an assistant turn with no following text item — a condition the OpenAI Responses API rejects with a 400 error. That cleanup now never happens for **same-provider OpenAI sessions** (the common case), re-introducing the exact class of 400 errors this PR sets out to fix.

The fix is to call `downgradeOpenAIReasoningBlocks` unconditionally for `isOpenAIResponsesApi`, independent of the cross-provider `signatureIncompatible` flag:

```
  const sanitizedOpenAI = isOpenAIResponsesApi
    ? downgradeOpenAIFunctionCallReasoningPairs(
        downgradeOpenAIReasoningBlocks(
          signatureIncompatible
            ? sanitizedModelSwitch
            : sanitizedModelSwitch,   // <-- always pass through downgrade
        ),
      )
    : sanitizedModelSwitch;
```

Or more simply:
```typescript
  const sanitizedOpenAI = isOpenAIResponsesApi
    ? downgradeOpenAIFunctionCallReasoningPairs(
        downgradeOpenAIReasoningBlocks(sanitizedModelSwitch),
      )
    : sanitizedModelSwitch;
```

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +322 to +326
if (strippedCount > 0) {
console.warn(
`[thinking-sig] Stripped ${strippedCount} non-round-trippable thinking signature(s) from session history`,
);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use project logger instead of console.warn

Both stripInvalidThinkingSignatures and downgradeThinkingBlocksOnModelSwitch use console.warn directly, while the rest of the codebase uses the structured log module (imported from ./logger.js). console.warn bypasses any log-level filtering, structured-logging sinks, or log routing the project relies on.

Consider importing and using log.warn(...) to match the surrounding code's logging conventions. The same applies to the identical console.warn call at line 406–410 in downgradeThinkingBlocksOnModelSwitch.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-helpers/openai.ts
Line: 322-326

Comment:
**Use project logger instead of `console.warn`**

Both `stripInvalidThinkingSignatures` and `downgradeThinkingBlocksOnModelSwitch` use `console.warn` directly, while the rest of the codebase uses the structured `log` module (imported from `./logger.js`). `console.warn` bypasses any log-level filtering, structured-logging sinks, or log routing the project relies on.

Consider importing and using `log.warn(...)` to match the surrounding code's logging conventions. The same applies to the identical `console.warn` call at line 406–410 in `downgradeThinkingBlocksOnModelSwitch`.

How can I resolve this? If you propose a fix, please make it concise.

@tag-assistant tag-assistant force-pushed the fix/thinking-signature-sanitization branch from 17f35f6 to b9fe216 Compare March 19, 2026 04:30
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b9fe216311

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

);
}
}
const compactResult = await compactEmbeddedPiSessionDirect({
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Pass observed overflow token count into compaction

Overflow recovery no longer extracts/passes the provider-reported token count into compaction; this call now omits currentTokenCount, so compaction loses the precise over-budget signal from errors like “277403 > 200000”. In that scenario compaction can under-trim and immediately re-overflow, turning a recoverable retry into a terminal context_overflow.

Useful? React with 👍 / 👎.

: attempt.yieldDetected
? "end_turn"
: (lastAssistant?.stopReason as string | undefined),
stopReason: attempt.clientToolCall ? "tool_calls" : undefined,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Emit end_turn stopReason for sessions_yield completions

This now reports stopReason only for clientToolCall and drops the previous yieldDetected -> "end_turn" mapping. sessions_yield sets yieldDetected (not clientToolCall), so successful yield exits now surface stopReason: undefined, which breaks lifecycle consumers that rely on end_turn to treat the parent run as cleanly finished and idle.

Useful? React with 👍 / 👎.

}

const { model, error, authStorage, modelRegistry } = await resolveModelAsync(
const { model, error, authStorage, modelRegistry } = resolveModel(
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use async model resolution to run dynamic model prep hooks

Switching this call from resolveModelAsync to resolveModel skips provider prepareDynamicModel hooks (implemented in the async resolver path), so providers that require async priming before dynamic model resolution (for example OpenRouter capability preload) now run with stale/default model metadata. That can mis-size context budgets and feature flags on first use, leading to incorrect runtime behavior.

Useful? React with 👍 / 👎.

@openclaw-barnacle
Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Apr 8, 2026
@openclaw-barnacle
Copy link
Copy Markdown

Closing due to inactivity.
If you believe this PR should be revived, post in #pr-thunderdome-dangerzone on Discord to talk to a maintainer.
That channel is the escape hatch for high-quality PRs that get auto-closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling size: XL stale Marked as stale due to inactivity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant