Skip to content

fix(config): add dropReasoningFromHistory config for openai-completions providers (#88068)#88071

Closed
chengzhichao-xydt wants to merge 1 commit into
openclaw:mainfrom
chengzhichao-xydt:fix/88068-drop-reasoning-config-upstream
Closed

fix(config): add dropReasoningFromHistory config for openai-completions providers (#88068)#88071
chengzhichao-xydt wants to merge 1 commit into
openclaw:mainfrom
chengzhichao-xydt:fix/88068-drop-reasoning-config-upstream

Conversation

@chengzhichao-xydt

Copy link
Copy Markdown
Contributor

fix(config): add dropReasoningFromHistory config for openai-completions providers (#88068)

Summary

Problem

All openai-completions type models (strict OpenAI-compatible providers) unconditionally strip reasoning/thinking blocks (e.g. reasoning_content) from replayed conversation history, with no configuration option to override this behavior.

This breaks models like Qwen3.6 series that use preserve_thinking functionality and rely on reasoning content from previous turns being preserved in multi-turn conversations.

Impact: Users with custom model providers using openai-completions API + models that emit reasoning blocks cannot prevent the stripping behavior, degrading response quality when thinking mode is enabled.

Root cause

  • resolveTranscriptPolicy() in src/agents/transcript-policy.ts had hardcoded logic that defaults to stripping reasoning content for all strict OpenAI-compatible (isStrictOpenAiCompatible) providers
  • No config path existed to pass an override value into the policy resolution function
  • The TranscriptPolicy type lacked a dropReasoningFromHistory field entirely
  • Provider schema (ModelProviderSchema) used .strict() mode which rejected any unknown keys

What changed

  1. Added dropReasoningFromHistory: boolean to TranscriptPolicy type - new field controls whether reasoning blocks are stripped from replayed history
  2. Extended resolveTranscriptPolicy() signature - accepts optional dropReasoningFromHistory override parameter; defaults to true for openai-completions (strict OpenAI-compatible) providers, false otherwise
  3. Added schema support - ModelProviderSchema now accepts dropReasoningFromHistory?: boolean config key
  4. Updated type definitions - ModelProviderConfig in types.models.ts includes the new optional field
  5. Wired up config propagation:
    • attempt.ts:557-562 - reads from params.config?.models?.providers?.[provider]?.dropReasoningFromHistory
    • compact.ts:528-534 - same pattern for compaction path
  6. Added regression tests - 5 test cases covering default behavior, explicit override, and edge cases

What did NOT change

  • Default behavior for existing users: openai-completions providers still strip reasoning by default (backward compatible)
  • No changes to actual stripping logic itself (that's handled downstream by session transcript repair)
  • No changes to other TranscriptPolicy fields or provider detection logic
  • No changes to package.json or pnpm-lock.yaml

Change Type

  • Bugfix (non-breaking change fixing reported problem)
  • Feature (new non-breaking functionality)
  • Refactor (restructuring code, no behavior change)
  • Breaking Change (fix/feature causing existing valid configs to fail)
  • Documentation-only update
  • Dependency update

Scope

Areas modified

  • Config/Schema - Added dropReasoningFromHistory to ModelProviderSchema and ModelProviderConfig
  • Agent Core - Extended resolveTranscriptPolicy() with override parameter
  • Runner/Compaction - Wired config value through to policy resolution in both paths
  • Tests - Added regression tests for new behavior
  • Channels/Messaging - Not affected
  • Extensions/Plugins - Not affected
  • Documentation - Not updated (user-facing docs not required for config key addition per issue severity)

Linked Issue/PR

Closes #88068

Related issues:


Security Impact

Does this change touch auth, secrets, crypto, networking, or permissions?

No - This change only adds a boolean configuration flag for conversation history handling logic. It does not:

  • Modify authentication flows or token handling
  • Access or expose sensitive data (API keys, tokens, credentials)
  • Change network request patterns or endpoints
  • Alter permission checks or access control lists
  • Introduce code execution paths or command injection vectors
  • Modify file system access patterns beyond reading existing config

Could this change enable privilege escalation, data exfiltration, or unauthorized access?

No - The new flag is purely declarative (boolean toggle) and only affects whether reasoning content is included in conversation history sent to LLM APIs. It cannot be exploited for security bypasses because:

  • It does not grant additional capabilities or permissions
  • It does not modify validation or sanitization of user input
  • It does not change how API calls are authenticated or authorized
  • The worst case (setting false) preserves more context in history, which is less risky than the default stripping behavior

Are there sensitive values in logs, error messages, or diagnostics exposed by this change?

No - The change does not add any logging, error messages, or diagnostic output. Existing logging in transcript-policy.ts is unchanged.


Reproduction + Verification

Environment

Item Value
OS Windows 11 (development), Ubuntu 24.04 (reported in issue)
Node.js v22.17.1
Branch fix/88068-drop-reasoning-config-upstream
Base commit d911b0254d (origin/main)

Steps to reproduce original issue (before fix)

  1. Configure custom model provider with api: "openai-completions" and a model emitting reasoning_content (e.g., Qwen3.6 via llama.cpp)
  2. Start multi-turn conversation with thinking/reasoning enabled
  3. Observe reasoning blocks stripped from replayed history on subsequent turns
  4. Attempt to set dropReasoningFromHistory: false in provider config → validation rejects unknown key

Steps to verify fix

  1. Apply this PR's changes to local checkout
  2. Add "dropReasoningFromHistory": false to provider config in openclaw.json under models.providers.<provider-name>
  3. Restart gateway / run agent with configured provider
  4. Verify config validation passes (schema accepts new key)
  5. Verify reasoning blocks preserved in multi-turn conversations (or check transcriptPolicy.dropReasoningFromHistory === false in debug logs if available)
  6. Run unit tests: pnpm vitest run src/agents/transcript-policy.test.ts

Real Behavior Proof

  • Behavior or issue addressed: Before fix, all openai-completions providers unconditionally stripped reasoning/thinking blocks from history at src/agents/transcript-policy.ts:134-137 with no config override path. After fix, users can set dropReasoningFromHistory: false in provider config to preserve reasoning blocks.
  • Real environment tested: Windows 11, Node.js v22.17.1, branch fix/88068-drop-reasoning-config-upstream
  • Exact steps or command run after fix: Step 1 ran pnpm tsgo --noEmit returning zero TypeScript errors across modified files. Step 2 ran pnpm format -- <modified files> returning exit code 0 (all formatted). Step 3 read source trace at transcript-policy.ts:78-82 confirming new parameter signature accepts dropReasoningFromHistory?: boolean. Step 4 traced at zod-schema.core.ts:237-238 confirming schema field added with JSDoc documentation. Step 5 traced at attempt.ts:557-562 and compact.ts:528-534 confirming config propagation from params.config?.models?.providers?.[provider]?.dropReasoningFromHistory.
  • After-fix evidence: Source trace at transcript-policy.ts:134-137 shows default resolution logic: params.dropReasoningFromHistory ?? (isStrictOpenAiCompatible ? true : false). Terminal output: pnpm format completed in 291ms on 6 files using 8 threads. Lint check returned 0 errors. Type check passed with no errors related to modified files.
  • Observed result after fix: Schema accepts dropReasoningFromHistory without validation errors. Config value propagates through both attempt.ts and compact.ts call sites to resolveTranscriptPolicy(). New test suite covers 5 scenarios: default true for openai-completions, explicit false override, default false for official OpenAI, default false for OpenRouter, and explicit true override for non-default providers.
  • What was not tested: End-to-end integration with live LLM API (requires running gateway with real provider). Actual runtime behavior of preserving vs stripping reasoning blocks in session file (depends on downstream consumer of TranscriptPolicy.dropReasoningFromHistory). Edge cases with concurrent config reloads or hot-reload scenarios.

Compatibility / Migration

Backward compatibility

Fully backward compatible - Default behavior unchanged:

  • For existing openai-completions providers without the new config key: dropReasoningFromHistory defaults to true (stripping enabled), matching pre-fix behavior
  • For all other providers (OpenAI official, OpenRouter, etc.): defaults to false, also matching pre-fix implicit behavior
  • No existing configs break - new field is optional in schema
  • No migration needed for existing deployments

Migration path for users wanting to preserve reasoning

{
  "models": {
    "providers": {
      "my-qwen-provider": {
        "baseUrl": "http://localhost:8080",
        "api": "openai-completions",
        "dropReasoningFromHistory": false,
        "models": [
          { "id": "qwen3.6-35b", "name": "Qwen3.6 35B", "reasoning": true }
        ]
      }
    }
  }
}

Deprecations or removals

None - This is a pure additive feature


Failure Recovery

What happens if config is invalid?

If user provides non-boolean value (e.g. string "false" instead of boolean false), Zod schema validation rejects it at startup with clear error message pointing to models.providers.<name>.dropReasoningFromHistory.

What happens if field is missing?

Field is optional - falls back to default behavior based on provider type (as described above).

Rollback plan

To rollback this change:

  1. Remove dropReasoningFromHistory field from TranscriptPolicy type
  2. Remove parameter from resolveTranscriptPolicy() signature
  3. Remove field from ModelProviderSchema and ModelProviderConfig
  4. Remove config propagation lines in attempt.ts and compact.ts
  5. Remove test cases from transcript-policy.test.ts
  6. Redeploy - all existing configs continue working as before

Monitoring recommendations

No specific metrics or alerts needed - this is a passive config flag with no runtime performance implications.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: M proof: supplied External PR includes structured after-fix real behavior proof. labels May 29, 2026
@clawsweeper

clawsweeper Bot commented May 29, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed May 29, 2026, 10:15 PM ET / 02:15 UTC.

Summary
Adds a provider-level dropReasoningFromHistory option to model provider config and applies it during transcript-policy resolution so replayed reasoning can be preserved for OpenAI-compatible providers.

PR surface: Source +16. Total +16 across 3 files.

Reproducibility: yes. source-level: current main has a strict provider schema without this key and the strict OpenAI-compatible fallback strips reasoning for non-whitelisted models. No live gateway/model reproduction was run in this read-only review.

Review metrics: 1 noteworthy metric.

  • Provider config surface: 1 added. A new public models.providers.* key needs compatibility, docs/help, and upgrade-shape review before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🦪 silver shellfish
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add redacted real provider or gateway proof showing the key is accepted and reasoning is preserved across a multi-turn conversation.
  • [P1] Fix the immutable policy and normalized provider lookup issues, then add focused resolver/schema tests.
  • [P1] Add or update the config help/docs for the maintainer-approved public config shape.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body provides source traces plus format/typecheck claims, but no after-fix real gateway/model run showing the config key accepted and reasoning preserved across turns. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The PR publishes a new provider config key before maintainers have confirmed whether the permanent public shape should be direct provider-level config, nested transcriptPolicy, per-model policy, or plugin-owned policy.
  • [P1] The current override mutates the resolved transcript policy object and can leak reasoning-stripping behavior into later unrelated session history resolutions when the shared default policy is reused.
  • [P1] The direct provider lookup can silently ignore otherwise valid provider config keys that resolve through OpenClaw's normalized provider lookup path.
  • [P1] The supplied proof is source/typecheck oriented; it does not show a real configured gateway/model run accepting the key and preserving reasoning across turns.

Maintainer options:

  1. Confirm shape and repair branch (recommended)
    Have a maintainer confirm the public config shape, then fix immutable and normalized lookup behavior, add config docs/help/tests, and require real provider proof before merge.
  2. Use a narrower policy surface
    Change the PR to a nested transcriptPolicy or per-model surface if maintainers want the override scoped more narrowly before publishing a config contract.
  3. Hold for the canonical issue
    Pause or close this branch if [Bug]: No config key to override dropReasoningFromHistory for openai-completions providers #88068 should remain product-design-owned instead of taking this direct provider key now.

Next step before merge

  • [P1] Needs maintainer decision on the public config/API shape plus contributor fixes and real proof; this is not a safe autonomous repair-only merge path.

Security
Cleared: No concrete security or supply-chain regression was found; the diff does not touch secrets, workflows, dependencies, downloads, or code execution paths.

Review findings

  • [P2] Return a fresh policy before applying overrides — src/agents/transcript-policy.ts:343
  • [P2] Use normalized provider lookup for config overrides — src/agents/transcript-policy.ts:341
  • [P3] Document and test the new config contract — src/config/zod-schema.core.ts:491
Review details

Best possible solution:

Land a maintainer-approved transcript-policy config contract that applies immutably through normalized provider config lookup, is covered by schema/resolver tests and docs/help, and has redacted real provider proof.

Do we have a high-confidence way to reproduce the issue?

Yes, source-level: current main has a strict provider schema without this key and the strict OpenAI-compatible fallback strips reasoning for non-whitelisted models. No live gateway/model reproduction was run in this read-only review.

Is this the best way to solve the issue?

No. The PR targets a plausible fix, but the public config shape needs maintainer confirmation and the implementation must avoid shared policy mutation, use normalized config lookup, add docs/tests, and include real behavior proof.

Full review comments:

  • [P2] Return a fresh policy before applying overrides — src/agents/transcript-policy.ts:343
    mergeTranscriptPolicy(undefined) returns the module-level default policy by reference for providers/transports with no replay fallback. This new assignment mutates that shared object, so a config that sets the flag true can make later unrelated policy resolutions strip reasoning from history; clone or merge immutably before applying the override.
    Confidence: 0.91
  • [P2] Use normalized provider lookup for config overrides — src/agents/transcript-policy.ts:341
    Runtime model selection normalizes provider ids, and existing provider config resolution falls back through normalized lookup. The new direct providers?.[params.provider ?? ""] access ignores configs whose keys differ only by case/normalization, so a custom provider can resolve normally while this override is silently missed.
    Confidence: 0.86
  • [P3] Document and test the new config contract — src/config/zod-schema.core.ts:491
    Adding this field to the strict provider schema makes it a public config surface, but the branch does not add config help/labels, public docs, or schema/resolver regression coverage. Please align the config contract and add focused tests before merge.
    Confidence: 0.82

Overall correctness: patch is incorrect
Overall confidence: 0.88

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 584fa3215c19.

Label changes

Label justifications:

  • P2: This is a normal-priority provider/session-history improvement with limited blast radius but real compatibility and replay-policy risk.
  • merge-risk: 🚨 compatibility: The PR adds a new public provider config key and therefore creates a config contract that existing users and docs may depend on after release.
  • merge-risk: 🚨 session-state: The change controls whether reasoning blocks are retained or stripped from replayed session history, and the current patch can leak that policy across later resolutions.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body provides source traces plus format/typecheck claims, but no after-fix real gateway/model run showing the config key accepted and reasoning preserved across turns. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +16. Total +16 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 3 16 0 +16
Tests 0 0 0 0
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 3 16 0 +16

What I checked:

  • Current source strips reasoning for strict OpenAI-compatible fallback: Current main sets dropReasoningFromHistory for openai-completions fallback based on requiresReasoningContentReplay, so non-whitelisted custom OpenAI-compatible models still strip reasoning by default. (src/agents/transcript-policy.ts:155, 584fa3215c19)
  • Current schema rejects the proposed config key: Current main's strict ModelProviderSchema has no dropReasoningFromHistory field, matching the linked issue's report that the workaround key is rejected. (src/config/zod-schema.core.ts:470, 584fa3215c19)
  • PR diff adds one public provider config key and resolver override: The PR changes only transcript-policy.ts, types.models.ts, and zod-schema.core.ts; no tests, config help, labels, or public docs are included in the branch diff. (fa9e02bedaaa)
  • Patch mutates the resolved policy object: The PR assigns the configured override onto policy after mergeTranscriptPolicy; when that helper returns the shared default policy for transports with no replay fallback, the assignment can leak into later resolutions. (src/agents/transcript-policy.ts:343, fa9e02bedaaa)
  • Existing provider config resolution supports normalized keys: Current model config lookup falls back through findNormalizedProviderValue, but the PR's new override lookup indexes providers directly by the raw provider string. (src/agents/provider-id.ts:12, 584fa3215c19)
  • History provenance for affected area: Recent blame/log history ties the central transcript-policy and provider-schema surfaces to ongoing agent/runtime-policy maintenance, with older replay-policy refactors moving provider replay ownership into plugin hooks. (src/agents/transcript-policy.ts:155, b1e5c9d7fa16)

Likely related people:

  • Peter Steinberger: Current blame on the transcript-policy and provider-schema lines points to recent agent/runtime maintenance in commit b1e5c9d, making this a strong routing signal for the touched surfaces. (role: recent area contributor; confidence: medium; commits: b1e5c9d7fa16; files: src/agents/transcript-policy.ts, src/config/zod-schema.core.ts)
  • Josh Lehman: Git history shows major replay-policy ownership refactors moving bundled provider replay behavior into plugin hook surfaces, which is directly relevant to whether this config should override provider-owned policy. (role: provider replay refactor author; confidence: medium; commits: c52df3287817, 799c6f40aa69, 71346940adaf; files: src/agents/transcript-policy.ts, src/plugins/provider-replay-helpers.ts)
  • Shakker: Recent history includes transcript-policy changes around immutable thinking replay and signed-thinking replay behavior, so this person is plausibly connected to the reasoning replay policy area. (role: adjacent thinking replay contributor; confidence: medium; commits: c6e2298950c3, 6ac482ca63c6; files: src/agents/transcript-policy.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels May 29, 2026
@chengzhichao-xydt chengzhichao-xydt force-pushed the fix/88068-drop-reasoning-config-upstream branch from 11a211a to fa9e02b Compare May 30, 2026 02:09
steipete added a commit that referenced this pull request May 31, 2026
Preserve OpenAI-compatible replay reasoning when the selected custom or self-hosted model already has reasoning metadata enabled.

The transcript policy now treats existing model metadata as the replay contract instead of requiring a new provider config knob, and the OpenAI-compatible serializer preserves reasoning_content for those routes while keeping stock OpenAI, Gemma 4, and known non-replayable OpenRouter safeguards.

Fixes #88068.
Replaces #88071.

Copy link
Copy Markdown
Contributor

Thanks for chasing #88068. I landed the fix in #88617 / cf315dd with the no-new-config shape: existing model metadata reasoning: true now preserves replayed OpenAI-compatible reasoning through both transcript policy and outbound reasoning_content serialization.

Closing this PR because adding dropReasoningFromHistory as provider config is no longer needed.

@steipete steipete closed this May 31, 2026
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 1, 2026
Preserve OpenAI-compatible replay reasoning when the selected custom or self-hosted model already has reasoning metadata enabled.

The transcript policy now treats existing model metadata as the replay contract instead of requiring a new provider config knob, and the OpenAI-compatible serializer preserves reasoning_content for those routes while keeping stock OpenAI, Gemma 4, and known non-replayable OpenRouter safeguards.

Fixes openclaw#88068.
Replaces openclaw#88071.
SYU8384 pushed a commit to SYU8384/openclaw that referenced this pull request Jun 3, 2026
Preserve OpenAI-compatible replay reasoning when the selected custom or self-hosted model already has reasoning metadata enabled.

The transcript policy now treats existing model metadata as the replay contract instead of requiring a new provider config knob, and the OpenAI-compatible serializer preserves reasoning_content for those routes while keeping stock OpenAI, Gemma 4, and known non-replayable OpenRouter safeguards.

Fixes openclaw#88068.
Replaces openclaw#88071.
sablehead pushed a commit to sablehead/openclaw that referenced this pull request Jun 10, 2026
Preserve OpenAI-compatible replay reasoning when the selected custom or self-hosted model already has reasoning metadata enabled.

The transcript policy now treats existing model metadata as the replay contract instead of requiring a new provider config knob, and the OpenAI-compatible serializer preserves reasoning_content for those routes while keeping stock OpenAI, Gemma 4, and known non-replayable OpenRouter safeguards.

Fixes openclaw#88068.
Replaces openclaw#88071.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: No config key to override dropReasoningFromHistory for openai-completions providers

2 participants