Skip to content

Agents: scope error rewrites in sanitizeUserFacingText behind errorContext flag (#11649)#11685

Closed
lailoo wants to merge 1 commit intoopenclaw:mainfrom
lailoo:fix/sanitize-error-context-11649
Closed

Agents: scope error rewrites in sanitizeUserFacingText behind errorContext flag (#11649)#11685
lailoo wants to merge 1 commit intoopenclaw:mainfrom
lailoo:fix/sanitize-error-context-11649

Conversation

@lailoo
Copy link

@lailoo lailoo commented Feb 8, 2026

Summary

Fixes #11649 (thorough fix)

This is the thorough fix for #11649, complementing the minimal fix in PR #11680 (length guard). Instead of relying on text length to distinguish errors from normal replies, this PR adds an explicit errorContext flag so error-specific rewrites only run when the caller knows the text originates from an error.

Problem

sanitizeUserFacingText() applies error-specific rewrites (billing, rate-limit, timeout, HTTP status, context overflow) to all text, including normal assistant replies. When the assistant discusses billing, credits, or payment topics, the sanitizer false-positives and replaces the entire reply.

Reproduction & Verification

Used scripts/reproduce-sanitize-false-positive.ts and scripts/verify-sanitize-false-positive.ts to call the actual sanitizeUserFacingText() function with real-world assistant reply text.

Before fix (main branch) — Bug reproduced:

❌ FALSE POSITIVE: Assistant discussing billing/credits/upgrade/plan
   Input:  "Sure, I can help you understand your billing situation..."
   Output: "⚠️ API provider returned a billing error — your API key has run out of credits..."
   BUG CONFIRMED: Normal reply was replaced with error message!

❌ FALSE POSITIVE: Short reply mentioning billing + upgrade
   Input:  "Check your billing page to upgrade your plan and add credits."
   Output: "⚠️ API provider returned a billing error..."

❌ FALSE POSITIVE: Reply about payment methods
   Input:  "Your payment method needs to be updated. Go to billing to fix it."
   Output: "⚠️ API provider returned a billing error..."

After fix — All verified:

✅ PASS: Normal assistant reply discussing billing (should NOT be replaced)
   → Text passed through unchanged
✅ PASS: Normal reply mentioning upgrade your plan (should NOT be replaced)
   → Text passed through unchanged
✅ PASS: Short billing error with errorContext=true (should be replaced)
   → Rewritten to: "⚠️ API provider returned a billing error..."
✅ PASS: HTTP 402 error with errorContext=true (should be replaced)
   → Rewritten to: "⚠️ API provider returned a billing error..."
✅ PASS: Normal text without billing keywords (should pass through)
   → Text passed through unchanged
✅ PASS: Role ordering error with errorContext=true (should be rewritten)
   → Rewritten to: "Message ordering conflict - please try again..."
✅ PASS: Role ordering text WITHOUT errorContext (should NOT be rewritten)
   → Text passed through unchanged
✅ PASS: Raw API error payload with errorContext=true (should be rewritten)
   → Rewritten to: "LLM error server_error: Something exploded"
✅ PASS: Raw API error payload WITHOUT errorContext (should NOT be rewritten)
   → Text passed through unchanged

✅ All 9 checks passed.

Effect on User Experience

Before fix — user asks about billing, gets a canned error instead of the real answer:

User: Can you explain how API billing works?

Assistant: ⚠️ API provider returned a billing error — your API key
has run out of credits or has an insufficient balance. Check your
provider's billing dashboard and top up or switch to a different\nAPI key.\n```\nThe user never sees the assistant's actual helpful reply.\n\n**After fix** — user gets the real answer:\n```\nUser: Can you explain how API billing works?\n\nAssistant: Sure, I can help you understand your billing situation.\nYour API credits are managed through the billing dashboard. To check\nyour current balance, go to Settings > Billing > Credits. If you need\nto upgrade your plan, you can do so from the same page.\n```\n\nMeanwhile, **real** billing errors (with `errorContext: true`) are still correctly rewritten:\n```\n[LLM returns stopReason: \"error\", text: \"insufficient credits\"]\n\n→ sanitizeUserFacingText(\"insufficient credits\", { errorContext: true })\n→ \"⚠️ API provider returned a billing error — your API key has run out\n   of credits or has an insufficient balance...\"\n```\n\n## Approach\n\nAdd an optional `opts.errorContext` parameter to `sanitizeUserFacingText()`. Error-specific rewrites are gated behind `errorContext: true`.\n\n**Safe operations that always run** (regardless of `errorContext`):\n- Strip `<final>` tags\n- Collapse consecutive duplicate paragraphs\n\n**Error-specific rewrites** (only with `errorContext: true`):\n- Billing error detection\n- Rate-limit / overloaded detection\n- Timeout detection\n- HTTP status code rewriting\n- Role ordering conflict rewriting\n- Raw API error payload formatting\n- Context overflow rewriting\n\n## Changes\n\n### `src/agents/pi-embedded-helpers/errors.ts`\n- `sanitizeUserFacingText(text, opts?)`: Add optional `{ errorContext?: boolean }` parameter\n- All error-specific rewrites gated behind `if (isError)` block\n- Default: `errorContext: false` (safe — no false-positives)\n\n### `src/agents/pi-embedded-utils.ts`\n- `extractAssistantText()`: Pass `{ errorContext: msg.stopReason === \"error\" }` to `sanitizeUserFacingText`\n- This is the key callsite — when the LLM returns `stopReason: \"error\"`, the text is an error message and should be rewritten\n\n### Other callsites (unchanged, safe defaults)\n- `normalize-reply.ts`: Processes final reply text — no `errorContext` (safe)\n- `agent-runner-execution.ts`: Streaming text — no `errorContext` (safe)\n- `sessions-helpers.ts`: Subagent reply text — no `errorContext` (safe)\n\n### Tests\n- 15 tests: existing error rewrite tests updated to use `{ errorContext: true }`\n- New tests: verify normal text passes through without `errorContext`\n- New test: billing discussion text preserved without `errorContext`\n\n## Why this is better than the length guard (#11680)\n\n| Approach | PR #11680 (length guard) | This PR (errorContext) |\n|---|---|---|\n| Mechanism | `trimmed.length < 500` | `opts.errorContext` flag |\n| False-positive risk | Low (but possible for short billing discussions) | **Zero** (only rewrites when caller confirms error) |\n| Semantic correctness | Heuristic | **Explicit** |\n| Breaking changes | None | None (optional param, safe default) |\n\n## Testing\n- 15 `sanitizeUserFacingText` tests pass\n- 1249 agent tests pass (`pnpm vitest run src/agents/`)\n- `pnpm lint` passes with 0 errors

@openclaw-barnacle openclaw-barnacle bot added the agents Agent runtime and tooling label Feb 8, 2026
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 8, 2026

Additional Comments (1)

src/agents/tools/sessions-helpers.ts
Missing errorContext propagation

sanitizeUserFacingText() now only performs error rewrites when { errorContext: true }, but this callsite still invokes it as sanitizeUserFacingText(joined) (src/agents/tools/sessions-helpers.ts:392). If joined can ever contain error-origin text (e.g. persisted assistant messages representing failures, or subagent/tool error payloads stored as assistant text blocks), this will regress UI behavior by surfacing raw HTTP codes/JSON/error strings instead of the sanitized user-facing messages. Either ensure this path never processes error-context text, or propagate an equivalent error flag here (and any similar callsites).

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/tools/sessions-helpers.ts
Line: 392:392

Comment:
**Missing errorContext propagation**

`sanitizeUserFacingText()` now only performs error rewrites when `{ errorContext: true }`, but this callsite still invokes it as `sanitizeUserFacingText(joined)` (`src/agents/tools/sessions-helpers.ts:392`). If `joined` can ever contain error-origin text (e.g. persisted assistant messages representing failures, or subagent/tool error payloads stored as assistant text blocks), this will regress UI behavior by surfacing raw HTTP codes/JSON/error strings instead of the sanitized user-facing messages. Either ensure this path never processes error-context text, or propagate an equivalent error flag here (and any similar callsites).

How can I resolve this? If you propose a fix, please make it concise.

@Takhoffman
Copy link
Contributor

Fixed in #12988.

This will go out in the next OpenClaw release.

If you still see this after updating to the first release that includes #12988, please open a new issue with:

  • your OpenClaw version
  • channel (Telegram/Slack/etc)
  • the exact prompt/response that got rewritten
  • whether Web UI showed the full text vs the channel being rewritten
  • relevant logs around send/normalize (if available)

Link back here for context.

@Takhoffman
Copy link
Contributor

Closing as superseded by the merged sanitize/error-context work:

This PR’s intent appears covered by those merged changes and current mainline tests.

@Takhoffman Takhoffman closed this Feb 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sanitizeUserFacingText false-positive: normal assistant replies containing billing keywords get replaced with billing error message

2 participants