Skip to content

fix(agents): fallback retry image safety and partial execution detection#1699

Open
BingqingLyu wants to merge 6 commits intomainfrom
fork-pr-57959-fix-fallback-image-safety-partial-execution
Open

fix(agents): fallback retry image safety and partial execution detection#1699
BingqingLyu wants to merge 6 commits intomainfrom
fork-pr-57959-fix-fallback-image-safety-partial-execution

Conversation

@BingqingLyu
Copy link
Copy Markdown
Owner

@BingqingLyu BingqingLyu commented Apr 27, 2026

Summary

  • Failure-mode-aware image handling: Replace unconditional image stripping on fallback retry with resolveRetryImages — only strip on format errors (model can't handle images) or cross-provider retries (privacy boundary)
  • Cross-provider image privacy: Strip user-supplied images when fallback targets a different provider, preventing unintended third-party disclosure
  • Prompt-detected image bypass fix: Add suppressPromptImageDetection to the embedded runner so detectAndLoadPromptImages is skipped on cross-provider retries — closes the gap where prompt-referenced local images bypassed the image strip
  • Partial tool execution detection: Thread previousFailureReason and previousPartialExecution through ModelFallbackRunOptions so the retry callback knows what the prior attempt did; warn fallback models about already-executed tools to prevent replaying non-idempotent operations

Builds on openclaw#55632 (prompt preservation for new sessions). Supersedes openclaw#44188. Closes openclaw#43481, openclaw#43492.

Design boundary

All changes stay at the runAgentAttempt / runWithModelFallback boundary. The pi-runner internals are not modified beyond accepting the suppressPromptImageDetection flag. This is intentional — the runner's internal loops (profile rotation, cooldown skips) are treated as opaque. See openclaw#44188 for the full design discussion.

Changes

failover-error.ts

  • New PartialExecution type with toolNames and didSendViaMessagingTool
  • FailoverError carries optional partialExecution field
  • sanitizeToolNames static method (caps at 20 entries, strips non-alphanumeric, truncates to 100 chars)
  • describeFailoverError returns partialExecution when present

model-fallback.ts

  • ModelFallbackRunOptions extended with previousFailureReason and previousPartialExecution
  • Fallback loop accumulates failure reason and merges partial execution across attempts (union tool names, OR messaging flag)
  • Threaded into runOptions for next candidate

attempt-execution.ts

  • resolveRetryImages: failure-mode-aware image handling (format → strip, cross-provider → strip, same-provider non-format → preserve)
  • buildPartialExecutionSystemContext: system prompt warning about already-executed tools (same-provider only, CWE-200)
  • runAgentAttempt extended with primaryProvider, previousFailureReason, previousPartialExecution
  • Both CLI and embedded paths use resolveRetryImages and effectiveExtraSystemPrompt

Embedded runner (params.ts, run.ts, attempt.ts)

  • New suppressPromptImageDetection param gates detectAndLoadPromptImages

agent-command.ts

  • Passes primaryProvider, previousFailureReason, previousPartialExecution from fallback runOptions into runAgentAttempt

Test plan

  • resolveRetryImages — 6 tests: first attempt, format error, rate_limit, cross-provider, undefined provider, undefined images
  • buildPartialExecutionSystemContext — 3 tests: tool listing, messaging warning, empty tools
  • FailoverError.sanitizeToolNames — 3 tests: cap at 20, truncate to 100, filter empty
  • FailoverError.partialExecution — 2 tests: carries when provided, undefined when absent
  • describeFailoverError — 2 tests: includes/omits partialExecution
  • runWithModelFallback — 4 tests: threads previousFailureReason, threads previousPartialExecution, merges across attempts, omits when absent
  • pnpm check — clean (0 warnings, 0 errors)
  • pnpm build — clean (no type errors, no INEFFECTIVE_DYNAMIC_IMPORT)

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cross-provider fallback should respect provider trust boundaries for images/attachments

2 participants