Problem
When a model fallback retry crosses providers (e.g. Anthropic → OpenAI), user-supplied images and attachments are forwarded to the fallback provider. Users who configure cross-provider fallbacks for text resilience may not expect their image data to be sent to a different provider.
The current behavior is binary: configured providers receive everything. There's no mechanism to distinguish "I trust this provider for text fallback" from "I trust this provider with my image data."
Proposed solution
Add provider-level trust configuration so users can control what data crosses provider boundaries on fallback:
- A config field like
agents.defaults.model.fallbackTrustPolicy or per-provider trust levels
resolveRetryImages checks whether the fallback is cross-provider and whether the target provider is trusted for image data
- Default behavior: preserve current behavior (forward everything) to avoid breaking existing setups, but allow users to restrict it
Context
Flagged by security review on #43331. The behavior is pre-existing (prior to #43331, images were stripped unconditionally on fallback — a compatibility hack, not a privacy boundary), but worth addressing properly.
Problem
When a model fallback retry crosses providers (e.g. Anthropic → OpenAI), user-supplied images and attachments are forwarded to the fallback provider. Users who configure cross-provider fallbacks for text resilience may not expect their image data to be sent to a different provider.
The current behavior is binary: configured providers receive everything. There's no mechanism to distinguish "I trust this provider for text fallback" from "I trust this provider with my image data."
Proposed solution
Add provider-level trust configuration so users can control what data crosses provider boundaries on fallback:
agents.defaults.model.fallbackTrustPolicyor per-provider trust levelsresolveRetryImageschecks whether the fallback is cross-provider and whether the target provider is trusted for image dataContext
Flagged by security review on #43331. The behavior is pre-existing (prior to #43331, images were stripped unconditionally on fallback — a compatibility hack, not a privacy boundary), but worth addressing properly.