Skip to content

fix: strip image parts for non-vision models with provider profiles#26498

Closed
hueilau wants to merge 1 commit into
NousResearch:mainfrom
hueilau:fix/strip-images-provider-profile-path
Closed

fix: strip image parts for non-vision models with provider profiles#26498
hueilau wants to merge 1 commit into
NousResearch:mainfrom
hueilau:fix/strip-images-provider-profile-path

Conversation

@hueilau

@hueilau hueilau commented May 15, 2026

Copy link
Copy Markdown
Contributor

Problem

When using non-vision models like DeepSeek v4-pro with the native provider, image_url content blocks from tools like browser_vision, computer_use screenshots, or /paste cause HTTP 400 errors:

unknown variant `image_url`, expected `text`

The image blocks persist in conversation history, making the session unusable.

Root Cause

In run_agent.py's _build_api_kwargs(), _prepare_messages_for_non_vision_model() is only called in the legacy flag path (line 9715), which is reached when get_provider_profile() returns None — i.e., unknown/unregistered providers.

Registered providers with profiles (DeepSeek, Kimi, MiniMax, etc.) take the provider profile path (line 9680-9705), which passes api_messages directly to _ct.build_kwargs() without stripping images.

Fix

Add the same _prepare_messages_for_non_vision_model() call to the provider profile path, mirroring the legacy path behavior. Vision-capable models are unaffected (the function is a no-op for them).

Verification

  • DeepSeek v4-pro: images are stripped before API call → no more 400 errors
  • Claude/GPT/Gemini: no-op, behavior unchanged
  • Other non-vision providers with profiles (Kimi, MiniMax): also protected

Related

Copilot AI review requested due to automatic review settings May 15, 2026 18:11
_propare_messages_for_non_vision_model() was only called in the legacy
flag path (no provider profile). Providers with registered profiles
(e.g. DeepSeek, Kimi) bypassed the strip, causing HTTP 400 errors when
image_url content blocks reached their non-vision APIs.

This mirrors the existing behavior in the legacy path, ensuring all
non-vision models get image stripping regardless of profile status.
Vision-capable models are unaffected (the function is a no-op for them).
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent loop, run_agent.py, prompt builder labels May 15, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #23743. See also #23750, #23818, #23904, #26364 — all fixing the same missing _prepare_messages_for_non_vision_model() call on the provider-profile path in _build_api_kwargs(). #25925 (merged) addressed a related runtime gate.

@hueilau hueilau force-pushed the fix/strip-images-provider-profile-path branch from 374b5c3 to 855aee1 Compare May 15, 2026 18:22
teknium1 added a commit that referenced this pull request May 17, 2026
…tors

Adds release-note attribution mappings for 9 contributors from group 3:
- @darvsum (PR #26766)
- @hueilau (PR #26498)
- @Timur00Kh (PR #27114)
- @Grogger (PR #27061)
- @lemassykoi (PR #27042)
- @draplater (PR #26707)
- @pr7426 (PR #27048)
- @therahul-yo (PR #26215)
- @flamiinngo (PR #27205)

#27154 dropped from this batch — already landed on main as 4e9cedc.
@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #27302 — your commit was cherry-picked onto current main as part of a batch salvage of low-risk new-contributor PRs. Authorship preserved (fix: strip image parts for non-vision models with provider profiles). Thanks for the contribution.

@teknium1 teknium1 closed this May 17, 2026
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent loop, run_agent.py, prompt builder P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants