Summary

Changes image handling behavior from returning an error to stripping image content and proceeding with text-only messages. This provides better UX by allowing requests to continue when users accidentally include images with non-vision models.

Changes

1. Behavior Change

• Previously: Non-vision models would return a 400 error when receiving messages with images
• Now: Non-vision models strip image content and continue with text-only messages

2. New Method: StripImageContent()

• Added to providers/message_helpers.go
• Removes image content from messages while preserving text parts
• Handles various content formats:
  • String content remains unchanged
  • Array with only text becomes a single string
  • Array with text and images keeps only text
  • Array with only images becomes empty string
  • Array with multiple text parts and images keeps all text parts

3. Updated Route Handler

• Modified api/routes.go to use the new stripping logic
• Changed error logging to info logging for image filtering
• Added image count tracking for better observability
• Added debug logging when images are stripped

4. Comprehensive Tests

• Added TestMessage_StripImageContent in tests/multimodal_test.go
• Tests cover all edge cases and content formats
• Ensures the stripping logic works correctly

Benefits

1. Better User Experience: Users don't get errors when accidentally including images with non-vision models
2. Backward Compatible: Existing text-only requests work exactly the same
3. Observable: Logs at INFO level when images are filtered, with image counts
4. Tested: Comprehensive test coverage ensures reliability

Technical Details

• When EnableVision is true and a non-vision model receives image content:

  • Logs INFO message with provider, model, and image count
  • Strips image content from all messages
  • Continues processing with text-only content
  • Logs DEBUG message confirming images were stripped

• The StripImageContent() method handles the OpenAPI message format where content can be:

  • A string (text only)
  • An array of content parts (text and/or images)

Testing

Run the tests with:

task test

The new tests verify:

• String content remains unchanged
• Text-only arrays become strings
• Images are properly filtered out
• Mixed content keeps only text parts
• Empty content when only images exist