Description

ImageContext.get_contexts() produces image_url content blocks that are non-compliant with both the OpenAI and Anthropic vision APIs. This is a regression introduced after the removal of litellm in v0.5.4 — litellm was likely normalizing the shape internally before forwarding to provider APIs, which masked the issue.

Problem

Three code paths in ImageContext.get_contexts() produce incorrect shapes:

The OpenAI vision API expects:

{"type": "image_url", "image_url": {"url": "https://..."}}

The Anthropic adapter's translate_image_url_block defensively handled both bare strings and dicts, so Anthropic worked by accident. But the OpenAI native adapter passes content blocks through as-is, so multimodal requests with image URLs fail.

Fix

• Always wrap image_url values in {"url": ...} dict format (matches OpenAI spec)
• Remove non-standard "format" key from base64 dicts (media type is already encoded in the data URI)
• Tighten the Anthropic adapter's translate_image_url_block to raise TypeError when image_url is not a dict. Since all image_url blocks are constructed internally by DataDesigner (not by end users), a malformed block indicates an internal bug and should fail loudly rather than be silently dropped.

Files affected

• packages/data-designer-config/src/data_designer/config/models.py — ImageContext.get_contexts(), _auto_resolve_context_value(), _format_base64_context()
• packages/data-designer-engine/src/data_designer/engine/models/clients/adapters/anthropic_translation.py — translate_image_url_block()
• Test files in both data-designer-config and data-designer-engine