Skip to content

Support using generated images as ImageContext for downstream columns #341

@nabinchha

Description

@nabinchha

Priority Level

High (Major improvement)

Is your feature request related to a problem? Please describe.

When an ImageColumnConfig generates images and a subsequent column references them via ImageContext, the pipeline breaks:

  1. Create mode: Image columns store relative file paths (e.g., images/col/uuid.png) in the record. ImageContext.get_contexts() doesn't know how to load file paths — it only handles raw base64 strings and URLs.
  2. Local files can't be sent to remote model endpoints: Even if a file path were treated as a URL, the remote model provider has no access to local files. Generated images must be converted to base64 before being sent over HTTP.
  3. Preview mode: Works only if the user explicitly sets data_type=BASE64, but the current example pattern uses data_type=URL which is incorrect for base64 data.

Example pipeline that fails

  config_builder.add_column(
      dd.ImageColumnConfig(
          name=f"{model}-image",
          prompt="A {{style}} portrait of {{subject}}...",
          model_alias=model,
      )
  )
  config_builder.add_column(
      dd.ImageColumnConfig(
          name=f"{model}-image-edit",
          prompt="Add a {{accessory}} to the image.",
          model_alias=model,
          multi_modal_context=[
              dd.ImageContext(
                  column_name=f"{model}-image",
                  data_type=dd.ModalityDataType.URL,
              )
          ],
      )
  )

Describe the solution you'd like

Handle it behind the scene with very little ux change.

Describe alternatives you've considered

N/A

Additional context

This is a follow up to #317

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request
No fields configured for Feature.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions