feat (fal): Set .providerMetaData for image model responses#6406
Conversation
8f19464 to
489e548
Compare
0450ffa to
ef25ab2
Compare
| targetImages.map(image => this.downloadImage(image.url, abortSignal)), | ||
| ); | ||
| const { | ||
| // @ts-expect-error - either image or images is present, not both. |
There was a problem hiding this comment.
is there a typesafe way to handle this?
There was a problem hiding this comment.
I'm not using image or images, the code is just to filter keys fro responseMetaData. It's the most efficient way as far as I know.
| { | ||
| width: 1024, | ||
| height: 1024, | ||
| content_type: 'image/png', |
There was a problem hiding this comment.
duplicates standardized prop?
| ...(contentType !== undefined ? { contentType } : undefined), | ||
| ...(fileName !== undefined ? { fileName } : undefined), | ||
| ...(fileData !== undefined ? { fileData } : undefined), | ||
| ...(fileSize !== undefined ? { fileSize } : undefined), |
There was a problem hiding this comment.
I normalized some of the providerMetaData properties that I think the way we normalized revised_prompt to revisedPrompt in the Open AI Image model.
Shall I also camelize debug_latents, debug_per_pass_latents, and their respective properties?
There was a problem hiding this comment.
wonder how far we should to here. usually we do this but it is also overhead
There was a problem hiding this comment.
wonder how far we should to here. usually we do this but it is also overhead
I agree. I think we should do it for properties that we see repeatedly across providers and models. But e.g. not go overboard with things like debug_latents which is specific to a single model from fal
I also wonder if we should pass through provider options that we don't handle ourselves. Otherwise we will always play catch up whenever some model from some provider exposes a new information. We could do it in a way that clearly communicates that we don't take responsibility, like .providerMetaData.fal.unsafe_otherData or similar.
Background
FalImage Models return extra information besides the images themselves. This pull request exposes that information as.providerMetaData.fal.*on thegenerateImage()return object.Example response
Summary
I updated the response schema for Fal's Image Model API responses and set the
.providerMetaDatakey to the return object ofFalImageModel#doGEnerate().I normalized the
.nsfwimage meta data since Fal models have two variationshas_nsfw_concepts- seems to be the standard onensfw_content_detected- only used by https://fal.ai/models/fal-ai/lcm/api#schema-output as far as I can tellVerification
I altered
examples/ai-core/src/generate-image/fal.tsTasks
pnpm changesetin the project root)pnpm prettier-fixin the project root)Related Issues
Part of #6405