Update GenAI visualizer to use new multimodal semantic conventions#12694
Update GenAI visualizer to use new multimodal semantic conventions#12694
Conversation
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 12694Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 12694" |
There was a problem hiding this comment.
Pull Request Overview
This pull request updates the GenAI visualizer in the Aspire Dashboard to support displaying images from blob and uri type message parts according to GenAI semantic conventions, replacing the previous custom "image" type implementation.
Key changes:
- Added a
TryGetPropertyValuehelper method toGenAIItemPartViewModelfor cleaner property access - Refactored image detection logic to support
blob(base64-encoded) anduri(http/https URLs) part types with MIME type validation - Updated the playground to demonstrate both image part types with proper semantic attributes
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/Aspire.Dashboard/Model/GenAI/GenAIItemPartViewModel.cs | Added TryGetPropertyValue helper method for retrieving additional properties |
| src/Aspire.Dashboard/Components/Dialogs/GenAIVisualizerDialog.razor.cs | Refactored image detection to support blob and uri types with MIME validation instead of custom "image" type |
| src/Aspire.Dashboard/Components/Dialogs/GenAIVisualizerDialog.razor | Updated method call from IsImagePart to TryGetImagePart |
| playground/Stress/Stress.ApiService/Program.cs | Updated test data to use new blob and uri types with proper semantic attributes |
src/Aspire.Dashboard/Components/Dialogs/GenAIVisualizerDialog.razor.cs
Outdated
Show resolved
Hide resolved
|
For other blobs, should we have a download button? I'm thinking for files like xlsx, pdf, etc. Also for text like "text/*" or "application/json", render as text? Is there any generalized "show these bytes as this mime type" browser component this could use? |
1e10fea to
35fc8f4
Compare
|
@stephentoub Improved:
|
|
|
||
| // AI generated list of extensions. | ||
| // Combines 100 commonly used mime types with https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types. | ||
| public static readonly Dictionary<string, string> MimeToExtension = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase) |
There was a problem hiding this comment.
Yet another example where dotnet/runtime#121017 will be useful.
|
Very nice. |
|
@stephentoub There is also a video element - https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/video Do you think that should be supported? |
I don't know how common it'll be, but I also don't think it can really hurt to have it, so, sure :) |
|
Done 
|
Description
Update GenAI visualizer to use the multimodal standard added in open-telemetry/semantic-conventions#2754
cc @stephentoub
Raw:
Preview:
Checklist
<remarks />and<code />elements on your triple slash comments?doc-ideatemplatebreaking-changetemplatediagnostictemplate