Skip to content

Improve dashboard GenAI telemetry UI#11227

Merged
JamesNK merged 38 commits intomainfrom
jamesnk/genai
Sep 8, 2025
Merged

Improve dashboard GenAI telemetry UI#11227
JamesNK merged 38 commits intomainfrom
jamesnk/genai

Conversation

@JamesNK
Copy link
Member

@JamesNK JamesNK commented Sep 3, 2025

Description

GenAI interactions are hard to read in Aspire's default telemetry experience:

  • GenAI information is spread across span attributes, span events and log entries.
  • Input and output content is embedded in complex JSON payloads. It is possible but awkward to read simple text messages from JSON payloads. Large large text messages, or messages which are JSON/XML encoded inside JSON, are very difficult to understand.
  • The standard has changed significately 3 times so far.

This PR adds a specialized GenAI dialog for viewing GenAI operations. When GenAI telemetry is detected then a button is added to GenAI spans and logs. The button launches a GenAI viewer dialog and displays content in an easy-to-use UI. This approach is similar to what Aspire does for log entries with exception messages, which have a dialog dedicated to viewing log entry exception details.

image

CircuitBrain is the button icon and seems like the most universial icon for GenAI at the moment. Feedback welcome.

The dialog displays highlevel information about the GenAI operation (name, model, start time, duration, token usage). It also displays all the input and output information. What were the system instructions, input messages and tool call results (including the message history of assistant responses) and output messages.

image

Clicking on a message displays its content in more detail. The "Raw" tab displays the raw text without formatting. The "Preview" tab displays content in a user friendly preview to the user:

  • JSON and XML content is indented and highlighted.
  • Other content is assumed to be markdown, and rendered as markdown.

There is a button to copy message content to the clipboard.

image

Other PR features/changes:

  • There have been 3 different ways of representing GenAI input and output content. All 3 are supported. Each is attempted, newest to oldest, to find where content is stored.
  • The dialog has "Next" and "Previous" buttons so you can navigate between GenAI operations in the current span without closing and reopening the dialog.
  • Change chat AI integration to use telemetry reported by MEAI instead of OpenAI or Azure.AI.Inference packages. MEAI is activitely updated. Telemetry in other packages is out of date and doesn't receive updates.

To do:

  • Add a configuration option to Aspire chat AI integration about capturing sensitive data (i.e. message content)
  • Move text to resource files

Major follow up work, that's not part of this PR:

  • Add ability to quickly filter traces and spans to known types. e.g. select "GenAI" in a dropdown to only see traces and spans that have GenAI operations in them.

Fixes # (issue)

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No
  • Does the change require an update in our Aspire docs?

@JamesNK JamesNK added this to the 9.5 milestone Sep 3, 2025
@JamesNK JamesNK requested review from adamint and Copilot September 3, 2025 02:17
@maddymontaquila
Copy link
Member

cc @LadyNaggaga

Bravo !!! Now, one suggestion replace the brain icon sparkles. This is awesome to see happen @JamesNK and @maddymontaquila

Yes! Agree that sparkles ✨ is the canonical AI icon these days. Circuit brain is more used re: cognitive svcs. Pulling this down to try out later today!

@shanselman
Copy link

I LOVE IT

@JamesNK
Copy link
Member Author

JamesNK commented Sep 4, 2025

This is ready to review. PR failures are related to telemetry changes and are fixed in #11237

@JamesNK JamesNK mentioned this pull request Sep 4, 2025
16 tasks
@JamesNK
Copy link
Member Author

JamesNK commented Sep 8, 2025

Display error information on failure:

image

@JamesNK
Copy link
Member Author

JamesNK commented Sep 8, 2025

Handle no message content. This can happen when libraries is recording telemetry, but requires additional configuration to include sensitive message content in recorded telemetry.

Idea here is link will explain why message content isn't available (security sensitive data) and how to enable it.

image

@davidfowl
Copy link
Member

Played with this on my ai chat sample and it works, but revealed that we need to update the CommunityToolkit Ollama integration to mirror the changs we made to the openai client and azure ai client packages.

Also, markdown does not show up nicely:

image

Copy link
Member

@davidfowl davidfowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should do some stress testing to make sure it doesn't fall over processing these types of spans.

@JamesNK
Copy link
Member Author

JamesNK commented Sep 8, 2025

Markdown? Markfixed.

image image

Copy link
Member

@mitchdenny mitchdenny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I've walked through the code and the changes seem reasonable to me and I tested it out locally as well. Seems like our existing telemetry truncation will help avoid the UX being overwhelmed with data although not sure if this is heavier on the DOM rendering or not (do we need extra limits to only show the last X entries with GenAI?).

The other thing I wondered was to what extend do we need to fuzz telemetry input to see if we can somehow bypass the safe rendering protections you've got in place for this since the rendering is a bit more elaborate here than elsewhere.

@maddymontaquila
Copy link
Member

thoughts on adding a button here too - maybe next to "View Logs" it could be "View LLM Details" or something
image

@JamesNK
Copy link
Member Author

JamesNK commented Sep 8, 2025

Yes, but buttons will need to move to a menu to fit in available UI space. And it can be in a future PR.

Create an issue.

@JamesNK
Copy link
Member Author

JamesNK commented Sep 8, 2025

The other thing I wondered was to what extend do we need to fuzz telemetry input to see if we can somehow bypass the safe rendering protections you've got in place for this since the rendering is a bit more elaborate here than elsewhere.

It's using the same Markdown rendering tech as dashboard copilot chat and interaction service (except now with safe URL schemes). I think it's fine.

@JamesNK JamesNK merged commit ec65ce7 into main Sep 8, 2025
305 checks passed
@JamesNK JamesNK deleted the jamesnk/genai branch September 8, 2025 23:56
@github-actions github-actions bot locked and limited conversation to collaborators Oct 9, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

ai area-dashboard area-integrations Issues pertaining to Aspire Integrations packages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants