Nondeterministic, multi-agent systems break traditional monitoring. As agents reach production, observability must be built in—not added after failures. This session covers modern agent observability: cross-framework tracing and evals, rigorous inner-loop practices, evolving context-specific evals, and always-on signals that connect behavior to business outcomes to measure value, cost, and ROI.
 |
| Watch the Session Replay | BRK252: From observability to ROI for AI agents on any framework |
| Complete the Hands-On Lab | LAB540: Observe, optimize and protect your hosted agents in Microsoft Foundry |
| Read The Dev Blog Post | Build 2026: From observability to ROI for AI agents on any framework |
| Download The Session Slides | BRK252 "Download Slides" Link From Session Page |
Reliable AI Agent development needs observability. Agents are inherently non-deterministic, creating new challenges in reliability and consistency for developers and operators. Delivering trustworthy solutions requires a unified end-to-end observability platform that can help you understand and optimize the cost, health and behavior of agents in real time.
In this session, you will walk through the end-to-end agent devops lifecycle with a focus on using observability tools and workflows to go from prototype to production. You will get started with out-of-the-box observabilty (plan-code-test-release) - then fast forward to production (monitor, analyze, optimize) and learn to hill climb with confidence using the new Foundry Optimizer capability -- with live demos at each step that show you how to:
- observe any agent, on any framework - with tracing and evals
- get code-first observability - with skills-based guided experiences
- optimize for scale - with Foundry Optimizer to do your hill climbing
- prove the value of your product - with Agent ROI using portal and API
By the end of this session you will be able to:
- Apply cross-framework tracing and evaluation techniques to gain visibility into nondeterministic, multi-agent systems
- Establish rigorous inner-loop practices that build observability into agents before they reach production
- Design and evolve context-specific evals that keep pace with changing agent behavior and requirements
- Connect always-on production signals to business outcomes to measure agent value, cost, and ROI
- Optimize for scale with a unified end-to-end observability capabilities
_Try these prompts in GitHub Copilot Chat to explore topics in more detail. Open a new chat session in Visual Studio Code (
Ctrl+Alt+Ion Windows/Linux,Cmd+Shift+Ion Mac) and use a prompt below as a starting point - or write your own! 💡 Tip: connect the Microsoft Learn MCP Server first to ground responses in official docs.
| Topic | Prompt |
|---|---|
| Distributed tracing in Microsoft Foundry | Walk me through how to enable distributed tracing for an AI agent in Microsoft Foundry, including how to view thread results and spans across tools and model calls. Reference Trace and observe AI agents in Microsoft Foundry and Observability in the Agent Framework |
| Cross-framework observability | Explain the core observability capabilities in Microsoft Foundry and how they cover the three stages of the AI application lifecycle (ideate, build, operate) for multi-agent systems. Ground your answer in Observability in generative AI. |
| Evals from inner loop to production | Show me how to run cloud evaluations with the Microsoft Foundry SDK and connect trace data to evaluation results so I can evolve context-specific evals as my agent changes. Use Run evaluations in the cloud by using the Microsoft Foundry SDK as the source. |
| Connecting observability to ROI* | How do I use the Agent Monitoring Dashboard and fleet-wide monitoring in Microsoft Foundry together with cost optimization to measure agent value, cost, and ROI in production? Reference Monitor agents with the Agent Monitoring Dashboard, Monitor agent health and performance across your fleet, and Optimize model cost and performance. |
- Microsoft Foundry — the unified platform for building, deploying, and managing AI agents
- Microsoft Foundry Control Plane — manage and monitor agent fleets at scale across environments
- Microsoft Foundry Observability — tracing, evaluations, and monitoring across the agent lifecycle
| Resource | Description |
|---|---|
| Microsoft Foundry Agent Service | Build, deploy, and manage AI agents on Microsoft Foundry |
| Microsoft Agent Framework | Open framework for building single- and multi-agent systems across .NET and Python |
| Microsoft Agent Framework Workflows | Orchestrate multi-agent workflows with durable, observable execution |
| Observability in generative AI | Core observability capabilities across the ideate, build, and operate stages |
| Trace and observe AI agents in Microsoft Foundry | Enable distributed tracing for agents and inspect spans in the Foundry portal |
| Agent Framework observability | OpenTelemetry-based tracing and logging for agents across frameworks |
| Run cloud evaluations with the Foundry SDK | Run continuous, context-specific evals tied to trace data |
| View evaluation results in the Foundry portal | Compare runs and track quality, safety, and performance over time |
| Monitor agents with the Agent Monitoring Dashboard | Always-on production signals for a single agent |
| Monitor agent health across your fleet | Cross-agent monitoring for production operations |
| Optimize model cost and performance | Connect observability signals to cost and ROI decisions |
| Plan and manage costs for Microsoft Foundry | Budgeting and cost monitoring for Foundry workloads |
| https://aka.ms/build26-next-steps | Explore lab and session repos to further your learning from Microsoft Build |
The Microsoft Learn MCP Server gives your AI agent direct access to Microsoft's official documentation — grounded, up-to-date answers about the products and services covered in this session.
VS Code — One click installation:
GitHub Copilot CLI — Run this to install the Learn MCP Server as a plugin:
/plugin install microsoftdocs/mcp
For more info, other clients, and to post questions, visit the Learn MCP Server repo.
 Sebastian Kohlmeier 📢 |
 Filisha Shah 📢 |
 Vivek Bhadauria 📢 |
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit Contributor License Agreements.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.