[Sample] Add Microsoft.Maui.Essentials.AI sample app with multi-agent workflow by mattleibow · Pull Request #33610 · dotnet/maui

mattleibow · 2026-01-19T21:16:54Z

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Description

This PR introduces the Essentials.AI.Sample application, a comprehensive demonstration of integrating AI capabilities into .NET MAUI applications using Microsoft.Extensions.AI abstractions. The sample showcases advanced AI patterns including multi-agent workflows, RAG (Retrieval-Augmented Generation), streaming responses, and structured output generation.

What's Added

Sample Application: Trip Planner

A cross-platform travel itinerary generator that demonstrates production-ready AI integration patterns:

Feature	Description
Multi-Agent Workflow	4 specialized agents collaborating via Microsoft.Agents.AI
RAG (Semantic Search)	Embedding-based destination matching before AI selection
Streaming Responses	Real-time `IAsyncEnumerable<T>` for progressive UI updates
Structured JSON Output	`ChatResponseFormat.ForJsonSchema<T>()` for typed responses
Function Calling	`AIFunction` tool for POI discovery
Conditional Translation	Automatic translation for non-English requests
Cross-Platform	iOS, Android, Windows, and macOS support

Agent Workflow Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    User Request                                 │
│           "Plan a 3-day trip to Maui in Indonesian"             │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│              1. Travel Planner Agent                            │
│    Extracts: destination, dayCount, targetLanguage              │
│    (no tools - structured JSON output only)                     │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│              2. Researcher Agent                                │
│    Step 1: RAG semantic search finds candidate destinations     │
│    Step 2: AI selects best match from candidates                │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│              3. Itinerary Planner Agent                         │
│    Uses findPointsOfInterest() tool, streams JSON itinerary     │
└─────────────────────────────────────────────────────────────────┘
                              │
               ┌──────────────┴──────────────┐
               │   targetLanguage check      │
               └──────────────┬──────────────┘
                              │
          ┌───────────────────┼───────────────────┐
          │                   │                   │
          ▼                   │                   ▼
   ┌────────────┐             │            ┌────────────┐
   │ "English"  │             │            │ Non-English│
   └─────┬──────┘             │            └─────┬──────┘
         │                    │                  │
         │                    │                  ▼
         │                    │   ┌─────────────────────────────┐
         │                    │   │   4. Translator Agent       │
         │                    │   │   Streams translated JSON   │
         │                    │   └──────────────┬──────────────┘
         │                    │                  │
         └────────────────────┼──────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                    5. Output Executor                           │
│         Final structured multi-day travel itinerary             │
└─────────────────────────────────────────────────────────────────┘

Services and Utilities

Service	Purpose
`StreamingJsonDeserializer`	Deserializes partial/incomplete JSON during streaming for progressive UI updates
`JsonMerger`	Merges incremental JSON updates smoothly (handles arrays, nested objects)
`BufferedChatClient`	Aggregates streaming responses into complete responses for downstream agents
`NonFunctionInvokingChatClient`	Prevents double tool invocation when local models are wrapped with function calling
`DataService`	Provides landmark data and semantic search via embeddings
`TaggingService`	AI-powered tag extraction from landmark descriptions
`ItineraryService`	Orchestrates the workflow agent invocation

Test Infrastructure

Category	Description
Device Test Base Classes	Reusable test bases for `IChatClient` and `IEmbeddingGenerator` implementations
OpenAI Device Tests	Validation tests using Azure OpenAI
Unit Tests	`StreamingJsonDeserializer`, `JsonMerger`, `BufferedChatClient`, `NonFunctionInvokingChatClient`
Test Data	Real AI response streams captured for reproducible testing
Benchmarks	Performance testing for streaming JSON deserialization

Test coverage includes:

Chat client: instantiation, messaging, streaming, JSON schema, function calling, cancellation
Embedding generator: instantiation, generation, concurrency, similarity, disposal
Streaming utilities: partial JSON parsing, incremental merging, edge cases

Library Skeleton (Microsoft.Maui.Essentials.AI)

This PR establishes the project structure for the Essentials.AI library:

src/AI/
├── samples/
│   └── Essentials.AI.Sample/     # Full sample application
├── src/
│   └── Essentials.AI/            # Library (platform APIs added in future PRs)
│       └── PublicAPI/            # Platform-specific API tracking
└── tests/
    ├── Essentials.AI.DeviceTests/
    ├── Essentials.AI.UnitTests/
    └── Essentials.AI.Benchmarks/

Setup Requirements

The sample requires Azure OpenAI credentials configured via user secrets:

{
    "AI": {
        "DeploymentName": "<chat-model-deployment>",
        "EmbeddingDeploymentName": "<embedding-model-deployment>",
        "Endpoint": "<azure-openai-endpoint>",
        "ApiKey": "<azure-openai-api-key>"
    }
}

See src/AI/samples/Essentials.AI.Sample/README.md for detailed setup instructions.

Key AI Patterns Demonstrated

Pattern	Implementation
Workflow-as-Agent	Multi-agent workflow registered as single `AIAgent` for clean invocation
RAG	Researcher uses embedding-based semantic search before AI selection
Structured Output	`ChatResponseFormat.ForJsonSchema<T>()` enforces C# record types
Function Calling	`AIFunctionFactory.Create()` defines the `findPointsOfInterest` tool
Streaming	Agents use `RunStreamingAsync()` to emit partial JSON progressively
Partial Deserialization	`StreamingJsonDeserializer` shows progressive updates from incomplete JSON
Conditional Edges	Workflow branches based on typed output conditions

Dependencies Added

Package	Purpose
`Microsoft.Extensions.AI`	Core AI abstractions (`IChatClient`, `IEmbeddingGenerator`)
`Microsoft.Extensions.AI.Abstractions`	AI content types and interfaces
`Microsoft.Agents.AI`	Multi-agent workflow framework
`OpenAI`	Azure OpenAI client (sample only, conditional compilation)

Future Work

This PR establishes the foundation. Subsequent PRs will add:

Apple Intelligence (AppleIntelligenceChatClient, NLEmbeddingGenerator)
Windows Copilot Runtime (PhiSilicaChatClient, PhiSilicaEmbeddingGenerator)
Android Gemini Nano (GeminiNanoChatClient)

These on-device implementations will allow the sample to run with local models instead of cloud services.

Copilot

Pull request overview

This PR introduces the Essentials.AI.Sample application demonstrating advanced AI integration patterns with .NET MAUI. The PR includes a trip planner sample showcasing multi-agent workflows, RAG, streaming responses, and structured output generation, along with comprehensive test infrastructure.

Changes:

Adds new AI sample application with multi-agent trip planning workflow
Introduces library skeleton for Microsoft.Maui.Essentials.AI
Adds extensive unit tests for streaming JSON deserialization, JSON merging, and buffered chat clients
Modifies build system to use XcodeProject instead of pre-built .xcframework.zip for Apple platforms
Includes comprehensive test data files with real AI response streams

Reviewed changes

Copilot reviewed 201 out of 251 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/PublicAPI.targets	Enables automatic PublicAPI generation in Debug mode
src/Core/src/Core.csproj	Switches from NativeReference to XcodeProject for Apple bindings
src/AppleBindings.targets	New shared build logic for Apple XcodeProject items with CI support
src/Core/AppleNative/PlatformInterop/build-xcframework.sh	Removed (replaced by XcodeProject build)
src/Core/AppleNative/PlatformInterop/MauiPlatformInterop.xcframework.zip	Removed
src/AI/tests/Essentials.AI.UnitTests/Tests/*	Comprehensive unit tests for streaming utilities
src/AI/tests/Essentials.AI.UnitTests/TestData/*	Test data with real AI streaming responses
src/AI/tests/Essentials.AI.UnitTests/TestHelpers/*	Test helper utilities
src/AI/tests/Essentials.AI.UnitTests/Models/Location.cs	Simple location model for tests
src/AI/tests/Essentials.AI.DeviceTests/Resources/appicon.svg	Device test app icon

mattleibow · 2026-02-02T18:55:32Z

/azp run maui-pr-devicetests

mattleibow · 2026-02-02T18:55:42Z

/azp run maui-pr-uitests

azure-pipelines · 2026-02-02T18:55:50Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2026-02-02T18:55:54Z

Azure Pipelines successfully started running 1 pipeline(s).

mattleibow · 2026-02-04T16:37:38Z

/azp run

azure-pipelines · 2026-02-04T16:37:58Z

Azure Pipelines successfully started running 3 pipeline(s).

… workflow Introduces the Essentials.AI module with a Trip Planner sample demonstrating production-ready AI integration patterns using Microsoft.Extensions.AI and Microsoft.Agents.AI. Sample Application (Essentials.AI.Sample): - 4-agent workflow: TravelPlanner → Researcher → ItineraryPlanner → Translator - RAG with embedding-based semantic search for destination matching - Streaming JSON responses with partial deserialization for progressive UI - Function calling (findPointsOfInterest tool) and structured JSON output - Conditional translation branching for non-English requests Services & Utilities: - StreamingJsonDeserializer, JsonMerger, BufferedChatClient - DataService (landmarks + embeddings), TaggingService, ItineraryService Library Skeleton (Essentials.AI): - Multi-platform TFM support with PublicAPI tracking - Foundation for future Apple Intelligence, Windows Copilot, Gemini Nano Test Infrastructure: - Unit tests: JSON streaming, merging, buffering (real AI response data) - Device tests: Reusable IChatClient/IEmbeddingGenerator base classes - Benchmarks: BenchmarkDotNet infrastructure

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [Microsoft.Extensions.Logging.Debug](https://dot.net/) ([source](https://github.com/dotnet/dotnet)) | nuget | patch | `10.0.2` -> `10.0.3` | | [Microsoft.Maui.Controls](https://github.com/dotnet/maui) | nuget | patch | `10.0.31` -> `10.0.41` | --- ### Release Notes <details> <summary>dotnet/maui (Microsoft.Maui.Controls)</summary> ### [`v10.0.41`](https://github.com/dotnet/maui/releases/tag/10.0.41): SR4.1 [Compare Source](dotnet/maui@10.0.40...10.0.41) #### What's Changed - \[release/10.0.1xx-sr4] Revert "Remove InternalsVisibleTo attributes for .NET MAUI Community … by [@github-actions](https://github.com/github-actions)\[bot] in dotnet/maui#34052 - Increment patch version from 40 to 41 by [@PureWeen](https://github.com/PureWeen) in dotnet/maui#34059 **Full Changelog**: dotnet/maui@10.0.40...10.0.41 ### [`v10.0.40`](https://github.com/dotnet/maui/releases/tag/10.0.40): SR4 [Compare Source](dotnet/maui@10.0.31...10.0.40) #### What's Changed .NET MAUI 10.0.40 introduces significant improvements across all platforms with focus on quality, performance, and developer experience. This release includes 143 commits with various improvements, bug fixes, and enhancements. #### AI - Improve write-tests-agent with best practices by [@sheiksyedm](https://github.com/sheiksyedm) in dotnet/maui#33860 - \[Sample] Add Microsoft.Maui.Essentials.AI sample app with multi-agent workflow by [@mattleibow](https://github.com/mattleibow) in dotnet/maui#33610 #### Ai Agents - Add FileLoggingProvider for MacCatalyst UI test logging by [@PureWeen](https://github.com/PureWeen) in dotnet/maui#33518 - Improve verify-tests-fail-without-fix Skill by [@kubaflo](https://github.com/kubaflo) in dotnet/maui#33513 - Add find-reviewable-pr skill from existing PR by [@PureWeen](https://github.com/PureWeen) via [@Copilot](https://github.com/Copilot) in dotnet/maui#33349 - Add learn-from-pr agent and enhance skills framework structure by [@PureWeen](https://github.com/PureWeen) via [@Copilot](https://github.com/Copilot) in dotnet/maui#33579 - Fix PS1 scripts for Windows compatibility by [@PureWeen](https://github.com/PureWeen) in dotnet/maui#33679 - Improve skills and scripts for better agent workflows by [@PureWeen](https://github.com/PureWeen) in dotnet/maui#33699 - \[XEXPR] Refactor test skills/agents to dispatcher pattern by [@PureWeen](https://github.com/PureWeen) via [@Copilot](https://github.com/Copilot) in https://github.com/dot...

mattleibow force-pushed the dev/ai-full-sample branch from f8d09df to 76a7e93 Compare January 19, 2026 21:32

This was referenced Jan 20, 2026

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

[android-arm64] The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#6408

Open

mattleibow marked this pull request as ready for review January 22, 2026 15:17

Copilot AI review requested due to automatic review settings January 22, 2026 15:17

Copilot started reviewing on behalf of mattleibow January 22, 2026 15:18 View session

Copilot AI reviewed Jan 22, 2026

View reviewed changes

kubaflo added area-ai-agents Copilot CLI agents, agent skills, AI-assisted development area-ai and removed area-ai-agents Copilot CLI agents, agent skills, AI-assisted development labels Jan 28, 2026

mattleibow force-pushed the dev/ai-full-sample branch from 76a7e93 to d44d3ae Compare February 2, 2026 16:36

This was referenced Feb 3, 2026

CI UItests failure System.TimeoutException : Timed out waiting for element... #33220

Open

System.TimeoutException #33245

Open

This was referenced Feb 4, 2026

[Known Build Error] iOS UITest screenshot baseline mismatches after ACES pool migration #33879

Open

System.Exception : Failed to launch Test AVD. #33862

Open

kubaflo approved these changes Feb 6, 2026

View reviewed changes

mattleibow force-pushed the dev/ai-full-sample branch from 2850d84 to 334e1c0 Compare February 7, 2026 13:12

mattleibow merged commit 0942bb9 into main Feb 9, 2026
24 of 27 checks passed

mattleibow deleted the dev/ai-full-sample branch February 9, 2026 23:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Sample] Add Microsoft.Maui.Essentials.AI sample app with multi-agent workflow#33610

[Sample] Add Microsoft.Maui.Essentials.AI sample app with multi-agent workflow#33610
mattleibow merged 1 commit intomainfrom
dev/ai-full-sample

mattleibow commented Jan 19, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

mattleibow commented Feb 2, 2026

Uh oh!

mattleibow commented Feb 2, 2026

Uh oh!

azure-pipelines bot commented Feb 2, 2026

Uh oh!

azure-pipelines bot commented Feb 2, 2026

Uh oh!

mattleibow commented Feb 4, 2026

Uh oh!

azure-pipelines bot commented Feb 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

mattleibow commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

What's Added

Sample Application: Trip Planner

Agent Workflow Architecture

Services and Utilities

Test Infrastructure

Library Skeleton (Microsoft.Maui.Essentials.AI)

Setup Requirements

Key AI Patterns Demonstrated

Dependencies Added

Future Work

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

mattleibow commented Feb 2, 2026

Uh oh!

mattleibow commented Feb 2, 2026

Uh oh!

azure-pipelines bot commented Feb 2, 2026

Uh oh!

azure-pipelines bot commented Feb 2, 2026

Uh oh!

mattleibow commented Feb 4, 2026

Uh oh!

azure-pipelines bot commented Feb 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mattleibow commented Jan 19, 2026 •

edited

Loading