Skip to content

Latest commit

 

History

History

README.md

IMPORTANT! All samples and other resources made available in this GitHub repository ("samples") are designed to assist in accelerating development of agents, solutions, and agent workflows for various scenarios. Review all provided resources and carefully test output behavior in the context of your use case. AI responses may be inaccurate and AI actions should be monitored with human oversight. Learn more in the transparency note for Agent Service.

Agents, solutions, or other output you create may be subject to legal and regulatory requirements, may require licenses, or may not be suitable for all industries, scenarios, or use cases. By using any sample, you are acknowledging that any output created using those samples are solely your responsibility, and that you will comply with all applicable laws, regulations, and relevant safety standards, terms of service, and codes of conduct.

Third-party samples contained in this folder are subject to their own designated terms, and they have not been tested or verified by Microsoft or its affiliates.

Microsoft has no responsibility to you or others with respect to any of these samples or any resulting output.

Important

Agent Optimizer is currently in limited preview and only available through a sign-up process. To access the service, complete the intake form. This preview is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

What this sample demonstrates

A realistic customer support agent for a consumer electronics company — ready for the agent optimizer in Foundry Agent Service. This sample demonstrates meaningful optimization delta: the baseline agent starts with a bare prompt and the agent optimizer rewrites instructions and discovers skills that dramatically improve response quality.

Use this sample to:

  • See how optimization transforms a weak baseline into a production-quality agent
  • Understand instruction optimization (prompt rewriting) and skill discovery
  • Learn the full optimization workflow with a realistic evaluation dataset

The Scenario

Contoso Electronics needs a support agent that handles:

  • Order status inquiries and tracking
  • Return/exchange processing
  • Warranty claims
  • Technical troubleshooting
  • Complaint handling with empathy
  • Escalation to human agents

The baseline prompt is deliberately bare: "You are a helpful customer support agent." The agent optimizer rewrites this with specific policies, tone guidelines, and procedural skills.

Optimization target What it does Expected improvement
Instruction Rewrites the system prompt with policies, formatting, guardrails Baseline ~0.40 → Optimized ~0.80+
Skill Discovers procedural skills (troubleshooting, returns, escalation) Adds structured behaviors

How It Works

Model Integration

The agent uses the Foundry SDK with the Responses protocol. It loads optimized instructions via load_config() and applies them to every model call.

Optimization Config Loading

Same pattern as the hello-world sample:

  1. Optimization candidateOPTIMIZATION_CANDIDATE_ID env var (during evaluation)
  2. Local baseline.agent_configs/baseline/instructions.md
  3. Hardcoded fallback — Bare prompt

Evaluation Dataset

eval.jsonl contains 10 evaluation tasks with ~40 criteria covering:

  • Order inquiries, returns, warranty claims
  • Technical troubleshooting, escalation decisions
  • Complaint handling, product recommendations
  • Safety guardrails (no medical/legal advice)

Running the Agent Locally

Prerequisites

  1. Azure Developer CLI (azd) v1.25.3+

    • Install azd
    • Install the agent optimizer extension: azd ext install azure.ai.agents
    • Authenticate: azd auth login
  2. Azure CLIaz login

  3. Python 3.12+

Environment Variables

See .env.example for the full list.

Variable Required Description
FOUNDRY_PROJECT_ENDPOINT Yes Foundry project endpoint. Auto-injected in hosted containers.
AZURE_AI_MODEL_DEPLOYMENT_NAME Yes Model deployment name (e.g., gpt-4.1-mini).
OPTIMIZATION_LOCAL_DIR Yes Path to agent config directory (default: .agent_configs).
APPLICATIONINSIGHTS_CONNECTION_STRING Recommended Enables telemetry.

Option 1: Azure Developer CLI (azd)

Initialize the agent project

azd ai agent init -m https://github.com/microsoft-foundry/foundry-samples/blob/main/samples/python/hosted-agents/bring-your-own/responses/optimization-customer-support/agent.manifest.yaml

The interactive flow prompts for your Azure subscription, region, and model deployment settings. This generates azure.yaml, infrastructure-as-code files, and configures the environment.

Provision and deploy

cd optimization-customer-support-python-responses
az login
azd auth login
azd provision
azd deploy

Invoke the agent

azd ai agent invoke "I ordered a laptop 3 days ago and haven't received a shipping confirmation. Order #12345."

Option 2: Foundry Toolkit VS Code Extension

  1. Clone this repo and open this sample folder in VS Code.
  2. Start locally: azd ai agent run
  3. Open Command Palette → Foundry Toolkit: Open Agent Inspector to chat.
  4. When ready: Foundry Toolkit: Deploy Hosted Agent.

Running Optimization

This sample ships with eval.yaml and eval.jsonl — run optimization out of the box.

Run optimization

azd ai agent optimize

Monitor progress

azd ai agent optimize status <job-id> --watch

Apply the best candidate

azd ai agent optimize apply --candidate <candidate-id>

Verify improvement

azd ai agent invoke "I want to return my headphones. They stopped working after 2 weeks."

Compare the response quality before and after optimization — you should see structured troubleshooting steps, empathetic language, and clear policy references.

Files

File Purpose
main.py Agent entry point — Responses handler + optimization config loading
agent.yaml Hosted agent deployment config
agent.manifest.yaml Template manifest for azd ai agent init
Dockerfile Container image build
requirements.txt Python dependencies (includes optimization wheel)
eval.yaml Agent optimizer configuration (dataset, evaluators, models)
eval.jsonl Full evaluation dataset (10 tasks, ~40 criteria)
eval-quick.jsonl Quick evaluation dataset (3 tasks — faster iteration)
.agent_configs/baseline/ Baseline agent config (bare instructions, model metadata)
skills/ Skill definitions (populated after skill optimization)
.env.example Environment variable documentation