Add OpenAI Agent Optimization guide by TomeHirata · Pull Request #18792 · mlflow/mlflow

TomeHirata · 2025-11-12T01:56:28Z

🛠 DevTools 🛠

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/18792/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/18792/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/18792/merge

Related Issues/PRs

n/a

What changes are proposed in this pull request?

We will add pages for how to use optimize_prompts with major agent frameworks. This PR introduces OpenAI Agent example.

Introduced a new documentation file for optimizing prompts with OpenAI Agents.
Updated the sidebar configuration to include a new category for "Optimize Prompts" with a link to the new guide.

How is this PR tested?

Existing unit/integration tests
New unit/integration tests
Manual tests

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

How should the PR be classified in the release notes? Choose one:

rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes should be selected for bug fixes, documentation updates, and other small changes. No should be selected for new features and larger changes. If you're unsure about the release classification of this PR, leave this unchecked to let the maintainers decide.

What is a minor/patch release?

Minor release: a release that increments the second part of the version number (e.g., 1.2.0 -> 1.3.0).
Bug fixes, doc updates and new features usually go into minor releases.
Patch release: a release that increments the third part of the version number (e.g., 1.2.0 -> 1.2.1).
Bug fixes and doc updates usually go into patch releases.

Yes (this PR will be cherry-picked and included in the next patch release)
No (this PR will be included in the next minor release)

- Introduced a new documentation file for optimizing prompts with OpenAI Agents. - Updated the sidebar configuration to include a new category for "Optimize Prompts" with a link to the new guide. Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

Copilot

Pull Request Overview

This PR adds documentation for optimizing prompts with OpenAI Agents. It introduces a new guide demonstrating how to use MLflow's optimize_prompts API with the OpenAI Agent framework to improve agent prompts automatically.

Adds a new documentation file for OpenAI Agent optimization with a complete working example
Updates the sidebar to create a nested category structure for "Optimize Prompts" with a link to the new guide

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
docs/sidebarsGenAI.ts	Converts the "Optimize Prompts" entry from a single doc link to a nested category containing the new OpenAI Agent optimization guide
docs/docs/genai/prompt-registry/optimize-prompts/openai-agent-optimization.mdx	New documentation page with prerequisites, setup instructions, and a complete code example for optimizing OpenAI Agent prompts

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

docs/docs/genai/prompt-registry/optimize-prompts/openai-agent-optimization.mdx

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

github-actions · 2025-11-12T02:14:25Z

Documentation preview for c6b6486 is available at:

https://pr-18792--mlflow-docs-preview.netlify.app/docs/latest/

Changed Pages (1)

genai/prompt-registry/optimize-prompts/openai-agent-optimization (added)

More info

Ignore this comment if this PR does not change the documentation.
The preview is updated when a new commit is pushed to this PR.
This comment was created by this workflow run.
The documentation was built by this workflow run.

harupy · 2025-11-12T17:16:57Z

@TomeHirata Sorry, I accidentally pushed the Update Branch button. Can you revert it with force push?

docs/docs/genai/prompt-registry/optimize-prompts/openai-agent-optimization.mdx

- Added `nest_asyncio` to the installation instructions for better compatibility in notebook environments. - Clarified the context for using `nest_asyncio` in the example. - Updated the type hint for the `exact_match` function to improve code clarity. Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

B-Step62 · 2025-11-13T02:58:29Z

docs/docs/genai/prompt-registry/optimize-prompts/openai-agent-optimization.mdx

+
+# Optimizing Prompts for OpenAI Agents
+
+<p style={{display: 'flex', justifyContent: 'center', margin: '1em 0'}}>


nit: the image is truncated, can we fix the stylying?

Is it truncated by styling? The original picture is this: https://mlflow.org/docs/latest/images/logos/openai-agent-logo.png, which is exactly what's displayed in the doc.

Can you fix the original image then?

Yeah, I thought about the same and googled, but couldn't find the original logo. It seems the file was added by you. Where did you find the original logo? I can file a follow-up if we can find the replacement.

docs/docs/genai/prompt-registry/optimize-prompts/openai-agent-optimization.mdx

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

harupy · 2025-11-13T05:39:20Z

docs/docs/genai/prompt-registry/optimize-prompts/openai-agent-optimization.mdx

+Here's a complete example of optimizing a question-answering agent:
+
+```python
+import asyncio


Suggested change

import asyncio

harupy

LGTM!

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com> Signed-off-by: Tian Lan <sky.blue266000@gmail.com>

Add OpenAI Agent Optimization guide

0114f6f

- Introduced a new documentation file for optimizing prompts with OpenAI Agents. - Updated the sidebar configuration to include a new category for "Optimize Prompts" with a link to the new guide. Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

TomeHirata requested a review from Copilot November 12, 2025 01:56

github-actions bot added v3.6.1 area/docs Documentation issues rn/none List under Small Changes in Changelogs. labels Nov 12, 2025

Copilot started reviewing on behalf of TomeHirata November 12, 2025 01:57 View session

Copilot finished reviewing on behalf of TomeHirata November 12, 2025 01:58

Copilot AI reviewed Nov 12, 2025

View reviewed changes

docs/docs/genai/prompt-registry/optimize-prompts/openai-agent-optimization.mdx Outdated Show resolved Hide resolved

TomeHirata added 2 commits November 12, 2025 10:59

lint

e0c6784

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

comment

d36bb3c

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

TomeHirata added the team-review Trigger a team review request label Nov 12, 2025

github-actions bot requested review from B-Step62, BenWilson2, WeichenXu123, daniellok-db, harupy, kevin-lyn, serena-ruan and xq-yin November 12, 2025 02:01

harupy reviewed Nov 13, 2025

View reviewed changes

docs/docs/genai/prompt-registry/optimize-prompts/openai-agent-optimization.mdx Outdated Show resolved Hide resolved

harupy reviewed Nov 13, 2025

View reviewed changes

docs/docs/genai/prompt-registry/optimize-prompts/openai-agent-optimization.mdx Outdated Show resolved Hide resolved

TomeHirata force-pushed the docs/openai-agent-optimize branch from ff6818d to c2ff6b1 Compare November 13, 2025 02:57

B-Step62 reviewed Nov 13, 2025

View reviewed changes

comment

b355d26

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

TomeHirata requested review from B-Step62 and harupy November 13, 2025 03:54

harupy reviewed Nov 13, 2025

View reviewed changes

harupy approved these changes Nov 13, 2025

View reviewed changes

comment

c6b6486

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

TomeHirata enabled auto-merge November 13, 2025 06:48

TomeHirata added this pull request to the merge queue Nov 13, 2025

Merged via the queue into mlflow:master with commit 95ba674 Nov 13, 2025
45 checks passed

TomeHirata deleted the docs/openai-agent-optimize branch November 13, 2025 07:39

Tian-Sky-Lan pushed a commit to Tian-Sky-Lan/mlflow that referenced this pull request Nov 24, 2025

Add OpenAI Agent Optimization guide (mlflow#18792)

017a942

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com> Signed-off-by: Tian Lan <sky.blue266000@gmail.com>


		# Optimizing Prompts for OpenAI Agents

		<p style={{display: 'flex', justifyContent: 'center', margin: '1em 0'}}>

Conversation

TomeHirata commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Install mlflow from this PR

Related Issues/PRs

What changes are proposed in this pull request?

How is this PR tested?

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Should this PR be included in the next patch release?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

github-actions bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

harupy commented Nov 12, 2025

Uh oh!

Uh oh!

Uh oh!

B-Step62 Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

TomeHirata Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

B-Step62 Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

TomeHirata Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

harupy Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

harupy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

TomeHirata commented Nov 12, 2025 •

edited

Loading

github-actions bot commented Nov 12, 2025 •

edited

Loading

TomeHirata Nov 13, 2025 •

edited

Loading