Skip to content

Revise prompt optimization documentation for clarity#18579

Merged
TomeHirata merged 7 commits intomlflow:masterfrom
TomeHirata:docs/optimize-prompt/framing
Nov 5, 2025
Merged

Revise prompt optimization documentation for clarity#18579
TomeHirata merged 7 commits intomlflow:masterfrom
TomeHirata:docs/optimize-prompt/framing

Conversation

@TomeHirata
Copy link
Collaborator

@TomeHirata TomeHirata commented Oct 30, 2025 •
edited by github-actions bot
Loading

🛠 DevTools 🛠

Open in GitHub Codespaces

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/18579/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/18579/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/18579/merge

Related Issues/PRs

n/a

What changes are proposed in this pull request?

Revise prompt optimization documentation for clarity and emphasis on framework agnosticism. Highlight key benefits and ease of integration with various agent frameworks, ensuring users understand the minimal code changes required for optimization.

How is this PR tested?

  • Existing unit/integration tests
  • New unit/integration tests
  • Manual tests

Does this PR require documentation update?

  • No. You can skip the rest of this section.
  • Yes. I've updated:
    • Examples
    • API references
    • Instructions

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/tracking: Tracking Service, tracking client APIs, autologging
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/scoring: MLflow Model server, model deployment tools, Spark UDFs
  • area/evaluation: MLflow model evaluation features, evaluation metrics, and evaluation workflows
  • area/gateway: MLflow AI Gateway client APIs, server, and third-party integrations
  • area/prompts: MLflow prompt engineering features, prompt templates, and prompt management
  • area/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionality
  • area/projects: MLproject format, project running backends
  • area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages

How should the PR be classified in the release notes? Choose one:

  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes should be selected for bug fixes, documentation updates, and other small changes. No should be selected for new features and larger changes. If you're unsure about the release classification of this PR, leave this unchecked to let the maintainers decide.

What is a minor/patch release?
  • Minor release: a release that increments the second part of the version number (e.g., 1.2.0 -> 1.3.0).
    Bug fixes, doc updates and new features usually go into minor releases.
  • Patch release: a release that increments the third part of the version number (e.g., 1.2.0 -> 1.2.1).
    Bug fixes and doc updates usually go into patch releases.
  • Yes (this PR will be cherry-picked and included in the next patch release)
  • No (this PR will be included in the next minor release)

@TomeHirata
Revise prompt optimization documentation for clarity and emphasis on …
0763e32 
…framework agnosticism. Highlight key benefits and ease of integration with various agent frameworks, ensuring users understand the minimal code changes required for optimization.

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
@TomeHirata TomeHirata requested a review from Copilot October 30, 2025 02:45
@github-actions github-actions bot added area/docs Documentation issues rn/none List under Small Changes in Changelogs. v3.5.2 labels Oct 30, 2025
Copilot AI reviewed Oct 30, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates the documentation for MLflow's prompt optimization feature to emphasize its framework-agnostic nature and ease of use. The changes reframe the feature's value proposition to highlight minimal code changes and universal compatibility across agent frameworks.

  • Added marketing-focused taglines emphasizing simplicity and lack of vendor lock-in
  • Restructured the benefits section to highlight framework agnosticism and minimal code changes
  • Updated section heading from "Use Agent Framework" to "Works with Any Agent Framework" with expanded framework-agnostic messaging

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@TomeHirata TomeHirata added the team-review Trigger a team review request label Oct 30, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 30, 2025 •
edited
Loading

Documentation preview for 7451985 is available at:

Changed Pages (1)
More info
  • Ignore this comment if this PR does not change the documentation.
  • The preview is updated when a new commit is pushed to this PR.
  • This comment was created by this workflow run.
  • The documentation was built by this workflow run.

@TomeHirata
update example dataset
fcd1f79 
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
@TomeHirata
Merge remote-tracking branch 'mlflow/master' into docs/optimize-promp…
8942fd0 
…t/framing
@TomeHirata
fix style
af04e8d 
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
@TomeHirata
lint
e6626d5 
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
@TomeHirata
css
670dd30 
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
TomeHirata
TomeHirata commented Nov 5, 2025
View reviewed changes
docs/docs/genai/prompt-registry/optimize-prompts.mdx

# Training data with inputs and expected outputs
# Training data with medical paper sentences and ground truth labels
raw_data = [
Copy link
Collaborator Author

@TomeHirata TomeHirata Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a bit more complex example so that users can see the improvement just by copy-pasting the example code

@TomeHirata
format
7451985 
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
daniellok-db
daniellok-db reviewed Nov 5, 2025
View reviewed changes
docs/docs/genai/prompt-registry/optimize-prompts.mdx
optimizer=GepaPromptOptimizer(
reflection_model="openai:/gpt-5", max_metric_calls=300
),
scorers=[Correctness(model="openai:/gpt-5-mini")],
Copy link
Collaborator

@daniellok-db daniellok-db Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any particular reason we use mini for scorer?

Copy link
Collaborator Author

@TomeHirata TomeHirata Nov 5, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for the cost purpose, mini should be sufficient 😄

daniellok-db
daniellok-db approved these changes Nov 5, 2025
View reviewed changes
Copy link
Collaborator

@daniellok-db daniellok-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall LGTM!

@TomeHirata TomeHirata added this pull request to the merge queue Nov 5, 2025
Merged via the queue into mlflow:master with commit e692cb0 Nov 5, 2025
48 of 50 checks passed
@TomeHirata TomeHirata deleted the docs/optimize-prompt/framing branch November 5, 2025 23:34
B-Step62 pushed a commit to B-Step62/mlflow that referenced this pull request Nov 7, 2025
@TomeHirata @B-Step62
Revise prompt optimization documentation for clarity (mlflow#18579)
3e729f6 
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
@github-actions github-actions bot added v3.6.1 and removed v3.6.0 labels Nov 8, 2025
B-Step62 pushed a commit to B-Step62/mlflow that referenced this pull request Nov 11, 2025
@TomeHirata @B-Step62
Revise prompt optimization documentation for clarity (mlflow#18579)
4fc0cd9 
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
B-Step62 pushed a commit that referenced this pull request Nov 11, 2025
@TomeHirata @B-Step62
Revise prompt optimization documentation for clarity (#18579)
ff1cb5e 
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
@B-Step62 B-Step62 added v3.6.0 and removed v3.6.1 labels Nov 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@daniellok-db daniellok-db daniellok-db approved these changes

@B-Step62 B-Step62 Awaiting requested review from B-Step62

@BenWilson2 BenWilson2 Awaiting requested review from BenWilson2

@harupy harupy Awaiting requested review from harupy

@kevin-lyn kevin-lyn Awaiting requested review from kevin-lyn

@serena-ruan serena-ruan Awaiting requested review from serena-ruan

@WeichenXu123 WeichenXu123 Awaiting requested review from WeichenXu123

@xq-yin xq-yin Awaiting requested review from xq-yin

Assignees

No one assigned

Labels

area/docs Documentation issues rn/none List under Small Changes in Changelogs. team-review Trigger a team review request v3.5.2 v3.6.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@TomeHirata @daniellok-db @B-Step62