Skip to content

Add LangChain optimization guide#18836

Merged
TomeHirata merged 3 commits intomlflow:masterfrom
TomeHirata:docs/optimize-langchain
Dec 1, 2025
Merged

Add LangChain optimization guide#18836
TomeHirata merged 3 commits intomlflow:masterfrom
TomeHirata:docs/optimize-langchain

Conversation

@TomeHirata
Copy link
Collaborator

@TomeHirata TomeHirata commented Nov 14, 2025

🛠 DevTools 🛠

Open in GitHub Codespaces

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/18836/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/18836/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/18836/merge

Related Issues/PRs

n/a

What changes are proposed in this pull request?

Add LangChain optimization guide

How is this PR tested?

  • Existing unit/integration tests
  • New unit/integration tests
  • Manual tests

Does this PR require documentation update?

  • No. You can skip the rest of this section.
  • Yes. I've updated:
    • Examples
    • API references
    • Instructions

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/tracking: Tracking Service, tracking client APIs, autologging
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/scoring: MLflow Model server, model deployment tools, Spark UDFs
  • area/evaluation: MLflow model evaluation features, evaluation metrics, and evaluation workflows
  • area/gateway: MLflow AI Gateway client APIs, server, and third-party integrations
  • area/prompts: MLflow prompt engineering features, prompt templates, and prompt management
  • area/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionality
  • area/projects: MLproject format, project running backends
  • area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages

How should the PR be classified in the release notes? Choose one:

  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes should be selected for bug fixes, documentation updates, and other small changes. No should be selected for new features and larger changes. If you're unsure about the release classification of this PR, leave this unchecked to let the maintainers decide.

What is a minor/patch release?
  • Minor release: a release that increments the second part of the version number (e.g., 1.2.0 -> 1.3.0).
    Bug fixes, doc updates and new features usually go into minor releases.
  • Patch release: a release that increments the third part of the version number (e.g., 1.2.0 -> 1.2.1).
    Bug fixes and doc updates usually go into patch releases.
  • Yes (this PR will be cherry-picked and included in the next patch release)
  • No (this PR will be included in the next minor release)

…optimization documentation

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
@github-actions github-actions bot added v3.6.1 area/docs Documentation issues rn/none List under Small Changes in Changelogs. labels Nov 14, 2025
@TomeHirata TomeHirata added the team-review Trigger a team review request label Nov 14, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 14, 2025

Documentation preview for 77dddb4 is available at:

Changed Pages (2)

More info
  • Ignore this comment if this PR does not change the documentation.
  • The preview is updated when a new commit is pushed to this PR.
  • This comment was created by this workflow run.
  • The documentation was built by this workflow run.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a comprehensive LangChain-specific optimization guide to the MLflow documentation, making it easier for LangChain users to understand how to optimize their prompts using MLflow's prompt optimization API.

Key Changes:

  • Added a new dedicated guide for LangChain prompt optimization with a complete working example
  • Updated the main optimization guide to reference framework-specific guides instead of inline examples
  • Integrated the new guide into the documentation navigation structure

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
docs/sidebarsGenAI.ts Added the new LangChain optimization guide to the sidebar navigation under "Optimize Prompts"
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx Created a new comprehensive guide demonstrating LangChain prompt optimization with prerequisites, complete example, and integration notes
docs/docs/genai/prompt-registry/optimize-prompts.mdx Refactored to replace inline LangChain example with a link to the dedicated guide, improving documentation organization

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

# Optimizing Prompts for LangChain

<p style={{display: 'flex', justifyContent: 'center', margin: '1em 0'}}>
<img src={useBaseUrl("/images/logos/langchain-logo.png")} alt="LangChain Logo" style={{width: 300, objectFit: 'contain'}} />
Copy link
Member

@harupy harupy Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need the logo?

Screenshot 2025-11-17 at 19 23 33

It would look better without it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's good to have some visuals? Without this logo, this page feels a bit too simple. I have the logo in https://mlflow.org/docs/latest/genai/tracing/integrations/listing/langchain/ too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, then we can keep it

]

# Step 4: Optimize the prompt
result = mlflow.genai.optimize_prompts(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got this warning during optimization:

Warning: Failed to log to mlflow: Invalid value "Task: Translate text from a specified source language to a specified target language and return only the translated text.\n\nInput format\n- You will receive three fields:\n  - input_language: the source language.\n  - output_language: the target language.\n  - text: the content to translate.\n\nOutput format\n- Return exactly one line containing only the translation.\n- Do not include quotes, labels, explanations, or additional text.\n- Do not add or remove sentences or punctuation that meaningfully changes the text.\n- Do not add leading/trailing spaces or blank lines.\n\nGeneral requirements\n- Preserve the original meaning and tone, choosing a standard, neutral, and polite/formal register when the target language distinguishes between formal and informal address.\n  - French: prefer \u201cvous\u201d forms by default (e.g., \u201callez-vous\u201d rather than \u201cvas-tu\u201d).\n  - Spanish: prefer formal register (\u201custed\u201d) when a choice is required.\n  - German: prefer \u201cSie\u201d register when a choice is required.\n- Maintain correct punctuation and capitalization for the target language, but keep formatting simple to ensure exact-match grading:\n  - Use standard ASCII punctuation without inserting extra spaces before ? or ! in French.\n  - Preserve obvious source punctuation (e.g., if the source is a question, the translation should be a question).\n  - Use standard hyphenation in interrogative inversions in French (e.g., \u201callez-vous\u201d).\n- Use correct diacritics and orthography in the target language (e.g., d\u00edas, Vielen Dank).\n- Do not transliterate or add explanations, alternatives, or parentheses.\n- Do not add or remove honorifics or politeness markers unless required for grammaticality and the neutral formal register.\n- If multiple valid translations exist, choose the most common, standard, and formal phrasing that is likely to be used in general contexts.\n\nExact matches for known phrases (ensure these exact outputs when they apply)\n- English \u2192 French\n  - \u201cHello, how are you?\u201d \u2192 \u201cBonjour, comment allez-vous?\u201d\n- English \u2192 Spanish\n  - \u201cGood morning\u201d \u2192 \u201cBuenos d\u00edas\u201d\n- English \u2192 German\n  - \u201cThank you very much\u201d \u2192 \u201cVielen Dank\u201d\n\nQuality checks before finalizing\n- Verify exact spelling, diacritics, capitalization, and punctuation.\n- Ensure there are no extra spaces (especially before question marks in French) and no trailing whitespace.\n- Output only the translated text, with nothing else." for parameter 'metrics[0].value' supplied: (timestamp=1763375298103). Please specify value as a valid double (64-bit floating point)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's a known issue, I'm talking to the gepa author about the solution

…agent integration

- Updated installation command to include `litellm`.
- Introduced a new system prompt for agent creation.
- Refactored the prediction function to utilize the new agent and user prompt.
- Removed outdated sections and examples for clarity.

Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
@TomeHirata TomeHirata requested a review from harupy November 19, 2025 05:24
@harupy harupy requested a review from Copilot November 19, 2025 08:53
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
Copy link
Member

@harupy harupy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@TomeHirata TomeHirata added this pull request to the merge queue Dec 1, 2025
Merged via the queue into mlflow:master with commit 72ffce5 Dec 1, 2025
50 of 52 checks passed
@TomeHirata TomeHirata deleted the docs/optimize-langchain branch December 1, 2025 01:46
BenWilson2 pushed a commit to BenWilson2/mlflow that referenced this pull request Dec 4, 2025
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
BenWilson2 pushed a commit that referenced this pull request Dec 4, 2025
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Documentation issues rn/none List under Small Changes in Changelogs. team-review Trigger a team review request v3.6.1 v3.7.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants