Add LangChain optimization guide#18836
Conversation
…optimization documentation Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
|
Documentation preview for 77dddb4 is available at: Changed Pages (2)
More info
|
There was a problem hiding this comment.
Pull Request Overview
This PR adds a comprehensive LangChain-specific optimization guide to the MLflow documentation, making it easier for LangChain users to understand how to optimize their prompts using MLflow's prompt optimization API.
Key Changes:
- Added a new dedicated guide for LangChain prompt optimization with a complete working example
- Updated the main optimization guide to reference framework-specific guides instead of inline examples
- Integrated the new guide into the documentation navigation structure
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
docs/sidebarsGenAI.ts |
Added the new LangChain optimization guide to the sidebar navigation under "Optimize Prompts" |
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx |
Created a new comprehensive guide demonstrating LangChain prompt optimization with prerequisites, complete example, and integration notes |
docs/docs/genai/prompt-registry/optimize-prompts.mdx |
Refactored to replace inline LangChain example with a link to the dedicated guide, improving documentation organization |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx
Outdated
Show resolved
Hide resolved
| # Optimizing Prompts for LangChain | ||
|
|
||
| <p style={{display: 'flex', justifyContent: 'center', margin: '1em 0'}}> | ||
| <img src={useBaseUrl("/images/logos/langchain-logo.png")} alt="LangChain Logo" style={{width: 300, objectFit: 'contain'}} /> |
There was a problem hiding this comment.
I think it's good to have some visuals? Without this logo, this page feels a bit too simple. I have the logo in https://mlflow.org/docs/latest/genai/tracing/integrations/listing/langchain/ too.
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx
Outdated
Show resolved
Hide resolved
| ] | ||
|
|
||
| # Step 4: Optimize the prompt | ||
| result = mlflow.genai.optimize_prompts( |
There was a problem hiding this comment.
Got this warning during optimization:
Warning: Failed to log to mlflow: Invalid value "Task: Translate text from a specified source language to a specified target language and return only the translated text.\n\nInput format\n- You will receive three fields:\n - input_language: the source language.\n - output_language: the target language.\n - text: the content to translate.\n\nOutput format\n- Return exactly one line containing only the translation.\n- Do not include quotes, labels, explanations, or additional text.\n- Do not add or remove sentences or punctuation that meaningfully changes the text.\n- Do not add leading/trailing spaces or blank lines.\n\nGeneral requirements\n- Preserve the original meaning and tone, choosing a standard, neutral, and polite/formal register when the target language distinguishes between formal and informal address.\n - French: prefer \u201cvous\u201d forms by default (e.g., \u201callez-vous\u201d rather than \u201cvas-tu\u201d).\n - Spanish: prefer formal register (\u201custed\u201d) when a choice is required.\n - German: prefer \u201cSie\u201d register when a choice is required.\n- Maintain correct punctuation and capitalization for the target language, but keep formatting simple to ensure exact-match grading:\n - Use standard ASCII punctuation without inserting extra spaces before ? or ! in French.\n - Preserve obvious source punctuation (e.g., if the source is a question, the translation should be a question).\n - Use standard hyphenation in interrogative inversions in French (e.g., \u201callez-vous\u201d).\n- Use correct diacritics and orthography in the target language (e.g., d\u00edas, Vielen Dank).\n- Do not transliterate or add explanations, alternatives, or parentheses.\n- Do not add or remove honorifics or politeness markers unless required for grammaticality and the neutral formal register.\n- If multiple valid translations exist, choose the most common, standard, and formal phrasing that is likely to be used in general contexts.\n\nExact matches for known phrases (ensure these exact outputs when they apply)\n- English \u2192 French\n - \u201cHello, how are you?\u201d \u2192 \u201cBonjour, comment allez-vous?\u201d\n- English \u2192 Spanish\n - \u201cGood morning\u201d \u2192 \u201cBuenos d\u00edas\u201d\n- English \u2192 German\n - \u201cThank you very much\u201d \u2192 \u201cVielen Dank\u201d\n\nQuality checks before finalizing\n- Verify exact spelling, diacritics, capitalization, and punctuation.\n- Ensure there are no extra spaces (especially before question marks in French) and no trailing whitespace.\n- Output only the translated text, with nothing else." for parameter 'metrics[0].value' supplied: (timestamp=1763375298103). Please specify value as a valid double (64-bit floating point)
There was a problem hiding this comment.
It appears https://github.com/gepa-ai/gepa/blob/2331a76cf0356f899fc87bebc3817c9ab0f1ca7b/src/gepa/logging/experiment_tracker.py#L109 emitted the warning message
There was a problem hiding this comment.
Yes, it's a known issue, I'm talking to the gepa author about the solution
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx
Outdated
Show resolved
Hide resolved
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx
Outdated
Show resolved
Hide resolved
…agent integration - Updated installation command to include `litellm`. - Introduced a new system prompt for agent creation. - Refactored the prediction function to utilize the new agent and user prompt. - Removed outdated sections and examples for clarity. Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx
Outdated
Show resolved
Hide resolved
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx
Show resolved
Hide resolved
There was a problem hiding this comment.
LGTM once https://github.com/mlflow/mlflow/pull/18836/files#r2541107105 is addressed!
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx
Show resolved
Hide resolved
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx
Show resolved
Hide resolved
docs/docs/genai/prompt-registry/optimize-prompts/langchain-optimization.mdx
Show resolved
Hide resolved
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>

🛠 DevTools 🛠
Install mlflow from this PR
For Databricks, use the following command:
Related Issues/PRs
n/a
What changes are proposed in this pull request?
Add LangChain optimization guide
How is this PR tested?
Does this PR require documentation update?
Release Notes
Is this a user-facing change?
What component(s), interfaces, languages, and integrations does this PR affect?
Components
area/tracking: Tracking Service, tracking client APIs, autologgingarea/models: MLmodel format, model serialization/deserialization, flavorsarea/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registryarea/scoring: MLflow Model server, model deployment tools, Spark UDFsarea/evaluation: MLflow model evaluation features, evaluation metrics, and evaluation workflowsarea/gateway: MLflow AI Gateway client APIs, server, and third-party integrationsarea/prompts: MLflow prompt engineering features, prompt templates, and prompt managementarea/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionalityarea/projects: MLproject format, project running backendsarea/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev serverarea/build: Build and test infrastructure for MLflowarea/docs: MLflow documentation pagesHow should the PR be classified in the release notes? Choose one:
rn/none- No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" sectionrn/breaking-change- The PR will be mentioned in the "Breaking Changes" sectionrn/feature- A new user-facing feature worth mentioning in the release notesrn/bug-fix- A user-facing bug fix worth mentioning in the release notesrn/documentation- A user-facing documentation change worth mentioning in the release notesShould this PR be included in the next patch release?
Yesshould be selected for bug fixes, documentation updates, and other small changes.Noshould be selected for new features and larger changes. If you're unsure about the release classification of this PR, leave this unchecked to let the maintainers decide.What is a minor/patch release?
Bug fixes, doc updates and new features usually go into minor releases.
Bug fixes and doc updates usually go into patch releases.