Test langchain v1 by B-Step62 · Pull Request #18802 · mlflow/mlflow

B-Step62 · 2025-11-12T05:25:58Z

🛠 DevTools 🛠

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/18802/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/18802/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/18802/merge

What changes are proposed in this pull request?

Enable LangChain v1 tests and fix them.

How is this PR tested?

Existing unit/integration tests
New unit/integration tests
Manual tests

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

How should the PR be classified in the release notes? Choose one:

rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes should be selected for bug fixes, documentation updates, and other small changes. No should be selected for new features and larger changes. If you're unsure about the release classification of this PR, leave this unchecked to let the maintainers decide.

What is a minor/patch release?

Minor release: a release that increments the second part of the version number (e.g., 1.2.0 -> 1.3.0).
Bug fixes, doc updates and new features usually go into minor releases.
Patch release: a release that increments the third part of the version number (e.g., 1.2.0 -> 1.2.1).
Bug fixes and doc updates usually go into patch releases.

Yes (this PR will be cherry-picked and included in the next patch release)
No (this PR will be included in the next minor release)

github-actions · 2025-11-12T05:40:09Z

Documentation preview for cd222c7 is available at:

https://pr-18802--mlflow-docs-preview.netlify.app/docs/latest/

More info

Ignore this comment if this PR does not change the documentation.
The preview is updated when a new commit is pushed to this PR.
This comment was created by this workflow run.
The documentation was built by this workflow run.

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

B-Step62 · 2025-11-20T02:53:00Z

tests/langchain/sample_code/openai_agent.py

-
-agent = create_openai_agent()
+llm = FakeOpenAI()
+agent = create_agent(llm, [add, multiply], system_prompt="You are a helpful assistant")


note: AgentExecutor is removed and now create_agent returns a langgraph-based agent.

B-Step62 · 2025-11-20T02:54:11Z

tests/langchain/test_langchain_autolog.py

-
-# NB: We run this test suite twice - once with langchain_community installed and once without.
-try:
-    from langchain_community.chat_models import ChatOpenAI


note: ChatOpenAI in the community version is deprecated since 0.0.10, we don't need to test

B-Step62 · 2025-11-20T02:55:03Z

tests/langchain/test_langchain_autolog.py

 # So the expected output will be the prompt itself.
 TEST_CONTENT = "What is MLflow?"

+_SIMPLE_MODEL_CODE_PATH = "tests/langchain/sample_code/simple_runnable.py"


note: Serialization-based logging does not work with LangChain v1. Switching to code-based logging only. Will update the doc in a follow-up.

B-Step62 · 2025-11-20T02:55:43Z

tests/langchain/test_langchain_autolog.py

    assert trace.data.spans[0].name == "always_fail"


-def test_llmchain_autolog(async_logging_enabled):


note: LLMChain has been deprecated since 0.1.17

B-Step62 · 2025-11-20T02:56:13Z

tests/langchain/test_langchain_autolog.py

+
+@skip_when_testing_trace_sdk
+@pytest.mark.parametrize("log_traces", [True, False])
+def test_langchain_tracing_evaluate(log_traces):


note: Updated evaluation test to use the new GenAI evaluate and removed the legacy test_langchain_evaluation.py

B-Step62 · 2025-11-20T02:56:42Z

tests/langchain/test_langchain_databricks_dependency_extraction.py



 def get_vector_search(
-    use_partner_package: bool,


note: We only need to test the partner package (databricks-langchain). The community version has been deprecated long ago.

B-Step62 · 2025-11-20T02:57:34Z

tests/langchain/test_langchain_document_conversion.py

note: Duplicates to a test case in callback test.

B-Step62 · 2025-11-20T02:59:20Z

tests/langchain/test_langchain_model_export.py

note: There are many tests that use chain classes deprecated long ago (0.1.x). Removed most of them.

Maybe we can remove them first in a separate PR to reduce the amount of diff?

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

BenWilson2

Quite the overhaul. All the legacy cleanup looks great!

tests/langchain/test_langchain_model_export.py

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> Signed-off-by: Tian Lan <sky.blue266000@gmail.com>

B-Step62 added the enable-dev-tests Enables cross-version tests for dev versions label Nov 12, 2025

B-Step62 force-pushed the fix-lc-dev branch 2 times, most recently from 38e3417 to 936fbac Compare November 14, 2025 18:05

B-Step62 changed the title ~~Test langchain v1~~ [WIP] Test langchain v1 Nov 14, 2025

B-Step62 added 8 commits November 20, 2025 11:47

test langchain v1

0fe9db4

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

update versions

50fcc2e

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

langgraph

0a80d0a

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

update autolog tests

5ea5c5c

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

Fix databricks tests

1279d0e

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

Update langchain eval test

8c85e13

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

Fix langchain model export tests

ebdea80

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

fix

f6d155f

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

B-Step62 force-pushed the fix-lc-dev branch from 06bb362 to f6d155f Compare November 20, 2025 02:49

B-Step62 commented Nov 20, 2025

View reviewed changes

tests/langchain/test_langchain_document_conversion.py

Copy link

Collaborator Author

B-Step62 Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: Duplicates to a test case in callback test.

B-Step62 commented Nov 20, 2025

View reviewed changes

B-Step62 added 2 commits November 20, 2025 12:02

fix

27202a1

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

fix?

4273a45

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

B-Step62 added team-review Trigger a team review request rn/none List under Small Changes in Changelogs. labels Nov 20, 2025

github-actions bot requested review from BenWilson2, daniellok-db, harupy and kevin-lyn November 20, 2025 10:31

github-actions bot requested review from TomeHirata, WeichenXu123, serena-ruan and xq-yin November 20, 2025 10:31

github-actions bot added area/build Build and test infrastructure for MLflow and removed area/build Build and test infrastructure for MLflow labels Nov 20, 2025

B-Step62 changed the title ~~[WIP] Test langchain v1~~ Test langchain v1 Nov 20, 2025

BenWilson2 approved these changes Nov 20, 2025

View reviewed changes

harupy reviewed Nov 20, 2025

View reviewed changes

tests/langchain/test_langchain_model_export.py Outdated Show resolved Hide resolved

B-Step62 added 2 commits November 21, 2025 16:51

comment

9f1f4d4

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

fix model test

cd222c7

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

B-Step62 added this pull request to the merge queue Nov 21, 2025

Merged via the queue into mlflow:master with commit 82edf51 Nov 21, 2025
69 checks passed

B-Step62 deleted the fix-lc-dev branch November 21, 2025 11:30

jimilp7 pushed a commit to backspace-org/mlflow that referenced this pull request Nov 21, 2025

Test langchain v1 (mlflow#18802)

e884899

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

Tian-Sky-Lan pushed a commit to Tian-Sky-Lan/mlflow that referenced this pull request Nov 24, 2025

Test langchain v1 (mlflow#18802)

5036f94

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> Signed-off-by: Tian Lan <sky.blue266000@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test langchain v1#18802

Test langchain v1#18802
B-Step62 merged 12 commits intomlflow:masterfrom
B-Step62:fix-lc-dev

B-Step62 commented Nov 12, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 12, 2025 •

edited

Loading

Uh oh!

B-Step62 Nov 20, 2025

Uh oh!

B-Step62 Nov 20, 2025 •

edited

Loading

Uh oh!

B-Step62 Nov 20, 2025

Uh oh!

B-Step62 Nov 20, 2025

Uh oh!

B-Step62 Nov 20, 2025

Uh oh!

B-Step62 Nov 20, 2025

Uh oh!

B-Step62 Nov 20, 2025

Uh oh!

B-Step62 Nov 20, 2025

Uh oh!

harupy Nov 20, 2025 •

edited

Loading

Uh oh!

BenWilson2 left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		assert trace.data.spans[0].name == "always_fail"


		def test_llmchain_autolog(async_logging_enabled):

Conversation

B-Step62 commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Install mlflow from this PR

What changes are proposed in this pull request?

How is this PR tested?

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Should this PR be included in the next patch release?

Uh oh!

github-actions bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

B-Step62 Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

B-Step62 Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

B-Step62 Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

B-Step62 Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

B-Step62 Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

B-Step62 Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

B-Step62 Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

B-Step62 Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

harupy Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BenWilson2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

B-Step62 commented Nov 12, 2025 •

edited

Loading

github-actions bot commented Nov 12, 2025 •

edited

Loading

B-Step62 Nov 20, 2025 •

edited

Loading

harupy Nov 20, 2025 •

edited

Loading