Support SparkDF trace handling in eval by BenWilson2 · Pull Request #20207 · mlflow/mlflow

BenWilson2 · 2026-01-22T02:38:19Z

Related Issues/PRs

#xxx

What changes are proposed in this pull request?

If traces are in a SparkDF, when transferring to a PandasDF, the StructType encodings are cast to dict type directly. This PR adds in parsing logic to ensure that the traces are deserialized properly.

How is this PR tested?

Existing unit/integration tests
New unit/integration tests
Manual tests

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

How should the PR be classified in the release notes? Choose one:

rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes should be selected for bug fixes, documentation updates, and other small changes. No should be selected for new features and larger changes. If you're unsure about the release classification of this PR, leave this unchecked to let the maintainers decide.

What is a minor/patch release?

Minor release: a release that increments the second part of the version number (e.g., 1.2.0 -> 1.3.0).
Bug fixes, doc updates and new features usually go into minor releases.
Patch release: a release that increments the third part of the version number (e.g., 1.2.0 -> 1.2.1).
Bug fixes and doc updates usually go into patch releases.

Yes (this PR will be cherry-picked and included in the next patch release)
No (this PR will be included in the next minor release)

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

github-actions · 2026-01-22T02:38:30Z

🛠 DevTools 🛠

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/20207/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/20207/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/20207/merge

Copilot

Pull request overview

This PR adds support for handling traces in SparkDF format within the evaluation pipeline. When a Spark DataFrame containing trace columns (stored as StructType) is converted to pandas via .toPandas(), the trace column becomes a dict instead of remaining as a Trace object or JSON string. This PR extends the deserialization logic to handle this dict format.

Changes:

Extended _deserialize_trace_column_if_needed function to handle dict-type trace values by calling Trace.from_dict()
Updated docstring to document the new dict handling behavior
Added comprehensive tests covering string, dict, and Trace object formats

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
mlflow/genai/evaluation/utils.py	Extended trace deserialization logic to handle dict format from Spark DataFrame conversion
tests/genai/evaluate/test_utils.py	Added parameterized test covering all three trace formats (dict, string, Trace object)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mlflow/genai/evaluation/utils.py

tests/genai/evaluate/test_utils.py

github-actions · 2026-01-22T02:46:16Z

Documentation preview for df5eec9 is available at:

https://pr-20207--mlflow-docs-preview.netlify.app/docs/latest/

More info

Ignore this comment if this PR does not change the documentation.
The preview is updated when a new commit is pushed to this PR.
This comment was created by this workflow run.
The documentation was built by this workflow run.

harupy · 2026-01-22T02:46:50Z

mlflow/genai/evaluation/utils.py

+        if isinstance(t, Trace):
+            return t


Suggested change

if isinstance(t, Trace):

return t

is this necessary?

good call - final match state should just pass through. Will update

harupy · 2026-01-22T02:59:33Z

tests/genai/evaluate/test_utils.py

+@pytest.fixture
+def sample_trace_dict():
+    return {
+        "info": {
+            "trace_id": "test-trace-id",
+            "trace_location": {
+                "type": "MLFLOW_EXPERIMENT",
+                "mlflow_experiment": {"experiment_id": "0"},
+            },
+            "request_time": "2024-01-21T12:00:00Z",
+            "state": "OK",
+            "trace_metadata": {},
+            "tags": {},
+            "assessments": [],
+        },
+        "data": {"spans": []},
+    }
+
+
+@pytest.mark.parametrize("trace_format", ["dict", "string", "trace_object"])


nit: Consider using pytest.param with explicit IDs for cleaner test output:

Suggested change

@pytest.fixture

def sample_trace_dict():

return {

"info": {

"trace_id": "test-trace-id",

"trace_location": {

"type": "MLFLOW_EXPERIMENT",

"mlflow_experiment": {"experiment_id": "0"},

},

"request_time": "2024-01-21T12:00:00Z",

"state": "OK",

"trace_metadata": {},

"tags": {},

"assessments": [],

},

"data": {"spans": []},

}

@pytest.mark.parametrize("trace_format", ["dict", "string", "trace_object"])

def _trace_test_cases():

data = {

"info": {

"trace_id": "test-trace-id",

"trace_location": {

"type": "MLFLOW_EXPERIMENT",

"mlflow_experiment": {"experiment_id": "0"},

},

"request_time": "2024-01-21T12:00:00Z",

"state": "OK",

"trace_metadata": {},

"tags": {},

"assessments": [],

},

"data": {"spans": []},

}

return [

pytest.param(data, dict, id="dict"),

pytest.param(json.dumps(data), str, id="string"),

pytest.param(Trace.from_dict(data), Trace, id="trace_object"),

]

@pytest.mark.parametrize(("trace_value", "expected_input_type"), _trace_test_cases())

🤖 Generated with Claude

Great suggestion!

harupy · 2026-01-22T03:00:48Z

mlflow/genai/evaluation/utils.py

+    if "trace" not in df.columns:
+        return df
+
+    def deserialize_trace(t):


Can we move this outside _deserialize_trace_column_if_needed?

yep will move out for better readability

harupy · 2026-01-22T03:06:02Z

mlflow/genai/evaluation/utils.py

+    def deserialize_trace(t):
+        if isinstance(t, Trace):
+            return t
+        if isinstance(t, str):
+            return Trace.from_json(t)
+        if isinstance(t, dict):
+            return Trace.from_dict(t)
+        return t


nit: Consider using a match statement for cleaner type dispatch:

Suggested change

def deserialize_trace(t):

if isinstance(t, Trace):

return t

if isinstance(t, str):

return Trace.from_json(t)

if isinstance(t, dict):

return Trace.from_dict(t)

return t

def deserialize_trace(t):

match t:

case Trace():

return t

case str():

return Trace.from_json(t)

case dict():

return Trace.from_dict(t)

case _:

return t

🤖 Generated with Claude

100% right call

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy

LGTM

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

Support SparkDF trace handling in eval

95a971d

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

Copilot AI review requested due to automatic review settings January 22, 2026 02:38

github-actions bot added v3.9.0 area/evaluation MLflow Evaluation rn/bug-fix Mention under Bug Fixes in Changelogs. labels Jan 22, 2026

BenWilson2 added the team-review Trigger a team review request label Jan 22, 2026

Copilot started reviewing on behalf of BenWilson2 January 22, 2026 02:38 View session

github-actions bot requested review from B-Step62 and harupy January 22, 2026 02:38

Copilot AI reviewed Jan 22, 2026

View reviewed changes

mlflow/genai/evaluation/utils.py Outdated Show resolved Hide resolved

tests/genai/evaluate/test_utils.py Outdated Show resolved Hide resolved

harupy reviewed Jan 22, 2026

View reviewed changes

github-actions bot assigned harupy Jan 22, 2026

updates

df5eec9

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

BenWilson2 requested a review from harupy January 22, 2026 15:01

harupy approved these changes Jan 22, 2026

View reviewed changes

BenWilson2 enabled auto-merge January 22, 2026 16:13

BenWilson2 added this pull request to the merge queue Jan 22, 2026

Merged via the queue into mlflow:master with commit 9bced0d Jan 22, 2026
66 of 70 checks passed

BenWilson2 deleted the eval-df-fix branch January 22, 2026 16:26

harupy pushed a commit to harupy/mlflow that referenced this pull request Jan 28, 2026

Support SparkDF trace handling in eval (mlflow#20207)

059cba9

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy pushed a commit to harupy/mlflow that referenced this pull request Jan 28, 2026

Support SparkDF trace handling in eval (mlflow#20207)

cabd909

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy pushed a commit that referenced this pull request Jan 28, 2026

Support SparkDF trace handling in eval (#20207)

949dad3

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

Conversation

BenWilson2 commented Jan 22, 2026

Related Issues/PRs

What changes are proposed in this pull request?

How is this PR tested?

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Should this PR be included in the next patch release?

Uh oh!

github-actions bot commented Jan 22, 2026

Install mlflow from this PR

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

harupy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Jan 22, 2026 •

edited

Loading