Support display in progress traces by B-Step62 · Pull Request #19265 · mlflow/mlflow

B-Step62 · 2025-12-08T05:10:11Z

🛠 DevTools 🛠

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/19265/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/19265/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/19265/merge

Related Issues

#16244

What changes are proposed in this pull request?

Current trace detail UI can only render a completed trace. However, now MLflow (SQL backend) support ingesting spans incrementally and the new GetTrace API can return spans from in-progress traces. This PR adds support to show in-progress trace.

The change can be broke down into three:

Call the new GetTrace endpoint when the server is OSS SQL backend (by checking the span location tag) to fetch spans from a in-progress trace.
Update trace renderer to handle multiple top-level spans. The root span is exported at the end, so the trace will have multiple roots until then.
Add auto-polling (only when the backend is OSS SQL) to fetch new spans automatically.

How is this PR tested?

Existing unit/integration tests
New unit/integration tests
Manual tests

In-progress.trace.mp4

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

Support displaying spans in a in-progress trace incrementally.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

How should the PR be classified in the release notes? Choose one:

rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes should be selected for bug fixes, documentation updates, and other small changes. No should be selected for new features and larger changes. If you're unsure about the release classification of this PR, leave this unchecked to let the maintainers decide.

What is a minor/patch release?

Minor release: a release that increments the second part of the version number (e.g., 1.2.0 -> 1.3.0).
Bug fixes, doc updates and new features usually go into minor releases.
Patch release: a release that increments the third part of the version number (e.g., 1.2.0 -> 1.2.1).
Bug fixes and doc updates usually go into patch releases.

Yes (this PR will be cherry-picked and included in the next patch release)
No (this PR will be included in the next minor release)

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

github-actions · 2025-12-08T05:17:47Z

Documentation preview for e2cfa2e is available at:

https://pr-19265--mlflow-docs-preview.netlify.app/docs/latest/

More info

Ignore this comment if this PR does not change the documentation.
The preview is updated when a new commit is pushed to this PR.
This comment was created by this workflow run.
The documentation was built by this workflow run.

mlflow/server/js/src/experiment-tracking/sdk/MlflowService.ts

mlflow/server/js/src/shared/web-shared/model-trace-explorer/ModelTraceExplorerDetailView.tsx

.../js/src/shared/web-shared/genai-traces-table/components/GenAiEvaluationTracesReviewModal.tsx

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

serena-ruan · 2025-12-12T06:26:29Z

mlflow/server/js/src/shared/web-shared/genai-traces-table/hooks/useGetTrace.tsx

+      // Stop polling if trace is completed
+      const traceState = data && isV3ModelTraceInfo(data.info) ? data.info.state : undefined;
+      return traceState === 'IN_PROGRESS' ? 1000 : false;


I wonder if this works well, if root_span is logged before child spans (e.g. concurrent span logging) then trace info status will be updated as OK and we stop polling.
Similar for databricks, if the trace info status is also generated from root span status then it's possible child spans are not logged?

Hmm that's true. If the root span is there, we can actually compare the span count and keep polling.

We still say status "OK", in that case, but I think it is valid since the traced operation itself is indeed completed, just trace logging is not complete.

Also there can be a case where a child span completely gets lost due to network issue, then this keep polling. But I think it is fine - we only poll while the trace modal is open and users will likely close it after waiting for a while.

Yeah the thing I want to avoid is that we stop polling and some spans are not logged, the user closes this page and then clicks into it again finding the structure changed. I previously discussed this with Daniel a bit, and feel like adding a refresh button may work better.

serena-ruan · 2025-12-12T06:26:29Z

mlflow/server/js/src/shared/web-shared/genai-traces-table/hooks/useGetTrace.tsx

+      // Stop polling if trace is completed
+      const traceState = data && isV3ModelTraceInfo(data.info) ? data.info.state : undefined;
+      return traceState === 'IN_PROGRESS' ? 1000 : false;


I wonder if this works well, if root_span is logged before child spans (e.g. concurrent span logging) then trace info status will be updated as OK and we stop polling.
Similar for databricks, if the trace info status is also generated from root span status then it's possible child spans are not logged?

Hmm that's true. If the root span is there, we can actually compare the span count and keep polling.

We still say status "OK", in that case, but I think it is valid since the traced operation itself is indeed completed, just trace logging is not complete.

Also there can be a case where a child span completely gets lost due to network issue, then this keep polling. But I think it is fine - we only poll while the trace modal is open and users will likely close it after waiting for a while.

Yeah the thing I want to avoid is that we stop polling and some spans are not logged, the user closes this page and then clicks into it again finding the structure changed. I previously discussed this with Daniel a bit, and feel like adding a refresh button may work better.

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

daniellok-db

looks good!

B-Step62 added 13 commits December 5, 2025 20:35

tree

9209f40

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

fix

f35f436

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

glitch

7e7b30f

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

Fix expand/collapse

7f5efd4

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

Remove dummy parent

83e0f7e

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

Show status

8990b15

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

fix

7ff2c0d

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

type

e5e69f6

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

fix

c5f27a4

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

fix

e1a508a

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

clean up

8ff0668

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

unit tests

721917c

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

Fix timeline view

b25bade

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

github-actions bot added area/tracing MLflow Tracing and its integrations rn/feature Mention under Features in Changelogs. labels Dec 8, 2025

prettify

365d2f0

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

daniellok-db reviewed Dec 11, 2025

View reviewed changes

mlflow/server/js/src/experiment-tracking/sdk/MlflowService.ts Outdated Show resolved Hide resolved

daniellok-db reviewed Dec 11, 2025

View reviewed changes

mlflow/server/js/src/shared/web-shared/model-trace-explorer/ModelTraceExplorerDetailView.tsx Outdated Show resolved Hide resolved

daniellok-db reviewed Dec 11, 2025

View reviewed changes

.../js/src/shared/web-shared/genai-traces-table/components/GenAiEvaluationTracesReviewModal.tsx Outdated Show resolved Hide resolved

github-actions bot assigned daniellok-db Dec 11, 2025

B-Step62 added 5 commits December 11, 2025 16:14

comments

8efe93c

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

Disable summary view if root is not present

560d72e

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

prettify

b8bd18b

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

key

fd26149

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

keys

59512d9

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

B-Step62 requested review from daniellok-db and hubertzub-db December 11, 2025 08:05

B-Step62 added the v3.8.0 label Dec 12, 2025

serena-ruan reviewed Dec 12, 2025

View reviewed changes

Check span count

e2cfa2e

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

daniellok-db approved these changes Dec 12, 2025

View reviewed changes

B-Step62 enabled auto-merge December 12, 2025 07:22

B-Step62 added this pull request to the merge queue Dec 12, 2025

Merged via the queue into mlflow:master with commit 11ed451 Dec 12, 2025
52 of 54 checks passed

B-Step62 deleted the support-display-in-progress-traces branch December 12, 2025 07:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support display in progress traces#19265

Support display in progress traces#19265
B-Step62 merged 20 commits intomlflow:masterfrom
B-Step62:support-display-in-progress-traces

B-Step62 commented Dec 8, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Dec 8, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

serena-ruan Dec 12, 2025

Uh oh!

B-Step62 Dec 12, 2025

Uh oh!

B-Step62 Dec 12, 2025

Uh oh!

serena-ruan Dec 12, 2025

Uh oh!

serena-ruan Dec 12, 2025

Uh oh!

B-Step62 Dec 12, 2025

Uh oh!

B-Step62 Dec 12, 2025

Uh oh!

serena-ruan Dec 12, 2025

Uh oh!

daniellok-db left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

B-Step62 commented Dec 8, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Install mlflow from this PR

Related Issues

What changes are proposed in this pull request?

How is this PR tested?

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Should this PR be included in the next patch release?

Uh oh!

github-actions bot commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daniellok-db left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

B-Step62 commented Dec 8, 2025 •

edited by github-actions bot

Loading

github-actions bot commented Dec 8, 2025 •

edited

Loading