Skip to content

Rebuild SearchTraces V2 request body on ENDPOINT_NOT_FOUND fallback#20963

Merged
harupy merged 2 commits intomlflow:masterfrom
brendanmaguire:fix-search-traces-v2
Feb 19, 2026
Merged

Rebuild SearchTraces V2 request body on ENDPOINT_NOT_FOUND fallback#20963
harupy merged 2 commits intomlflow:masterfrom
brendanmaguire:fix-search-traces-v2

Conversation

@brendanmaguire
Copy link
Contributor

@brendanmaguire brendanmaguire commented Feb 18, 2026 •
edited
Loading

Related Issues/PRs

No related issues/PRs

What changes are proposed in this pull request?

When a tracking server returns ENDPOINT_NOT_FOUND for the V3 search traces API, the client fell back to V2 but reused the V3 request body (containing proto-serialised locations) instead of constructing a fresh V2 body with experiment_ids, causing that call to fail with INVALID_PARAMETER_VALUE against older servers.

This change creates a new v2 body for the fallback request.

How is this PR tested?

  • Existing unit/integration tests
  • New unit/integration tests
  • Manual tests

Manual testing was performed by querying a local MLFlow server running 3.0.0 with a 3.8.1 client.

Does this PR require documentation update?

  • No. You can skip the rest of this section.
  • Yes. I've updated:
    • Examples
    • API references
    • Instructions

Does this PR require updating the MLflow Skills repository?

  • No. You can skip the rest of this section.
  • Yes. Please link the corresponding PR or explain how you plan to update it.

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/tracking: Tracking Service, tracking client APIs, autologging
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/scoring: MLflow Model server, model deployment tools, Spark UDFs
  • area/evaluation: MLflow model evaluation features, evaluation metrics, and evaluation workflows
  • area/gateway: MLflow AI Gateway client APIs, server, and third-party integrations
  • area/prompts: MLflow prompt engineering features, prompt templates, and prompt management
  • area/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionality
  • area/projects: MLproject format, project running backends
  • area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages

How should the PR be classified in the release notes? Choose one:

  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes should be selected for bug fixes, documentation updates, and other small changes. No should be selected for new features and larger changes. If you're unsure about the release classification of this PR, leave this unchecked to let the maintainers decide.

What is a minor/patch release?
  • Minor release: a release that increments the second part of the version number (e.g., 1.2.0 -> 1.3.0).
    Bug fixes, doc updates and new features usually go into minor releases.
  • Patch release: a release that increments the third part of the version number (e.g., 1.2.0 -> 1.2.1).
    Bug fixes and doc updates usually go into patch releases.
  • Yes (this PR will be cherry-picked and included in the next patch release)
  • No (this PR will be included in the next minor release)

@github-actions github-actions bot added community Community/external contribution size/M labels Feb 18, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Feb 18, 2026

🛠 DevTools 🛠

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/20963/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/20963/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/20963/merge

@github-actions
Copy link
Contributor

github-actions bot commented Feb 18, 2026

@brendanmaguire Thank you for the contribution! Could you fix the following issue(s)?

⚠ DCO check

The DCO check failed. Please sign off your commit(s) by following the instructions here. See https://github.com/mlflow/mlflow/blob/master/CONTRIBUTING.md#sign-your-work for more details.

@github-actions github-actions bot added v3.10.0 area/tracking Tracking service, tracking client APIs, autologging rn/bug-fix Mention under Bug Fixes in Changelogs. labels Feb 18, 2026
@harupy
Copy link
Member

harupy commented Feb 19, 2026

Thanks! Can you fix the failed CI checks?

@harupy harupy requested a review from Copilot February 19, 2026 00:50
Copilot AI reviewed Feb 19, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a bug in the MLflow tracking client where falling back from the V3 to V2 SearchTraces API would fail due to sending an incorrectly formatted request body. When a tracking server returns ENDPOINT_NOT_FOUND for the V3 search traces API, the client now properly constructs a fresh V2 request body with experiment_ids instead of reusing the V3 body with proto-serialized locations.

Changes:

  • Fixed SearchTraces V3-to-V2 fallback by creating a new V2 request body with experiment_ids instead of reusing the V3 body
  • Added comprehensive test coverage for the fallback scenario

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
mlflow/store/tracking/rest_store.py Creates a new SearchTraces (V2) request with experiment_ids parameter when falling back from V3 API
tests/store/tracking/test_rest_store.py Adds test to verify V3-to-V2 fallback creates correct request body with experiment_ids

@github-actions
Copy link
Contributor

github-actions bot commented Feb 19, 2026 •
edited
Loading

Documentation preview for 185ecbc is available at:

More info
  • Ignore this comment if this PR does not change the documentation.
  • The preview is updated when a new commit is pushed to this PR.
  • This comment was created by this workflow run.
  • The documentation was built by this workflow run.

@harupy harupy changed the title Fix: rebuild SearchTraces V2 request body on ENDPOINT_NOT_FOUND fallback Rebuild SearchTraces V2 request body on ENDPOINT_NOT_FOUND fallback Feb 19, 2026
@brendanmaguire
Fix: rebuild SearchTraces V2 request body on ENDPOINT_NOT_FOUND fallback
f64769c 
When a tracking server returns ENDPOINT_NOT_FOUND for the V3 search
traces API, the client fell back to V2 but reused the V3 request body
(containing proto-serialised `locations`) instead of constructing a
fresh V2 body with `experiment_ids`, causing that call to fail with
INVALID_PARAMETER_VALUE against older servers.

Signed-off-by: Brendan Maguire <1093243+brendanmaguire@users.noreply.github.com>
@brendanmaguire
Copy link
Contributor Author

brendanmaguire commented Feb 19, 2026

Thanks! Can you fix the failed CI checks?

@harupy I think all are green now

harupy
harupy approved these changes Feb 19, 2026
View reviewed changes
Copy link
Member

@harupy harupy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@harupy harupy enabled auto-merge February 19, 2026 06:10
@harupy harupy added this pull request to the merge queue Feb 19, 2026
Merged via the queue into mlflow:master with commit 5186c9a Feb 19, 2026
51 of 53 checks passed
daniellok-db pushed a commit to daniellok-db/mlflow that referenced this pull request Feb 20, 2026
@brendanmaguire @daniellok-db
Rebuild SearchTraces V2 request body on ENDPOINT_NOT_FOUND fallba…
0426f29 
…ck (mlflow#20963)

Signed-off-by: Brendan Maguire <1093243+brendanmaguire@users.noreply.github.com>
daniellok-db pushed a commit that referenced this pull request Feb 20, 2026
@brendanmaguire @daniellok-db
Rebuild SearchTraces V2 request body on ENDPOINT_NOT_FOUND fallba…
7f2039d 
…ck (#20963)

Signed-off-by: Brendan Maguire <1093243+brendanmaguire@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@harupy harupy harupy approved these changes

Assignees

@harupy harupy

Labels

area/tracking Tracking service, tracking client APIs, autologging community Community/external contribution rn/bug-fix Mention under Bug Fixes in Changelogs. size/M v3.10.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@brendanmaguire @harupy