Skip to content

Fix 403 Forbidden for artifact list via query param when default_permission=NO_PERMISSIONS#21220

Merged
WeichenXu123 merged 7 commits intomasterfrom
copilot/fix-403-forbidden-load-model
Mar 4, 2026
Merged

Fix 403 Forbidden for artifact list via query param when default_permission=NO_PERMISSIONS#21220
WeichenXu123 merged 7 commits intomasterfrom
copilot/fix-403-forbidden-load-model

Conversation

Copy link
Contributor

Copilot AI commented Feb 28, 2026

Closes #21201

When default_permission = NO_PERMISSIONS is configured, users with explicit experiment permissions get 403 when calling load_model() on a logged model in MLflow 3.10. The artifact download triggers a list request using a query parameter (?path=…) rather than a URL path parameter, causing permission extraction to fail and fall through to the restrictive default.

Root Cause

_get_experiment_id_from_view_args() only read request.view_args["artifact_path"] (the path parameter used by download/upload/delete endpoints). The list artifacts endpoint uses GET /mlflow-artifacts/artifacts?path=<experiment_id>/models/<model_id>/artifacts — a query parameter — so the experiment ID was never extracted, and permission checks silently fell back to default_permission.

Fix

  • mlflow/server/auth/__init__.py: Check request.args.get("path") as a fallback when the URL path parameter is absent:
# Before
if artifact_path := request.view_args.get("artifact_path"):
    ...

# After — also covers list/delete endpoints that pass path as a query param
if artifact_path := (request.view_args.get("artifact_path") or request.args.get("path")):
    ...
  • tests/server/auth/fixtures/no_permission_auth.ini: New auth config fixture with default_permission = NO_PERMISSIONS to reproduce the bug scenario.
  • tests/server/auth/test_auth.py: Regression test test_proxy_artifact_list_query_param_uses_experiment_permission — verifies an authorized user (MANAGE) is not blocked on artifact list via query param, and an unauthorized user still gets 403.

How is this PR tested?

  • Existing unit/integration tests
  • New unit/integration tests

Does this PR require documentation update?

  • No. You can skip the rest of this section.

Does this PR require updating the MLflow Skills repository?

  • No. You can skip the rest of this section.

Release Notes

Is this a user-facing change?

  • Yes. Give a description of this change to be included in the release notes for MLflow users.

Fixed a regression in the basic-auth plugin where loading a logged model artifact (load_model()) returned 403 Forbidden for users with explicit experiment permissions when the server was configured with default_permission = NO_PERMISSIONS.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/tracking: Tracking Service, tracking client APIs, autologging

How should the PR be classified in the release notes? Choose one:

  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes

Should this PR be included in the next patch release?

  • Yes (this PR will be cherry-picked and included in the next patch release)
  • No (this PR will be included in the next minor release)

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.openai.com
    • Triggering command: /home/REDACTED/work/_temp/setup-uv-cache/builds-v0/.tmpDhavJd/bin/python /home/REDACTED/work/_temp/setup-uv-cache/builds-v0/.tmpDhavJd/bin/python -m uvicorn mlflow.server.auth:create_app --host 127.0.0.1 --port 33221 /dev/null -lFoundation 0/x64/bin/git uname -p (dns block)
    • Triggering command: /home/REDACTED/work/_temp/setup-uv-cache/builds-v0/.tmpDhavJd/bin/python /home/REDACTED/work/_temp/setup-uv-cache/builds-v0/.tmpDhavJd/bin/python -m uvicorn mlflow.server.auth:create_app --host 127.0.0.1 --port 49913 d -n 10 (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>[BUG] 403 Forbiden when calling load_model() MLFlow 3.10</issue_title>
<issue_description>### Issues Policy acknowledgement

  • I have read and agree to submit bug reports in accordance with the issues policy

Where did you encounter this bug?

Local machine

MLflow version

  • Client: 3.10
  • Tracking server: 3.10

System information

  • OS Platform and Distribution: Debian 13
  • Python version: 3.12

Describe the problem

The same permissions that work on MLFlow 3.9 doesnt work anymore on MLFlow 3.10 when trying to call load_model().

The only solution is to grant the user admin right.

Tracking information

System information: Linux mlflow/mlflow#1 SMP PREEMPT_DYNAMIC Debian 6.12.73-1 (2026-02-17)
Python version: 3.12.12
MLflow version: 3.10.0
MLflow module location: /home/llr/test/.venv/lib/python3.12/site-packages/mlflow/__init__.py
Tracking URI: REDACTED
Registry URI: REDACTED
MLflow environment variables: 
  MLFLOW_TRACKING_PASSWORD: REDACTED
  MLFLOW_TRACKING_URI: REDACTED
  MLFLOW_TRACKING_USERNAME: lomoulin
MLflow dependencies: 
  Flask: 3.1.3
  Flask-CORS: 6.0.2
  Flask-WTF: 1.2.2
  alembic: 1.18.4
  click: 8.3.1
  cryptography: 46.0.5
  docker: 7.1.0
  fastapi: 0.115.14
  graphene: 3.4.3
  gunicorn: 25.1.0
  huey: 2.6.0
  matplotlib: 3.10.8
  mlflow-skinny: 3.10.0
  mlflow-tracing: 3.10.0
  numpy: 2.4.2
  pandas: 2.3.3
  pyarrow: 23.0.1
  scikit-learn: 1.8.0
  scipy: 1.17.1
  skops: 0.13.0
  sqlalchemy: 2.0.47
  uvicorn: 0.41.0

Code to reproduce issue

import os

import mlflow
from mlflow.server import get_app_client

client = get_app_client(app_name="basic-auth", tracking_uri=os.environ["MLFLOW_TRACKING_URI"])

user = client.get_user(os.environ["MLFLOW_TRACKING_USERNAME"])
experiments_ids = [x.experiment_id for x in user.experiment_permissions if x.permission == "MANAGE"]


def get_model_by_run(run_id):
    run = mlflow.get_run(run_id)
    if run.info.experiment_id not in experiments_ids:
        raise Exception("Permission not found")  # Dont happen
    model_id = run.outputs.model_outputs[0].model_id
    model = mlflow.get_logged_model(model_id)
    model = mlflow.sklearn.load_model(model.model_uri)  # Exception raised here
    return model


get_model_by_run("3b618db3eec1460d9e08660f03b2a499")

Stack trace

Traceback (most recent call last):
  File "/home/llr/test/.venv/lib/python3.12/site-packages/mlflow/utils/request_utils.py", line 63, in augmented_raise_for_status
    response.raise_for_status()
  File "/home/llr/test/.venv/lib/python3.12/site-packages/requests/models.py", line 1026, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://REDACTED/api/2.0/mlflow-artifacts/artifacts?path=102%2Fmodels%2Fm-44c79763c30f4e68b454434e2b9ba263%2Fartifacts

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/llr/test/test.py", line 28, in <module>
    get_model_by_run("3b618db3eec1460d9e08660f03b2a499")
  File "/home/llr/test/test.py", line 24, in get_model_by_run
    model = mlflow.sklearn.load_model(model.model_uri)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/llr/test/.venv/lib/python3.12/site-packages/mlflow/sklearn/__init__.py", line 761, in load_model
    local_model_path = _download_artifact_from_uri(artifact_uri=model_uri, output_path=dst_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/llr/test/.venv/lib/python3.12/site-packages/mlflow/tracking/artifact_utils.py", line 123, in _download_artifact_from_uri
    return repo.download_artifacts(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/llr/test/.venv/lib/python3.12/site-packages/mlflow/store/artifact/models_artifact_repo.py", line 249, in download_artifacts
    model_path = self.repo.download_artifacts(artifact_path, dst_path)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/llr/test/.venv/lib/python3.12/site-packages/mlflow/store/artifact/artifact_repo.py", line 294, in download_artifacts
    root_listing = self.list_artifacts(artifact_path)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/llr/test/.venv/lib/python3.12/site-packages/mlflow/store/artifact/http_artifact_repo.py", line 89, in list_artifacts
    augmented_raise_for_status(resp)
  File "/home/llr/test/.venv/lib/python3.12/site-packages/mlflow/utils/request_utils.py", line 66, in augmented_raise_for_status
    raise HTTPError(
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://REDACTE...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes mlflow/mlflow#21201

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

Co-authored-by: WeichenXu123 <19235986+WeichenXu123@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix 403 Forbidden error when calling load_model with MLFlow 3.10 Fix 403 Forbidden for artifact list via query param when default_permission=NO_PERMISSIONS Feb 28, 2026
@WeichenXu123 WeichenXu123 marked this pull request as ready for review March 2, 2026 10:07
Copilot AI review requested due to automatic review settings March 2, 2026 10:07
@github-actions github-actions bot added size/S Small PR (10-49 LoC) area/tracking Tracking service, tracking client APIs, autologging rn/bug-fix Mention under Bug Fixes in Changelogs. labels Mar 2, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Mar 2, 2026

Documentation preview for f450d4d is available at:

More info
  • Ignore this comment if this PR does not change the documentation.
  • The preview is updated when a new commit is pushed to this PR.
  • This comment was created by this workflow run.
  • The documentation was built by this workflow run.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a permission-check regression in the basic-auth plugin when listing artifacts via the mlflow-artifacts proxy endpoint using a ?path=... query parameter, particularly when default_permission=NO_PERMISSIONS.

Changes:

  • Extend experiment-id extraction for artifact-proxy authorization to fall back to request.args["path"] when no artifact_path URL parameter is present.
  • Add an auth config fixture with default_permission = NO_PERMISSIONS to reproduce the restrictive-default scenario.
  • Add a regression test ensuring artifact listing via query param honors explicit experiment permissions.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
mlflow/server/auth/__init__.py Updates experiment-id extraction logic for artifact-proxy authorization to support query-param paths.
tests/server/auth/fixtures/no_permission_auth.ini Adds a test auth configuration with default_permission=NO_PERMISSIONS.
tests/server/auth/test_auth.py Adds a regression test covering artifact list via ?path=... under restrictive defaults.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: WeichenXu <weichen.xu@databricks.com>
WeichenXu123 and others added 4 commits March 2, 2026 19:46
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: WeichenXu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
This reverts commit 8ab6c65.

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
params={"path": f"{experiment_id}/models/m-abc123/artifacts"},
auth=(username1, password1),
)
assert response.status_code != 403
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test is only for auth layer, the backend flask server does not fully set up artifact storage configure, so response.status_code == 200 will fail

Copy link
Collaborator

@WeichenXu123 WeichenXu123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@WeichenXu123 WeichenXu123 added this pull request to the merge queue Mar 2, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 2, 2026
@WeichenXu123 WeichenXu123 added this pull request to the merge queue Mar 4, 2026
Merged via the queue into master with commit dc8ef3c Mar 4, 2026
54 of 56 checks passed
@WeichenXu123 WeichenXu123 deleted the copilot/fix-403-forbidden-load-model branch March 4, 2026 01:26
daniellok-db pushed a commit to daniellok-db/mlflow that referenced this pull request Mar 5, 2026
…mission=NO_PERMISSIONS` (mlflow#21220)

Signed-off-by: WeichenXu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WeichenXu123 <19235986+WeichenXu123@users.noreply.github.com>
Co-authored-by: WeichenXu <weichen.xu@databricks.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
daniellok-db pushed a commit to daniellok-db/mlflow that referenced this pull request Mar 5, 2026
…mission=NO_PERMISSIONS` (mlflow#21220)

Signed-off-by: WeichenXu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WeichenXu123 <19235986+WeichenXu123@users.noreply.github.com>
Co-authored-by: WeichenXu <weichen.xu@databricks.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
daniellok-db pushed a commit that referenced this pull request Mar 5, 2026
…mission=NO_PERMISSIONS` (#21220)

Signed-off-by: WeichenXu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WeichenXu123 <19235986+WeichenXu123@users.noreply.github.com>
Co-authored-by: WeichenXu <weichen.xu@databricks.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/tracking Tracking service, tracking client APIs, autologging rn/bug-fix Mention under Bug Fixes in Changelogs. size/S Small PR (10-49 LoC) v3.10.1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] 403 Forbiden when calling load_model() MLFlow 3.10

3 participants