[Endpoints] [7/x] Add rest store implementation by BenWilson2 · Pull Request #19008 · mlflow/mlflow

BenWilson2 · 2025-11-24T17:58:01Z

🥞 Stacked PR

Use this link to review incremental changes.

stack/endpoints/rest-2 [Files changed]
- stack/endpoints/cache [Files changed]
  - stack/endpoints/litellm [Files changed]
    - stack/endpoints/ui-apis [Files changed]
      - stack/endpoints/ui-create-endpoint [Files changed]
        
        stack/endpoints/ui-tabs [Files changed]
        
        stack/endpoints/key-management [Files changed]
        
        stack/endpoints/passphrase [Files changed]

Related Issues/PRs

#xxx

What changes are proposed in this pull request?

Adds the rest store implementation and e2e backend tests

How is this PR tested?

Existing unit/integration tests
New unit/integration tests
Manual tests

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

How should the PR be classified in the release notes? Choose one:

rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes should be selected for bug fixes, documentation updates, and other small changes. No should be selected for new features and larger changes. If you're unsure about the release classification of this PR, leave this unchecked to let the maintainers decide.

What is a minor/patch release?

Minor release: a release that increments the second part of the version number (e.g., 1.2.0 -> 1.3.0).
Bug fixes, doc updates and new features usually go into minor releases.
Patch release: a release that increments the third part of the version number (e.g., 1.2.0 -> 1.2.1).
Bug fixes and doc updates usually go into patch releases.

Yes (this PR will be cherry-picked and included in the next patch release)
No (this PR will be included in the next minor release)

github-actions · 2025-12-02T22:57:04Z

Documentation preview for 03c364d is available at:

https://pr-19008--mlflow-docs-preview.netlify.app/docs/latest/

More info

Ignore this comment if this PR does not change the documentation.
The preview is updated when a new commit is pushed to this PR.
This comment was created by this workflow run.
The documentation was built by this workflow run.

TomeHirata · 2025-12-05T04:49:44Z

mlflow/store/tracking/rest_store.py



-class RestStore(AbstractStore):
+class RestStore(RestGatewayStoreMixin, AbstractStore):


Since AbstractStore inherits GatewayStoreMixin, RestGatewayStoreMixin needs to be passed first, correct? Depending on the order of parent classes seems risky, can we add a comment for the future reminder?

TomeHirata · 2025-12-05T04:52:04Z

mlflow/store/tracking/gateway/rest_mixin.py

+from mlflow.utils.proto_json_utils import message_to_json
+
+
+class RestGatewayStoreMixin:


Should we implement get_resource_endpoint_configs as well?

Ah, we can't create any REST endpoints for that API (that would allow any user to get any decrypted key). In order to make sure that we don't ever accidentally add that as a feature, I removed this API from the standard store mixins and put it in store/tracking/gateway with some NB notes and guidance about how dangerous use of this API outside of the tracking server is.

Understood, where did you make the change to remove get_resource_endpoint_configs from the general store mixin?

Ah it was for the feedback in sql-store that @B-Step62 mentioned... it's the right call to separate this out for future dev security.

TomeHirata · 2025-12-05T04:52:57Z

mlflow/server/handlers.py

+    # Endpoint Bindings APIs
+    CreateGatewayEndpointBinding: _create_endpoint_binding,
+    DeleteGatewayEndpointBinding: _delete_endpoint_binding,
+    ListGatewayEndpointBindings: _list_endpoint_bindings,


Shall we add proto for get_resource_endpoint_configs?

^ same answer as preceding question

TomeHirata · 2025-12-08T05:06:24Z

mlflow/entities/gateway_endpoint.py

+        proto.resource_type = (
+            self.resource_type.value
+            if isinstance(self.resource_type, GatewayResourceType)
+            else self.resource_type


Do we need else branch? We only expect GatewayResourceType in this field.

TomeHirata

LGTM

B-Step62 · 2025-12-08T08:42:15Z

mlflow/server/handlers.py

+        CreateGatewaySecret(),
+        schema={
+            "secret_name": [_assert_required, _assert_string],
+            "secret_value": [_assert_required, _assert_string],


These assertion logic seems to print the value directly in the error message. Even if it does not, these are pretty generic utils so we can accidentally add display logic. Should we have a special validation logic to avoid the risk of such leakage?

Also worth checking the other general logic like message_to_json does not expose unencrypted values to the rest error or the server logs.

Good catch - we need a separate validation handler that is only used for the APIs interfacing with plaintext API Keys to guard against any potential future log leakage. Updated

B-Step62 · 2025-12-08T08:44:16Z

mlflow/server/handlers.py

+        UpdateGatewaySecret(),
+        schema={
+            "secret_id": [_assert_required, _assert_string],
+            "secret_value": [_assert_required, _assert_string],


Is secret value required? Shall we allow users to update auth config only?

Good idea - this definitely makes the 4 providers that have complex configs a bit more flexible. We can handle single-key missing value entries guarded in the UI (do not allow editing a key that only has API KEY to set an empty string).

B-Step62 · 2025-12-08T08:48:26Z

mlflow/store/tracking/gateway/rest_mixin.py

+        self,
+        endpoint_id: str,
+        model_definition_id: str,
+        weight: int = 1,


Shouldn't this be float?

yep - missed updating that!

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

BenWilson2 changed the title ~~rest store implementation~~ [Endpoints] [7/x] Add rest store implementation Nov 24, 2025

BenWilson2 marked this pull request as ready for review November 24, 2025 22:01

BenWilson2 force-pushed the stack/endpoints/rest-2 branch from da8167e to e6b9861 Compare November 24, 2025 22:19

github-actions bot added area/tracking Tracking service, tracking client APIs, autologging rn/feature Mention under Features in Changelogs. labels Nov 24, 2025

BenWilson2 force-pushed the stack/endpoints/rest-2 branch 2 times, most recently from eee09fd to 0a0523d Compare November 26, 2025 03:40

This was referenced Nov 26, 2025

[Endpoints] [11/x] Main Gateway page and create endpoint #19042

Closed

[Endpoints] [12/x] Add Left hand nav and management stub #19070

Closed

[Endpoints] [13/x] Add the API Key management pages #19071

Closed

BenWilson2 force-pushed the stack/endpoints/rest-2 branch from 0a0523d to 8738ac2 Compare November 27, 2025 06:31

BenWilson2 mentioned this pull request Nov 27, 2025

[Endpoints] [14/x] Models management UI #19074

Closed

29 tasks

BenWilson2 force-pushed the stack/endpoints/rest-2 branch 2 times, most recently from f57e5af to 4fc4111 Compare December 2, 2025 02:14

BenWilson2 mentioned this pull request Dec 2, 2025

[Endpoints] [14/x] Add encryption availability endpoints #19176

Closed

29 tasks

BenWilson2 force-pushed the stack/endpoints/rest-2 branch 2 times, most recently from d3c8fb5 to e736d1c Compare December 2, 2025 22:49

BenWilson2 force-pushed the stack/endpoints/rest-2 branch 3 times, most recently from 7e7a193 to 6915f33 Compare December 4, 2025 00:27

github-actions bot requested review from TomeHirata, WeichenXu123, kevin-lyn, serena-ruan and xq-yin December 4, 2025 14:49

BenWilson2 force-pushed the stack/endpoints/rest-2 branch 3 times, most recently from da5f51d to 68f01c4 Compare December 5, 2025 04:29

TomeHirata reviewed Dec 5, 2025

View reviewed changes

BenWilson2 force-pushed the stack/endpoints/rest-2 branch 10 times, most recently from dc518e8 to 77da1a1 Compare December 7, 2025 01:57

TomeHirata reviewed Dec 8, 2025

View reviewed changes

TomeHirata approved these changes Dec 8, 2025

View reviewed changes

B-Step62 reviewed Dec 8, 2025

View reviewed changes

rest store implementation

03c364d

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

BenWilson2 force-pushed the stack/endpoints/rest-2 branch from 77da1a1 to 03c364d Compare December 8, 2025 19:34

BenWilson2 added this pull request to the merge queue Dec 8, 2025

Merged via the queue into mlflow:master with commit 5afc02e Dec 8, 2025
68 of 71 checks passed

BenWilson2 deleted the stack/endpoints/rest-2 branch December 8, 2025 21:22



		class RestStore(AbstractStore):
		class RestStore(RestGatewayStoreMixin, AbstractStore):

		from mlflow.utils.proto_json_utils import message_to_json


		class RestGatewayStoreMixin:

Conversation

BenWilson2 commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🥞 Stacked PR

Related Issues/PRs

What changes are proposed in this pull request?

How is this PR tested?

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Should this PR be included in the next patch release?

Uh oh!

github-actions bot commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TomeHirata left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

BenWilson2 commented Nov 24, 2025 •

edited

Loading

github-actions bot commented Dec 2, 2025 •

edited

Loading