[ML] Inference endpoints UI serverless: Enables adaptive allocations and allow user to set max allocations by alvarezmelissa87 · Pull Request #222726 · elastic/kibana

alvarezmelissa87 · 2025-06-05T00:05:02Z

Summary

Related issue: #221827

The changes in this PR for now will only apply in serverless.

This PR adds the following changes in a serverless environment:

removes the allocations/threads input fields from the inference endpoints UI creation and replaces it with an input for max allocations
adds informative text for the user when adaptive allocations will be enabled
always sets adaptive allocations to be enabled and min_allocations to 0

Entry points tested:

Inference endpoints list page > Add endpoint button
Playground > Connect to an LLM button
Connectors list page > Create connector button
AI Assistant > Set up GenAI Connector button
Index management > create index with mapping > add semantic text field

TASKS

- [ ] implement helper class to calculate appropriate value for num_threads based on max allocations specified by the user. This will be done keeping in mind that it will be optimized for search with high resource use.

ML nodes will set a default number of threads in serverless trained model APIs - this will require a backend change (I will link PR here when available)
- Until that change is made, num_allocations will be defaulted to 1 as the endpoint currently requires that parameter
minimum allocations will always be 0
Add serverless check in AI Connector to ensure behavior is the same

TO NOTE

The field overrides added are a temporary solution until the endpoint returning the service's configurable fields can be updated.

As the code is shared with the AI Connector - this behavior will also apply for Elasticsearch service when on serverless.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

…rles

…apping'

Samiul-TheSoccerFan

Looks good so far! Just wondering, are any changes needed in the Index Management package as well? Since it's also possible to create inference endpoints from there, it might be worth validating whether any updates are required on that side too.

...es/shared/kbn-inference-endpoint-ui-common/src/components/configuration/helptext_callout.tsx

alvarezmelissa87 · 2025-06-26T00:13:11Z

...gins/shared/triggers_actions_ui/public/application/sections/action_connector_form/helpers.ts

+const MIN_ALLOCATIONS = 0;
+const DEFAULT_NUM_THREADS = 1;
+
+export const getInferenceApiParams = (data: any, enforceAdaptiveAllocations: boolean) => {


Note on this function - ideally we should be doing this in the form serializer but because right now we need enforceAdaptiveAllocations (which isn't available outside of the component) we need to keep this as an external function.
Once this change is included in all environments and we no longer need that flag - this will be moved to the serializer. cc @jcger 🙏

jcger · 2025-06-26T10:43:24Z

...red/triggers_actions_ui/public/application/sections/action_connector_form/connector_form.tsx


 // TODO: Remove when https://github.com/elastic/kibana/issues/133107 is resolved
 const formDeserializer = (data: ConnectorFormSchema): ConnectorFormSchema => {
+  if (


I couldn't find a better alternative. We have to hardcode the serializer/deserializer this way. We'll open an issue to improve our framework

@jcger - thank you so much for your feedback 🙏
I updated with all suggested changes here 1e4dd5a

renamed to isServerless in all areas for the connector

stored the value in context

moved data manipulation to serializer/deserializer

jcger · 2025-06-26T10:49:41Z

...tions_ui/public/application/sections/action_connector_form/create_connector_flyout/index.tsx

       */
-
-      const { actionTypeId, name, config, secrets } = data;
+      const connectorData = getInferenceApiParams(data, !!enforceAdaptiveAllocations) ?? data;


I think it would be better to add the condition here to check that it's only called when the connector type is the inference connector.

This data manipulation now lives in the form serializer/deserializer so this helper function is no longer needed.
Changes made here 1e4dd5a

jcger · 2025-06-26T10:56:24Z

src/platform/packages/shared/kbn-alerts-ui-shared/src/common/types/action_types.ts

 export interface ActionConnectorFieldsProps {
  readOnly: boolean;
  isEdit: boolean;
+  enforceAdaptiveAllocations?: boolean;


It's too specific for the inference connector. For now, it's using the value of isServerless, let's call it that instead. If the requirements for determining when the inference connector should enforceAdapativeAllocations change, we can adapt. For now, let's go for the mininum required changes for the current needs, and keep it as easy as possible. I'll add an extra comment for the lazy loading components that don't set the context the same way we do in the rest of the plugin

jcger · 2025-06-26T15:52:25Z

x-pack/platform/plugins/shared/stack_connectors/public/connector_types/inference/connector.tsx


 const InferenceAPIConnectorFields: React.FunctionComponent<ActionConnectorFieldsProps> = ({
  isEdit,
+  enforceAdaptiveAllocations,


Let's use isServerless via the Kibana context instead.
The component should render like this:

<InferenceServiceFormFields http={http} isEdit={isEdit} enforceAdaptiveAllocations={isServerless} toasts={toasts} />

Updated in 1e4dd5a

jcger · 2025-06-26T15:53:21Z

x-pack/platform/plugins/shared/triggers_actions_ui/public/plugin.ts

          actionTypeRegistry,
          ruleTypeRegistry,
          share: pluginsStart.share,
+          enforceAdaptiveAllocations: !!pluginsStart.serverless,


lets call this isServerless. Same for the rest

Updated in 1e4dd5a

… remove deprecated plugin deps

jcger

I'd recommend testing that the field is shown when it should be, ensuring we don't break it with a change to the isServerless feature. Approving because that test isn't on our side, and because it could be done in a future PR

jcger · 2025-06-27T07:57:02Z

...ggers_actions_ui/public/application/sections/action_connector_form/connector_form_fields.tsx

 interface ConnectorFormFieldsProps {
  actionTypeModel: ActionTypeModel | null;
  isEdit: boolean;
+  enforceAdaptiveAllocations?: boolean;


this can be removed now, I can't see it being used

Good catch! Removed in 670417b

jcger · 2025-06-27T08:00:06Z

...red/triggers_actions_ui/public/application/sections/action_connector_form/connector_form.tsx

  setResetForm?: (value: ResetForm) => void;
 }
+
+interface ProviderConfig {


nit: rename it to something that makes it clear that it's only used/needed by the inference connector, something like InferenceConnectorProviderConfig

Updated in 82c3ef5

cnasikas · 2025-06-27T08:42:19Z

...red/triggers_actions_ui/public/application/sections/action_connector_form/connector_form.tsx

Could you please add tests for the new logic?

alvarezmelissa87 · 2025-06-27T15:56:52Z

@elasticmachine merge upstream

…elds

elasticmachine · 2025-06-27T17:08:32Z

⏳ Build in-progress

Buildkite Build
Commit: df866a8
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-222726-df866a8d27c8
Elasticsearch Serverless Deployment

History

💔 Build #313295 failed 1e4dd5a
💛 Build #312875 was flaky a47f4c4
💛 Build #312679 was flaky 2c9edf2
💛 Build #312361 was flaky 6a18001
💔 Build #312298 failed f19094a

cc @alvarezmelissa87

alvarezmelissa87 · 2025-06-27T21:11:35Z

Created a follow up issue #225700 for adding tests.

kibanamachine · 2025-07-01T21:48:39Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 222726 locally
cc: @alvarezmelissa87

…UI (#249098) ### Summary Adds tests for serverless adaptive allocations feature (#222726). ### Run Tests ``` yarn test:jest --no-collectCoverage x-pack/platform/packages/shared/kbn-inference-endpoint-ui-common/src/components/inference_service_form_fields.test.tsx yarn test:jest --no-collectCoverage x-pack/platform/packages/shared/kbn-inference-endpoint-ui-common/src/components/inference_flyout_wrapper.test.tsx ``` ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [x] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

…UI (elastic#249098) ### Summary Adds tests for serverless adaptive allocations feature (elastic#222726). ### Run Tests ``` yarn test:jest --no-collectCoverage x-pack/platform/packages/shared/kbn-inference-endpoint-ui-common/src/components/inference_service_form_fields.test.tsx yarn test:jest --no-collectCoverage x-pack/platform/packages/shared/kbn-inference-endpoint-ui-common/src/components/inference_flyout_wrapper.test.tsx ``` ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [x] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

alvarezmelissa87 self-assigned this Jun 5, 2025

alvarezmelissa87 requested a review from darnautov June 5, 2025 00:05

alvarezmelissa87 added the Feature:Inference UI ML Inference endpoints UI and AI connector label Jun 5, 2025

This comment was marked as resolved.

Sign in to view

alvarezmelissa87 added v9.1.0 v8.19.0 labels Jun 6, 2025

alvarezmelissa87 added 4 commits June 10, 2025 09:15

add ability to hide internal specified fields

d143501

ensure fields are in config but hidden in UI

ee4c273

add helptext callout for max allocations

0971e5b

always enable adaptive allocations for elasticsearch service on serve…

757c886

…rles

alvarezmelissa87 force-pushed the ml-inference-endpoints-remove-allocations-fields branch from 2d51451 to 4254225 Compare June 10, 2025 15:15

alvarezmelissa87 requested a review from Samiul-TheSoccerFan June 10, 2025 15:16

ensure AiConnector shows changes for es service on serverless

33cb73e

alvarezmelissa87 force-pushed the ml-inference-endpoints-remove-allocations-fields branch from 4254225 to 33cb73e Compare June 10, 2025 16:02

alvarezmelissa87 changed the title ~~[WIP][ML] Inference endpoints UI serverless: enable adaptive allocations and allow user to set max allocations~~ [ML] Inference endpoints UI serverless: enable adaptive allocations and allow user to set max allocations Jun 10, 2025

alvarezmelissa87 marked this pull request as ready for review June 10, 2025 16:03

alvarezmelissa87 requested review from a team as code owners June 10, 2025 16:03

[CI] Auto-commit changed files from 'node scripts/styled_components_m…

12d2156

…apping'

alvarezmelissa87 mentioned this pull request Jun 10, 2025

[ML] Trained models: Warn users of the implication of a min_allocations=0 configuration #218631

Open

alvarezmelissa87 added backport:version Backport to applied version labels release_note:fix release_note:enhancement and removed release_note:fix labels Jun 10, 2025

Samiul-TheSoccerFan added the ci:project-deploy-elasticsearch Create an Elasticsearch Serverless project label Jun 10, 2025

This comment was marked as resolved.

Sign in to view

Samiul-TheSoccerFan reviewed Jun 10, 2025

View reviewed changes

...es/shared/kbn-inference-endpoint-ui-common/src/components/configuration/helptext_callout.tsx Outdated Show resolved Hide resolved

alvarezmelissa87 commented Jun 26, 2025

View reviewed changes

jcger reviewed Jun 26, 2025

View reviewed changes

change to isServerless. data manipulation to serializer/deserializer.…

1e4dd5a

… remove deprecated plugin deps

jcger approved these changes Jun 27, 2025

View reviewed changes

cnasikas reviewed Jun 27, 2025

View reviewed changes

fix types

82c3ef5

alvarezmelissa87 added v9.2.0 and removed v9.1.0 v8.19.0 labels Jun 27, 2025

remove unused prop

670417b

Merge branch 'main' into ml-inference-endpoints-remove-allocations-fi…

df866a8

…elds

alvarezmelissa87 mentioned this pull request Jun 27, 2025

[ML] Inference endpoints UI/ Inference Connector serverless adaptive allocations: add tests #225700

Closed

alvarezmelissa87 merged commit 536ddcc into elastic:main Jun 27, 2025
10 checks passed

alvarezmelissa87 deleted the ml-inference-endpoints-remove-allocations-fields branch June 27, 2025 21:12

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Jul 1, 2025

peteharverson added backport:skip This PR does not require backporting and removed backport missing Added to PRs automatically when the are determined to be missing a backport. backport:version Backport to applied version labels labels Jul 2, 2025

joana-cps mentioned this pull request Jul 4, 2025

[ML][AI Connector] AI Connector flyout sections and field improvements #226580

Closed

4 tasks

peteharverson changed the title ~~[ML] Inference endpoints UI serverless: enable adaptive allocations and allow user to set max allocations~~ [ML] Inference endpoints UI serverless: Enables adaptive allocations and allow user to set max allocations Sep 24, 2025

saikatsarkar056 mentioned this pull request Jan 16, 2026

Add tests for serverless adaptive allocations in inference endpoints UI #249098

Merged

10 tasks

This was referenced Feb 2, 2026

[9.1] Fix AI Connector form fields resetting to default value when cleared by user (#251095) #251290

Merged

[8.19] Fix AI Connector form fields resetting to default value when cleared by user (#251095) #251293

Merged

Conversation

alvarezmelissa87 commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

TASKS

TO NOTE

Checklist

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

Samiul-TheSoccerFan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jcger left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alvarezmelissa87 commented Jun 27, 2025

Uh oh!

elasticmachine commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⏳ Build in-progress

History

Uh oh!

alvarezmelissa87 commented Jun 27, 2025

Uh oh!

Uh oh!

kibanamachine commented Jul 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

16 participants

alvarezmelissa87 commented Jun 5, 2025 •

edited

Loading

elasticmachine commented Jun 27, 2025 •

edited

Loading