feat(security, uiam): make `@kbn\es` to manage UIAM containers when in UIAM mode by azasypkin · Pull Request #243752 · elastic/kibana

azasypkin · 2025-11-21T08:22:00Z

Summary

Make @kbn\es to manage UIAM containers when in Serverless + UIAM mode as it's going to be a default for Serverless at some point.

NOTE: This PR includes some groundwork for initializing UIAM in tests (i.e., passing the --uiam flag from test configs down to @kbn/es APIs). However, it doesn't include tests yet. The reason is that I'd like to start with Scout tests, but it's not yet possible to have a dedicated Scout config where we can toggle UIAM mode. I'm working with @elastic/appex-qa to add the remaining changes to make this happen, but it will likely go in a follow-up. If you feel like we need to move the test-related changes to a separate PR, just let me know.

How to test

First, run Elasticsearch Severless in the UIAM mode:

$ yarn es serverless --projectType oblt --uiam
---
   │ info es01 is running.
   │        Container Name: es01
   │        Container Id:   9bb06af54a4b98c4dd3f75e21e6b1a47e5bad7cf2b8e0d7f18e3ec0e3321cf72
   │
   │        View logs:            docker logs -f es01
   │        Shell access:         docker exec -it es01 /bin/bash
   │
   │ info es02 is running.
   │        Container Name: es02
   │        Container Id:   688e43ce813c68450c70c87f2ccc29738e06e0a45a7ccd4041c86718ac103866
   │
   │        View logs:            docker logs -f es02
   │        Shell access:         docker exec -it es02 /bin/bash
   │
   │ info es03 is running.
   │        Container Name: es03
   │        Container Id:   658fe85658de8d0915e63480dc6b96b1e5e4e092e48314b001965c039eb104f4
   │
   │        View logs:            docker logs -f es03
   │        Shell access:         docker exec -it es03 /bin/bash
...
 info Waiting for "uiam-cosmosdb" container (starting)…
 info Waiting for "uiam" container (starting)…
   │ info The "uiam-cosmosdb" container is up and ready.
   │        Container Name: uiam-cosmosdb
   │        Container Id:   e0ea6a43bfc5379b10cf5af941b6677e0cb8d90e075b4b36ff0c487ff188386a
   │
   │        View logs:            docker logs -f uiam-cosmosdb
   │        Shell access:         docker exec -it uiam-cosmosdb /bin/bash
   │
   │ info The "uiam" container is up and ready.
   │        Container Name: uiam
   │        Container Id:   b4a9b5f56bc2155201fc942e22d1fde3d51453e60b45cf74693b43535edd1cb4
   │
   │        View logs:            docker logs -f uiam
   │        Shell access:         docker exec -it uiam /bin/bash
   │
...
 succ ES cluster is ready
 succ Created role mapping for mock identity provider. You can now login using cloud-saml-kibana realm

Then, run Kibana Serverless in the UIAM mode:

$ yarn start --serverless=oblt --uiam
---
...
Kibana will be configured to support UIAM.
[2025-11-26T09:17:56.417+01:00][INFO ][root] Kibana is starting
...
[2025-11-26T09:19:34.241+01:00][INFO ][status] Kibana is now available

At this point, you should be able to log in into Kibana through a mock IdP provider/select as you'd normally do.

azasypkin · 2025-11-21T08:27:14Z

Hey @elastic/kibana-operations!

The PR is still in draft, but I wanted to check if the approach (incorporating UIAM into @kbn/es directly) makes sense to you (no need to do a full review yet)?

The idea is that UIAM will become a mandatory part of the ECS offering soon (and eventually ECH as well), so logically, making UIAM bootstrapping (running and initializing Cosmos DB Emulator + UIAM service) part of the local ES Serverless bootstrapping makes sense to me (vs. running it as a separate script). This should hopefully also make it easy to use UIAM in our API integration/functional/Scout tests.

But before moving forward with this approach, I want to check if you're fine with this and whether there are any gotchas I should be aware of.

Thanks!

azasypkin · 2025-11-21T08:31:57Z

src/platform/packages/private/kbn-mock-idp-utils/src/constants.ts

 export const MOCK_IDP_UIAM_ORGANIZATION_ID = '1234567890';
 export const MOCK_IDP_UIAM_PROJECT_ID = 'abcde1234567890';

+export const MOCK_IDP_UIAM_SERVICE_INTERNAL_URL = 'http://uiam-cosmosdb:8090';


note: since both ES and UIAM service are running within the same Docker network they can use this internal addressing. Kibana should still rely on localhost/127.0.0.1

azasypkin · 2025-11-21T08:32:44Z

src/platform/packages/shared/kbn-es/src/utils/docker.test.ts

  maybeCreateDockerNetwork,
  maybePullDockerImage,
-  printESImageInfo,
+  printDockerImageInfo,


note: renamed since now it's equally used for ES and UIAM images/containers and there is nothing specific to ES in that function.

azasypkin · 2025-11-21T08:33:54Z

src/platform/packages/shared/kbn-es/src/utils/docker.ts

+/**
+ * Pulls and prints info about the Docker image.
+ */
+async function setupDockerImage({ log, image }: { log: ToolingLog; image: string }) {


note: extracted this from setupDocker to use both of these functions for both ES and UIAM containers (Cosmos DB emulator + UIAM service)

src/platform/packages/shared/kbn-es/src/utils/docker_uiam.ts

azasypkin · 2025-11-21T08:47:45Z

src/platform/packages/shared/kbn-es/src/utils/docker_uiam.ts

+const UIAM_DOCKER_REGISTRY = 'docker.elastic.co';
+const UIAM_DOCKER_REPO = `${UIAM_DOCKER_REGISTRY}/cloud-ci/uiam`;
+// Taken from GitOps version file for UIAM service (dev env, services/uiam/versions.yaml)
+const UIAM_DOCKER_LATEST_VERIFIED_TAG = 'git-fb324ba1e88f';


note: we'd need help from QA/Operations teams to switch to something like :latest-verified, but that's not critical.

as in set up a pipeline to keep verifying new versions and pushing them to our kibana-ci/ folder in the container registry?

Yep, if possible!

let's add that to the tasks in the original issue

Yep, added to the list of tasks in the related meta issue

src/platform/packages/shared/kbn-es/src/utils/docker_uiam.ts

azasypkin · 2025-11-21T09:05:44Z

src/platform/packages/shared/kbn-es/src/utils/docker_uiam.ts

+      `uiam.tokens.jwt.signing.secret=${MOCK_IDP_UIAM_SIGNING_SECRET}`,
+
+      '--health-cmd',
+      'timeout 1 bash -c "</dev/tcp/localhost/8090"',


note: unfortunately, we cannot use curl-based health check here as we do for CosmosDB Emulator since there is no curl in UIAM service OS image, so I'm just checking if port is open.

azasypkin · 2025-11-21T09:07:33Z

src/platform/packages/shared/kbn-es/src/utils/docker_uiam.ts

+
+  const fetchDispatcher = new Agent({ connect: { rejectUnauthorized: false } });
+
+  log.info(chalk.bold(`Initializing Cosmos DB (${COSMOS_DB_ENDPOINT}/${UIAM_COSMOS_DB_NAME})…`));


note: I don't have any health check here since by the time we call this function Cosmos DB emulator container should be healthy (based on the Docker health check). Let's see if it's stable enough.

Ikuni17 · 2025-11-21T09:45:30Z

@azasypkin

But before moving forward with this approach, I want to check if you're fine with this and whether there are any gotchas I should be aware of.

Your approach seems fine to me since this is tightly coupled with Serverless. Splitting into another UIAM package would likely end up with recreating some Docker management functions or a circular dependency on @kbn/es.

If we wanted proper separation I think it would make sense to split all the Docker stuff into a new generic package to wrap Docker and utilize that in both @kbn/es and @kbn/some-new-uiam-pkg. That's a lot of extra effort that probably isn't necessary right now.

azasypkin · 2025-11-21T09:53:03Z

Great, thank you for the feedback @Ikuni17! I'll move forward with this approach and tag you all for review once it's ready.

If we wanted proper separation I think it would make sense to split all the Docker stuff into a new generic package to wrap Docker and utilize that in both @kbn/es and @kbn/some-new-uiam-pkg. That's a lot of extra effort that probably isn't necessary right now.

Yeah, that sounds like a good long-term plan (not for now, but in the future). I know other teams are also juggling with different Docker setups in their scripts and would highly benefit from a common and more generic (and opinionated Kibana-centric) Docker package/tools.

src/platform/packages/shared/kbn-es/src/utils/docker.ts

delanni · 2025-11-21T14:50:11Z

src/platform/packages/shared/kbn-es/src/utils/docker.ts

+      await setupDockerImage({ log, image: uiamImage });
+    }
+
+    uimaNodeNames.push(...(await runUiamContainers(log)));


do these images need to be started before all the others?

Not necessarily (at least today). Even though both ES and Kibana depend on UIAM, we just need to start them before anyone is using Kibana or ES with UIAM credentials.

maybe then this startup can be started in parallel with the other nodes? I am trying to look for traces of how long these containers' startup is taking but I believe they have a 15s maximum timeout per container to start up? so ideally it's all <30s?

maybe then this startup can be started in parallel with the other nodes?

Yeah, I think we can start UIAM containers in parallel with ES containers, or are you asking if we can start both Cosmos DB Emulator container and UIAM service container at the same time too? The former is definitely yes, the latter, likely yes too, but I'd need to double-check.

This will make the code more complex, but, I agree, it's worth it considering that we want to use this for CI and tests.

but I believe they have a 15s maximum timeout per container to start up? so ideally it's all <30s?

That's correct, although I'm testing on my local laptop, we might see a different picture on CI, so yeah, let me see if I can spin these up in parallel. I'm confident that UIAM container is quite fast to bootstrap, but I'm not so confident about Cosmos DB emulator.

Okay, done! I had to change how we expose UIAM ports (initially, we were exposing the UIAM service port from the UIAM Cosmos DB container, so there was a dependency), but now it seems to be working well.

…n UIAM mode

…sDB emulator HTTPS setup

azasypkin · 2025-11-25T16:55:32Z

src/platform/packages/shared/kbn-es/src/utils/docker.test.ts

  waitForSecurityIndex: jest.fn(),
 }));

+jest.mock('./docker_uiam', () => {


note: I test docker_uiam unit separately to not over-complicate base docker tests.

elasticmachine · 2025-11-25T20:30:51Z

💚 Build Succeeded

Buildkite Build
Commit: f8eeb9a

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/mock-idp-utils`	25	27	+2
`@kbn/uiam-dev-cli`	4	-	-4
`mockIdpPlugin`	5	7	+2
total			-0

Unknown metric groups

API count

id	before	after	diff
`@kbn/mock-idp-utils`	31	33	+2
`@kbn/uiam-dev-cli`	4	-	-4
`mockIdpPlugin`	5	7	+2
total			-0

History

💔 Build #365274 failed 7d9c918
💔 Build #363551 failed ae22055

jloleysens

🌔

kc13greiner

LGTM! 🚀 works great!

dmlemeshko · 2025-11-28T12:04:11Z

src/platform/packages/shared/kbn-scout/src/config/config.test.ts

    expect(scoutConfig).toEqual(expectedConfig);
  });
+
+  it(`should return a properly structured 'ScoutTestConfig' object for 'serverless=es' in UIAM mode`, async () => {


thank you for adding the tests

dmlemeshko · 2025-11-28T12:42:14Z

src/platform/packages/shared/kbn-scout/src/config/utils/utils.ts

 };
+
+export const getOrganizationId = (kbnServerArgs: string[]) => {
+  return getopts(kbnServerArgs)['xpack.cloud.organization_id'] as string | undefined;


question: it can be undefined only for stateful use case? Do we want to adjust stateful config and have a default value as well?

it can be undefined only for stateful use case?

In theory, it should be undefined only for the tests we treat as on-prem-only. The tests for both stateful ECH and stateless ECS should have the xpack.cloud.organization_id configuration set as, unlike xpack.cloud.serverless.* it's not serverless-specific.

Do we want to adjust stateful config and have a default value as well?

It's probably not used anywhere, but we can, yeah. Either in this PR or in a follow-up (#244518). Do we have some base config that we use for all our stateful tests where we already configure xpack.cloud.* (aka treat them as tests for our managed offerings)?

dmlemeshko · 2025-11-28T12:44:45Z

src/platform/packages/shared/kbn-scout/src/config/serverless/serverless.base.config.ts

      '--xpack.cloud.billing_url=/billing/overview/',
      '--xpack.cloud.deployments_url=/deployments',
      '--xpack.cloud.id=ftr_fake_cloud_id',
+      `--xpack.cloud.organization_id=${MOCK_IDP_UIAM_ORGANIZATION_ID}`,


With this in place all the serverless configs will have the same predefined org id. Do we want to have a similar predefined value for stateful? maybe also imported from '@kbn/mock-idp-utils'?

Discussed over Slack and agreed that it makes sense to add Org ID in all configs where we configure xpack.cloud - will do it in #244518. Thanks!

dmlemeshko

code changes LGTM, left few questions

…n UIAM mode (elastic#243752) ## Summary Make `@kbn\es` to manage UIAM containers when in Serverless + UIAM mode as it's going to be a default for Serverless at some point. __NOTE:__ This PR includes some groundwork for initializing UIAM in tests (i.e., passing the `--uiam` flag from test configs down to `@kbn/es` APIs). However, it doesn't include tests yet. The reason is that I'd like to start with Scout tests, but it's not yet possible to have a dedicated Scout config where we can toggle UIAM mode. I'm working with @elastic/appex-qa to add the remaining changes to make this happen, but it will likely go in a follow-up. If you feel like we need to move the test-related changes to a separate PR, just let me know. ## How to test First, run Elasticsearch Severless in the UIAM mode: ```shell $ yarn es serverless --projectType oblt --uiam --- │ info es01 is running. │ Container Name: es01 │ Container Id: 9bb06af54a4b98c4dd3f75e21e6b1a47e5bad7cf2b8e0d7f18e3ec0e3321cf72 │ │ View logs: docker logs -f es01 │ Shell access: docker exec -it es01 /bin/bash │ │ info es02 is running. │ Container Name: es02 │ Container Id: 688e43ce813c68450c70c87f2ccc29738e06e0a45a7ccd4041c86718ac103866 │ │ View logs: docker logs -f es02 │ Shell access: docker exec -it es02 /bin/bash │ │ info es03 is running. │ Container Name: es03 │ Container Id: 658fe85658de8d0915e63480dc6b96b1e5e4e092e48314b001965c039eb104f4 │ │ View logs: docker logs -f es03 │ Shell access: docker exec -it es03 /bin/bash ... info Waiting for "uiam-cosmosdb" container (starting)… info Waiting for "uiam" container (starting)… │ info The "uiam-cosmosdb" container is up and ready. │ Container Name: uiam-cosmosdb │ Container Id: e0ea6a43bfc5379b10cf5af941b6677e0cb8d90e075b4b36ff0c487ff188386a │ │ View logs: docker logs -f uiam-cosmosdb │ Shell access: docker exec -it uiam-cosmosdb /bin/bash │ │ info The "uiam" container is up and ready. │ Container Name: uiam │ Container Id: b4a9b5f56bc2155201fc942e22d1fde3d51453e60b45cf74693b43535edd1cb4 │ │ View logs: docker logs -f uiam │ Shell access: docker exec -it uiam /bin/bash │ ... succ ES cluster is ready succ Created role mapping for mock identity provider. You can now login using cloud-saml-kibana realm ``` Then, run Kibana Serverless in the UIAM mode: ```shell $ yarn start --serverless=oblt --uiam --- ... Kibana will be configured to support UIAM. [2025-11-26T09:17:56.417+01:00][INFO ][root] Kibana is starting ... [2025-11-26T09:19:34.241+01:00][INFO ][status] Kibana is now available ``` At this point, you should be able to log in into Kibana through a mock IdP provider/select as you'd normally do. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

## Summary Add basic UIAM Scout API tests: * ✅ It should be possible to login via UIAM flow * ✅ It should be possible to invalidate UIAM session token * ✅ It should be possible to refresh UIAM session token ~~__Blocked By:__ #244306 and #243752 --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>

## Summary Add basic UIAM Scout API tests: * ✅ It should be possible to login via UIAM flow * ✅ It should be possible to invalidate UIAM session token * ✅ It should be possible to refresh UIAM session token ~~__Blocked By:__ elastic#244306 and elastic#243752 --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>

azasypkin commented Nov 21, 2025

View reviewed changes

src/platform/packages/shared/kbn-es/src/utils/docker_uiam.ts Outdated Show resolved Hide resolved

azasypkin commented Nov 21, 2025

View reviewed changes

src/platform/packages/shared/kbn-es/src/utils/docker_uiam.ts Show resolved Hide resolved

azasypkin commented Nov 21, 2025

View reviewed changes

src/platform/packages/shared/kbn-es/src/utils/docker_uiam.ts Outdated Show resolved Hide resolved

azasypkin commented Nov 21, 2025

View reviewed changes

delanni reviewed Nov 21, 2025

View reviewed changes

src/platform/packages/shared/kbn-es/src/utils/docker.ts Outdated Show resolved Hide resolved

delanni reviewed Nov 21, 2025

View reviewed changes

azasypkin added 2 commits November 25, 2025 13:16

feat(security, uiam): make @kbn\es to manage UIAM containers when i…

adcd200

…n UIAM mode

feat: run and initialize UIAM containers in parallel + simplify Cosmo…

7d9c918

…sDB emulator HTTPS setup

azasypkin force-pushed the issue-xxx-uiam-integrated-dev-tooling branch from ae22055 to 7d9c918 Compare November 25, 2025 16:48

azasypkin commented Nov 25, 2025

View reviewed changes

Changes from node scripts/regenerate_moon_projects.js --update

f8eeb9a

azasypkin marked this pull request as ready for review November 26, 2025 08:34

azasypkin requested review from a team as code owners November 26, 2025 08:34

azasypkin added dev release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting labels Nov 26, 2025

azasypkin requested a review from kc13greiner November 26, 2025 09:05

jloleysens approved these changes Nov 26, 2025

View reviewed changes

azasypkin added 2 commits November 26, 2025 16:47

fix: allow using --uiam flag in Scout configs

5182a95

Merge branch 'main' into issue-xxx-uiam-integrated-dev-tooling

a3e105c

kc13greiner approved these changes Nov 26, 2025

View reviewed changes

azasypkin added 2 commits November 27, 2025 11:39

Merge branch 'main' into issue-xxx-uiam-integrated-dev-tooling

4776b9b

feat(tests): extend kbn-scout and kbn-test to propagate UIAM parameters

68e9b99

azasypkin mentioned this pull request Nov 27, 2025

feat(security,tests): add basic UIAM Scout API tests #244518

Merged

delanni approved these changes Nov 27, 2025

View reviewed changes

azasypkin self-assigned this Nov 27, 2025

dmlemeshko reviewed Nov 28, 2025

View reviewed changes

dmlemeshko approved these changes Nov 28, 2025

View reviewed changes

azasypkin merged commit 6f6ef4c into elastic:main Nov 28, 2025
14 checks passed

azasypkin deleted the issue-xxx-uiam-integrated-dev-tooling branch November 28, 2025 13:24

kibanamachine added the v9.3.0 label Nov 28, 2025


		const fetchDispatcher = new Agent({ connect: { rejectUnauthorized: false } });

		log.info(chalk.bold(`Initializing Cosmos DB (${COSMOS_DB_ENDPOINT}/${UIAM_COSMOS_DB_NAME})…`));

Conversation

azasypkin commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How to test

Uh oh!

azasypkin commented Nov 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

azasypkin Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ikuni17 commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

azasypkin commented Nov 21, 2025

Uh oh!

Uh oh!

delanni Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Nov 25, 2025

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

API count

History

Uh oh!

jloleysens left a comment

Choose a reason for hiding this comment

Uh oh!

kc13greiner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

azasypkin Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmlemeshko left a comment

Choose a reason for hiding this comment

azasypkin commented Nov 21, 2025 •

edited

Loading

azasypkin Nov 21, 2025 •

edited

Loading

Ikuni17 commented Nov 21, 2025 •

edited

Loading

delanni Nov 21, 2025 •

edited

Loading

azasypkin Nov 28, 2025 •

edited

Loading