[Spacetime][Fleet] Introduce airgapped config for bundled packages by criamico · Pull Request #202435 · elastic/kibana

criamico · 2024-12-02T11:18:14Z

Summary

[Spacetime] Improving Integrations experience on airgapped envs using the existing xpack.fleet.isAirGapped configuration key:

Loading integrations is now much faster and doesn't attempt to contact the registry at all
Installing an uninstalled bundled packages should now be possible
NOTE: Setting the isAirGapped skips the calls to registry altogether

Testing

In kibana.dev.yml:
- Make sure that APM and other bundled packages are not present in preconfiguration
- Configure xpack.fleet.isAirGapped: true
Copy the zip files from the EPR for some common bundled packages to them in them in kibana/x-pack/plugins/fleet/target/bundled_packages
Navigate to Integrations and verify that the page loads faster than in the past (it would take long because of retries)
Navigate to apm page or toapp/integrations/detail/apm-8.4.2/overview
Try to install it it should succeed correctly

Checklist

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

elasticmachine · 2024-12-02T11:20:41Z

Pinging @elastic/fleet (Team:Fleet)

criamico · 2024-12-02T11:21:33Z

@elasticmachine merge upstream

juliaElastic · 2024-12-02T12:50:47Z

Tested the scenario in the description and it seems to work.
I am wondering though how would users discover the correct URL to navigate to a bundled package, since loading the integrations landing page takes a long time to load.
Maybe we could use the isAirGapped flag to also prevent reaching out to the epr URL for any requests, so the bundled packages would load quickly on the landing page.

criamico · 2024-12-02T13:29:39Z

Maybe we could use the isAirGapped flag to also prevent reaching out to the epr URL for any requests, so the bundled packages would load quickly on the landing page.

@juliaElastic I think you're right, I was also thinking something like that. I didn't have much time to work on this issue last week so I opened the PR with what I had working.

As a follow up we could even show the isAirgapped flag on the settings page to make it more visible.

kpollich · 2024-12-02T13:42:14Z

Maybe we could use the isAirGapped flag to also prevent reaching out to the epr URL for any requests, so the bundled packages would load quickly on the landing page.

I think we could do this in the epm/requests.ts module to make sure these requests can never fire. This would be a root fix for any external requests to EPR. We can handle this in a follow-up PR that closes #167195

criamico · 2024-12-02T15:03:20Z

I think we could do this in the epm/requests.ts module to make sure these requests can never fire. This would be a root fix for any external requests to EPR.

@kpollich @juliaElastic
I pushed 61ff101 that does just that. I tested it locally and it's actually faster than I thought, so this could actually fix #167195 too.

EDIT: I need to fix some linting errors and do some more testing, so I'm putting this back to draft

criamico · 2024-12-04T13:55:19Z

@elasticmachine merge upstream

criamico · 2024-12-05T13:22:48Z

@kpollich @juliaElastic
The way I'm using isAirGapped config here, I'm excluding altogether the calls to the registry. I'm thinking that this could be an issue if the user has RegistryUrl (or registryProxyUrl) set up along with isAirGapped, as the local registry won't be reached at all. I could add additional checks to avoid stopping the calls when registryUrl is present. What do you think?

kpollich · 2024-12-05T14:19:21Z

I could add additional checks to avoid stopping the calls when registryUrl is present. What do you think?

I think the condition should be something like isAirgapped && !(registryUrl || registryProxyUrl) e.g. "if this is an airgapped environment with no configured registry URL" e.g.

const hasRegistryUrl = registryUrl || registryProxyURl
const shouldSkipRegistryRequests = isAirgapped && !hasRegistryUrl

criamico · 2024-12-05T14:21:56Z

I could add additional checks to avoid stopping the calls when registryUrl is present. What do you think?

I think the condition should be something like isAirgapped && !(registryUrl || registryProxyUrl) e.g. "if this is an airgapped environment with no configured registry URL" e.g.
const hasRegistryUrl = registryUrl || registryProxyURl
const shouldSkipRegistryRequests = isAirgapped && !hasRegistryUrl

@kpollich
That's what I was thinking too. It would only leave unresolved the case when the registryUrl is configured but unreachable. I'm not sure what should be the approach in that case.

kpollich · 2024-12-05T14:25:25Z

when the registryUrl is configured but unreachable

I think we want errors to surface in this case because it's a configuration issue that the user needs to resolve. So this seems acceptable to me.

criamico · 2024-12-09T08:57:28Z

@elasticmachine merge upstream

criamico · 2024-12-10T16:15:16Z

@elasticmachine merge upstream

x-pack/plugins/fleet/server/routes/epm/file_handler.ts

juliaElastic · 2024-12-11T10:30:52Z

x-pack/plugins/fleet/server/services/epm/registry/requests.ts

+  if (res) {
+    return res?.body;
+  }
+  throw new RegistryResponseError('isAirGapped config enabled, registry not reacheable');


is this line for testing?

juliaElastic · 2024-12-11T10:33:20Z

x-pack/plugins/fleet/server/services/epm/airgapped.ts

+
+  const shouldSkipRegistryRequests = isAirGapped && !hasRegistryUrls;
+
+  return { hasRegistryUrls, isAirGapped, shouldSkipRegistryRequests };


we should probably create a documentation request to clarify what combination of configs prevents requests to EPR

I think we should skip requests when isAirGapped:true is set, regardless of whether registryUrl is set. Currently if I set both, the requests are still made.

Tested without registryUrl and isAirGapped:true, it looks good, I'm seeing the integrations load quickly and the installed integrations are also showing up.
For some reason I don't see all bundled packages, e.g. endpoint which is not installed.

I think we should skip requests when isAirGapped:true is set, regardless of whether registryUrl is set. Currently if I set both, the requests are still made.

This was my first idea too but having an airgapped environment with a local registry is a legit use case that we are already supporting (see the docs).

If I just block the requests when isAirGapped is set to true, I'm going to break this use case for many users, that's why @kpollich proposed to check both conditions.

For some reason I don't see all bundled packages, e.g. endpoint which is not installed.

I noticed it too, I'm not sure why. I can open a follow up ticket to check what's going on.

I see, it's ok then

juliaElastic

LGTM

criamico · 2024-12-11T15:34:19Z

@elasticmachine merge upstream

elasticmachine · 2024-12-11T17:22:51Z

💚 Build Succeeded

Buildkite Build
Commit: 70d26a1

Metrics [docs]

✅ unchanged

History

💚 Build #258872 succeeded b20d001
💚 Build #257546 succeeded 57839fe
💚 Build #257310 succeeded d17d493
💔 Build #257054 failed d868d18
💔 Build #256937 failed 54901c5
💔 Build #256831 failed 419ff38

cc @criamico

…lastic#202435) Closes elastic#136617 Closes elastic#167195 ## Summary [Spacetime] Improving Integrations experience on airgapped envs using the existing `xpack.fleet.isAirGapped` configuration key: - Loading integrations is now much faster and doesn't attempt to contact the registry at all - Installing an uninstalled bundled packages should now be possible NOTE: Setting the `isAirGapped` skips the calls to registry altogether ### Testing - In `kibana.dev.yml`: - Make sure that APM and other bundled packages are not present in preconfiguration - Configure `xpack.fleet.registryUrl` to an unreacheable host, i.e. `xpack.fleet.registryUrl: http://notworking` - Configure `xpack.fleet.isAirGapped: true` - Copy the zip files from the EPR for some common bundled packages to them in `them in kibana/x-pack/plugins/fleet/target/bundled_packages` - Navigate to Integrations and verify that the page loads faster than in the past (it would take long because of retries) - Navigate to apm page or to`app/integrations/detail/apm-8.4.2/overview` - Try to install it it should succeed correctly ### Checklist - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

chais0n · 2025-03-06T16:40:24Z

Just to add some context here - All of my customers are Air-gapped and every single one of them hosts a local EPR container. However, in testing and other situations, it would be nice that it didn't take the page forever to load if they don't have a Registry URL configured.

This makes perfect sense to me:

const hasRegistryUrl = registryUrl || registryProxyURl
const shouldSkipRegistryRequests = isAirgapped && !hasRegistryUrl

If they have a Registry URL configured and it's not working then I agree that is a configuration/environment issue that needs to be investigated.

…uilt rules OOM tests in 8.19 (#245087) **Partially addresses:** #188090 ## Summary Fleet doesn't have #202435 backported to 8.19. It blocks package removal for packages installed by direct upload. With #228777 we have a We have a mock prebuilt rules package installed via upload at the global hook. This PR fixes the issue by skipping mock prebuilt rules package installation.

[Spacetime][Fleet] Introduce airgapped config for bundled packages

683ed40

criamico added Team:Fleet Team label for Observability Data Collection Fleet team Spacetime release_note:fix backport:skip This PR does not require backporting labels Dec 2, 2024

criamico self-assigned this Dec 2, 2024

criamico marked this pull request as ready for review December 2, 2024 11:20

criamico requested a review from a team as a code owner December 2, 2024 11:20

Merge branch 'main' into 136617_airgapped_bundled_packages

a84150d

Block calls to registry when airgapped true

61ff101

criamico marked this pull request as draft December 2, 2024 16:43

criamico added 4 commits December 3, 2024 16:41

Fix linting errors and return types

14a61ff

Fix linter and error messages

30e258e

Fix errors

419ff38

Fix more types

7d544c0

elasticmachine and others added 4 commits December 4, 2024 13:55

Merge branch 'main' into 136617_airgapped_bundled_packages

54901c5

Handle more errors

f5dbe21

more linting errors

d868d18

Add tests and improve error naming

d17d493

criamico marked this pull request as ready for review December 5, 2024 10:47

criamico added 2 commits December 5, 2024 16:52

Consider registryUrl and registryProxyUrl when skipping requests

00fff21

improve debug messages

57839fe

lukeelmers mentioned this pull request Dec 6, 2024

Kibana-wide support for airgapped mode #203336

Closed

Merge branch 'main' into 136617_airgapped_bundled_packages

e824cf2

Merge branch 'main' into 136617_airgapped_bundled_packages

b20d001

juliaElastic reviewed Dec 11, 2024

View reviewed changes

x-pack/plugins/fleet/server/routes/epm/file_handler.ts Outdated Show resolved Hide resolved

juliaElastic reviewed Dec 11, 2024

View reviewed changes

juliaElastic approved these changes Dec 11, 2024

View reviewed changes

remove uncorrect header

72fe7f4

Merge branch 'main' into 136617_airgapped_bundled_packages

70d26a1

criamico enabled auto-merge (squash) December 11, 2024 15:35

criamico mentioned this pull request Dec 11, 2024

[Fleet] UI improvements for airgapped environments #203846

Open

4 tasks

criamico merged commit c3f96bb into elastic:main Dec 11, 2024

kibanamachine added the v9.0.0 label Dec 11, 2024

criamico deleted the 136617_airgapped_bundled_packages branch December 12, 2024 08:12

banderror mentioned this pull request Feb 28, 2025

[Security Solution] Should not require EPR access in air-gapped environments #181808

Closed

criamico mentioned this pull request Jun 24, 2025

[Fleet] Agent policies not updating in airgapped environment with multiple versions of bundled packages in use #224954

Closed

criamico mentioned this pull request Aug 5, 2025

[Fleet] Bundled packages update not working whenisAirGapped is configured #230603

Closed

criamico mentioned this pull request Oct 7, 2025

[Fleet] Improve installation of bundled packages in airgapped environments #230992

Merged

2 tasks

maximpn mentioned this pull request Dec 3, 2025

[Security Solution] Prevent global mock package installation for prebuilt rules OOM tests in 8.19 #245087

Merged


		const shouldSkipRegistryRequests = isAirGapped && !hasRegistryUrls;

		return { hasRegistryUrls, isAirGapped, shouldSkipRegistryRequests };

Conversation

criamico commented Dec 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Checklist

Uh oh!

elasticmachine commented Dec 2, 2024

Uh oh!

criamico commented Dec 2, 2024

Uh oh!

juliaElastic commented Dec 2, 2024

Uh oh!

criamico commented Dec 2, 2024

Uh oh!

kpollich commented Dec 2, 2024

Uh oh!

criamico commented Dec 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

criamico commented Dec 4, 2024

Uh oh!

criamico commented Dec 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kpollich commented Dec 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

criamico commented Dec 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kpollich commented Dec 5, 2024

Uh oh!

criamico commented Dec 9, 2024

Uh oh!

criamico commented Dec 10, 2024

Uh oh!

Uh oh!

juliaElastic Dec 11, 2024

Choose a reason for hiding this comment

Uh oh!

juliaElastic Dec 11, 2024

Choose a reason for hiding this comment

Uh oh!

juliaElastic Dec 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

criamico Dec 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juliaElastic Dec 11, 2024

Choose a reason for hiding this comment

Uh oh!

juliaElastic left a comment

Choose a reason for hiding this comment

Uh oh!

criamico commented Dec 11, 2024

Uh oh!

elasticmachine commented Dec 11, 2024

💚 Build Succeeded

Metrics [docs]

History

Uh oh!

chais0n commented Mar 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

criamico commented Dec 2, 2024 •

edited

Loading

criamico commented Dec 2, 2024 •

edited

Loading

criamico commented Dec 5, 2024 •

edited

Loading

kpollich commented Dec 5, 2024 •

edited

Loading

criamico commented Dec 5, 2024 •

edited

Loading

juliaElastic Dec 11, 2024 •

edited

Loading

criamico Dec 11, 2024 •

edited

Loading