Add support for discovery content packages based on datasets#922
Merged
mrodm merged 7 commits intoelastic:mainfrom Jul 10, 2025
Merged
Add support for discovery content packages based on datasets#922mrodm merged 7 commits intoelastic:mainfrom
mrodm merged 7 commits intoelastic:mainfrom
Conversation
mrodm
commented
Jul 8, 2025
| type: array | ||
| items: | ||
| type: object | ||
| additionalProperties: false |
Contributor
Author
There was a problem hiding this comment.
Previously, it was allowed to defined additional properties in discovery.fields.
I think this should be disallowed.
Contributor
Author
There was a problem hiding this comment.
Added changelog entry as bugfix for this
jsoriano
previously approved these changes
Jul 8, 2025
jsoriano
reviewed
Jul 8, 2025
Member
jsoriano
left a comment
There was a problem hiding this comment.
Maybe we should extend documentation of these fields.
1 task
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
💚 Build Succeeded
History
cc @mrodm |
jsoriano
approved these changes
Jul 9, 2025
juliaElastic
added a commit
to elastic/kibana
that referenced
this pull request
Jul 15, 2025
## Summary Adjusted to datasets field elastic/package-spec#922 Closes elastic/ingest-dev#5685 Added kibana task to check discovery dataset fields of content packages in EPR, check recent data ingested for matching fields, and install content packages if not installed yet. Added telemetry mappings: elastic/telemetry#4940 To verify: - enable feature flag in `kibana.dev.yml`: `xpack.fleet.enableExperimental: ['enableAutoInstallContentPackages']` - TODO start local epr with test package that has a `discovery.datasets` with a value (e.g. system.cpu), depends on [package-spec](elastic/package-spec#922) change to be available - enroll an agent and wait until some data is ingested to the dataset above - wait 10m for the task to run - verify that the test package is installed Kibana logs: ``` [2025-07-10T14:07:13.177+02:00][INFO ][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [runTask()] started [2025-07-10T14:07:13.206+02:00][DEBUG][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] ESQL query took: 11ms [2025-07-10T14:07:13.207+02:00][DEBUG][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] Found datasets with data: fleet_server.agent_versions, fleet_server.agent_status, elastic_agent.fleet_server [2025-07-10T14:07:13.207+02:00][DEBUG][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] Content packages to install: kubernetes_otel@1.1.0 [2025-07-10T14:07:13.208+02:00][DEBUG][plugins.fleet] Kicking off install of kubernetes_otel-1.1.0 from registry [2025-07-10T14:07:15.605+02:00][INFO ][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] runTask ended: success ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Michel Losier <mikelosier@gmail.com>
Bluefinger
pushed a commit
to Bluefinger/kibana
that referenced
this pull request
Jul 22, 2025
## Summary Adjusted to datasets field elastic/package-spec#922 Closes elastic/ingest-dev#5685 Added kibana task to check discovery dataset fields of content packages in EPR, check recent data ingested for matching fields, and install content packages if not installed yet. Added telemetry mappings: elastic/telemetry#4940 To verify: - enable feature flag in `kibana.dev.yml`: `xpack.fleet.enableExperimental: ['enableAutoInstallContentPackages']` - TODO start local epr with test package that has a `discovery.datasets` with a value (e.g. system.cpu), depends on [package-spec](elastic/package-spec#922) change to be available - enroll an agent and wait until some data is ingested to the dataset above - wait 10m for the task to run - verify that the test package is installed Kibana logs: ``` [2025-07-10T14:07:13.177+02:00][INFO ][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [runTask()] started [2025-07-10T14:07:13.206+02:00][DEBUG][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] ESQL query took: 11ms [2025-07-10T14:07:13.207+02:00][DEBUG][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] Found datasets with data: fleet_server.agent_versions, fleet_server.agent_status, elastic_agent.fleet_server [2025-07-10T14:07:13.207+02:00][DEBUG][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] Content packages to install: kubernetes_otel@1.1.0 [2025-07-10T14:07:13.208+02:00][DEBUG][plugins.fleet] Kicking off install of kubernetes_otel-1.1.0 from registry [2025-07-10T14:07:15.605+02:00][INFO ][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] runTask ended: success ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Michel Losier <mikelosier@gmail.com>
kertal
pushed a commit
to kertal/kibana
that referenced
this pull request
Jul 25, 2025
## Summary Adjusted to datasets field elastic/package-spec#922 Closes elastic/ingest-dev#5685 Added kibana task to check discovery dataset fields of content packages in EPR, check recent data ingested for matching fields, and install content packages if not installed yet. Added telemetry mappings: elastic/telemetry#4940 To verify: - enable feature flag in `kibana.dev.yml`: `xpack.fleet.enableExperimental: ['enableAutoInstallContentPackages']` - TODO start local epr with test package that has a `discovery.datasets` with a value (e.g. system.cpu), depends on [package-spec](elastic/package-spec#922) change to be available - enroll an agent and wait until some data is ingested to the dataset above - wait 10m for the task to run - verify that the test package is installed Kibana logs: ``` [2025-07-10T14:07:13.177+02:00][INFO ][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [runTask()] started [2025-07-10T14:07:13.206+02:00][DEBUG][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] ESQL query took: 11ms [2025-07-10T14:07:13.207+02:00][DEBUG][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] Found datasets with data: fleet_server.agent_versions, fleet_server.agent_status, elastic_agent.fleet_server [2025-07-10T14:07:13.207+02:00][DEBUG][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] Content packages to install: kubernetes_otel@1.1.0 [2025-07-10T14:07:13.208+02:00][DEBUG][plugins.fleet] Kicking off install of kubernetes_otel-1.1.0 from registry [2025-07-10T14:07:15.605+02:00][INFO ][plugins.fleet.fleet:auto-install-content-packages-task:1.0.1] [AutoInstallContentPackagesTask] runTask ended: success ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Michel Losier <mikelosier@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Add discovery of content packages based on datasets .
It allows to define a new
discovery.datasetskey in the package manifests.Checklist
test/packagesthat prove my change is effective.spec/changelog.yml.Related issues