Skip to content

[aws] Ingest managed insights from Security Hub#14354

Merged
chrisberkhout merged 3 commits intoelastic:mainfrom
chrisberkhout:aws-securityhub_insights-managed-insights
Jul 1, 2025
Merged

[aws] Ingest managed insights from Security Hub#14354
chrisberkhout merged 3 commits intoelastic:mainfrom
chrisberkhout:aws-securityhub_insights-managed-insights

Conversation

@chrisberkhout
Copy link
Copy Markdown
Contributor

@chrisberkhout chrisberkhout commented Jun 27, 2025

Proposed commit message

[aws] Ingest managed insights from Security Hub

The AWS Security Hub API's GetInsights endpoint[1] returns only custom
insights. Managed insights[2] are built-in, and can only be retrieved
by ARN.

Fetch a hard-coded list of managed insights first, then fetch pages of
custom insights as before.

This has been tested against the live API.

[1]: https://docs.aws.amazon.com/securityhub/1.0/APIReference/API_GetInsights.html
[2]: https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-managed-insights.html

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

@chrisberkhout chrisberkhout self-assigned this Jun 27, 2025
@chrisberkhout chrisberkhout requested a review from a team as a code owner June 27, 2025 16:50
@chrisberkhout chrisberkhout added enhancement New feature or request Integration:aws AWS Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Jun 27, 2025
@elasticmachine
Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@chrisberkhout chrisberkhout requested review from a team as code owners June 27, 2025 16:52
@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

elastic-vault-github-plugin-prod bot commented Jun 27, 2025

🚀 Benchmarks report

Package aws 👍(13) 💚(7) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
guardduty 1061.57 887.31 -174.26 (-16.42%) 💔
inspector 2012.07 1492.54 -519.53 (-25.82%) 💔

To see the full report comment with /test benchmark fullreport

Copy link
Copy Markdown
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit message, s/retieved/retrieved/.

Comment on lines +81 to +84
# This transform's job is to fail and end pagination when finished.
# It will set a value if the last request was the first in the sequence
# (which collects the managed insights), or if there is another page of
# custom insights to fetch.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's enough HTTPJSON magic here that I think the comment needs to explain more about how it is actually working.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a sentence. Here's the full new comment:

      # This transform's job is to fail and end pagination when finished.
      # It will set a value if the last request was the first in the sequence
      # (which collects the managed insights), or if there is another page of
      # custom insights to fetch. Otherwise, there will be a value template
      # error (.last_response.body.NextToken doesn't exist) and the pagination
      # sequence will end (because fail_on_template_error is set).

@chrisberkhout chrisberkhout force-pushed the aws-securityhub_insights-managed-insights branch from ea0efa3 to 0444f3e Compare June 30, 2025 08:43
@chrisberkhout chrisberkhout requested a review from efd6 June 30, 2025 08:44
@elastic-sonarqube
Copy link
Copy Markdown

@elasticmachine
Copy link
Copy Markdown

💚 Build Succeeded

History

cc @chrisberkhout

Copy link
Copy Markdown
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@chrisberkhout chrisberkhout merged commit 08dd8a2 into elastic:main Jul 1, 2025
7 checks passed
@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

Package aws - 3.9.0 containing this change is available at https://epr.elastic.co/package/aws/3.9.0/

robester0403 pushed a commit to robester0403/integrations that referenced this pull request Jul 8, 2025
The AWS Security Hub API's GetInsights endpoint[1] returns only custom
insights. Managed insights[2] are built-in, and can only be retrieved
by ARN.

Fetch a hard-coded list of managed insights first, then fetch pages of
custom insights as before.

This has been tested against the live API.

[1]: https://docs.aws.amazon.com/securityhub/1.0/APIReference/API_GetInsights.html
[2]: https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-managed-insights.html
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:aws AWS Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aws] Security Hub integration to ingest managed insights

3 participants