Skip to content

[aws.config] fails to collect all compliance data due to ThrottlingException #15764

@andrewkroh

Description

@andrewkroh

Integration Name

AWS [aws]

Dataset Name

aws.config

Integration Version

9.2.0

Agent Version

9.2.0

Agent Output Type

elasticsearch

Elasticsearch Version

9.2.0

OS Version and Architecture

Official Elastic Agent docker container running on EKS (amd64)

Software/API Version

No response

Error Message

single event error object returned by evaluation: {"error":{"code":"400","id":"400 Bad Request","message":"GetComplianceDetailsByConfigRule: POST https://config.us-east-2.amazonaws.com {"__type":"ThrottlingException","message":"Rate exceeded"}"}}

Event Original

No response

What did you do?

Deployed Elastic Agent 9.2.0 using the official docker container on EKS, with Elastic Stack 9.2.0. Configured the AWS integration to collect config compliance data. Attached trace logs of all HTTP requests and responses made by the integration (see aws-config-rate-limit-trace.csv).

What did you see?

Elastic Agent health status page shows "Degraded" for the aws.config datastream. The attached aws-config-rate-limit-trace.csv contains the HTTP trace logs showing each request and response. The integration stops collecting config compliance data when it hits the ThrottlingException.

What did you expect to see?

The integration should handle or avoid AWS API throttling (ThrottlingException), and be able to collect all AWS config compliance data even if throttling is encountered. The error should not cause the datastream to become degraded or stop collecting data.

Anything else?

The attached aws-config-rate-limit-trace.csv contains the full trace of HTTP requests and responses for debugging. Please let me know if additional debug logs or configuration are needed.

aws-config-rate-limit-trace.csv

Metadata

Metadata

Assignees

Labels

Integration:awsAWSTeam:Security-Service IntegrationsSecurity Service Integrations team [elastic/security-service-integrations]bugSomething isn't working, use only for issues

Type

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions