POST _ingest/pipeline/logs-aws.cloudtrail-2.31.2/_simulate
{
"docs": [
{
"_source": {
"event": {
"original": "{\"eventVersion\":\"1.10\",\"userIdentity\":{\"type\":\"IAMUser\",\"principalId\":\"AAAAAAAAAAAAAAAAAAAAA\",\"arn\":\"arn:aws:iam::111111111111:user/services/example\",\"accountId\":\"111111111111\",\"accessKeyId\":\"BBBBBBBBBBBBBBBBBBBB\",\"userName\":\"example\"},\"eventTime\":\"2025-03-07T00:37:45Z\",\"eventSource\":\"ec2.amazonaws.com\",\"eventName\":\"TerminateInstances\",\"awsRegion\":\"us-east-2\",\"sourceIPAddress\":\"192.51.100.77\",\"userAgent\":\"APN/1.0 HashiCorp/1.0 Terraform/1.11.0 (+https://www.terraform.io) terraform-provider-aws/4.61.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go/1.44.232 (go1.19.7; linux; amd64)\",\"requestParameters\":{\"instancesSet\":{\"items\":[{\"instanceId\":\"i-00000000000000000\"}]},\"force\":false,\"skipOsShutdown\":false},\"responseElements\":{\"requestId\":\"1cd8b93c-b87b-49e5-b569-beec383329b8\",\"instancesSet\":{\"items\":[{\"instanceId\":\"i-00000000000000000\",\"currentState\":{\"code\":32,\"name\":\"shutting-down\"},\"previousState\":{\"code\":16,\"name\":\"running\"}}]}},\"requestID\":\"1cd8b93c-b87b-49e5-b569-beec383329b8\",\"eventID\":\"d07df1c3-b488-4434-a18a-e136e1b5f7f7\",\"readOnly\":false,\"eventType\":\"AwsApiCall\",\"managementEvent\":true,\"recipientAccountId\":\"111111111111\",\"eventCategory\":\"Management\",\"tlsDetails\":{\"tlsVersion\":\"TLSv1.3\",\"cipherSuite\":\"TLS_AES_128_GCM_SHA256\",\"clientProvidedHostHeader\":\"ec2.us-east-2.amazonaws.com\"}}"
}
}
}
]
}
Parsed an AWS cloudtrail log with the provided ingest pipeline.
But when an error does occur, there are some things I expect to see, but did not:
Integration Name
AWS [aws]
Dataset Name
aws.cloudtrail
Integration Version
2.31.2 (also fails on 2.42.0)
Agent Version
n/a
Agent Output Type
elasticsearch
Elasticsearch Version
8.17.3
OS Version and Architecture
n/a
Software/API Version
n/a
Error Message
{ "error": { "message": "cannot access method/field [forEach] from a null def reference" } }Event Original
What did you do?
Parsed an AWS cloudtrail log with the provided ingest pipeline.
What did you see?
The data was ingested mostly as expected, except it contains an
error.message.What did you expect to see?
No
error.messagein the event.But when an error does occur, there are some things I expect to see, but did not:
error.messagethat provides details about the processor type and tag.event.originalfor debugging purposes. (The pipeline needs to appendpreserve_original_eventtotagson error.) I had to go back to the source system to find the failing log by offset.Anything else?
integrations/packages/aws/data_stream/cloudtrail/elasticsearch/ingest_pipeline/default.yml
Lines 51 to 58 in c77fd8f