This directory contains the cloudtrail plugin, which can fetch log files containing cloudtrail events, parse the log files, and emit sinsp/scap events (e.g. the events used by Falco) for each cloudtrail log entry.

The plugin can be configured to obtain log files from:

• A S3 bucket
• A SQS queue that passes along SNS notifications about new log files
• A local filesystem path

The plugin also exports fields that extract information from a cloudtrail event, such as the event time, the aws region, S3 bucket/EC2 instance names, etc.

The event source for cloudtrail events is aws_cloudtrail.

Here is the current set of supported fields:

When reading log files from a S3 bucket or when reading SNS notifications from a SQS queue, the plugin needs authentication credentials and to be configured with an AWS Region. The plugin relies on the same authentication mechanisms used by the AWS Go SDK:

• Environment Variables: specify the aws region with AWS_REGION=xxx, the access key id with AWS_ACCESS_KEY_ID=xxx, and the secret key with AWS_SECRET_ACCESS_KEY=xxx. Here's a sample command line:

AWS_DEFAULT_REGION=us-west-1 AWS_ACCESS_KEY_ID=XXX AWS_SECRET_ACCESS_KEY=XXX falco -c <path-to-falco.yaml> -r <path-to-falco-rules>

• Shared Configuration Files: specify the aws region in a file at $HOME/.aws/config and the credentials in a file at $HOME/.aws/credentials. Here are example files:

[default]
region = us-west-1

[default]
aws_access_key_id=<YOUR-AWS-ACCESS-KEY-ID-HERE>
aws_secret_access_key=<YOUR-AWS-SECRET-ACCESS-KEY-HERE>

The format of the initialization string is a json object. Here's an example:

{"sqsDelete": false, "s3DownloadConcurrency": 64, "useS3SNS": true}

The json object has the following properties:

• sqsDelete: value is boolean. If true, then the plugin will delete sqs messages from the queue immediately after receiving them. (Default: true)
• s3DownloadConcurrency: value is numeric. Controls the number of background goroutines used to download S3 files. (Default: 1)
• s3Interval: value is string. Download log files matching the specified time interval. Note that this matches log file names, not event timestamps. CloudTrail logs usually cover the previous 5 minutes of activity. See Time Intervals below for possible formats.
• useS3SNS: value is boolean. If true, then the plugin will expect SNS messages to originate from S3 instead of directly from Cloudtrail (Default: false)
• s3AccountList: value is string. Download log files matching the specified account IDs (in a comma separated list) in an organization trail. See Read From S3 Bucket Directly below for more details.
• sqsOwnerAccount: value is string. The AWS account ID that owns the SQS queue in case the queue is owned by a different account. Not required by default.
• aws: value is object. AWS SDK config override block.
  • profile: value is string. Overrides shared AWS profile (for example default). (Default: empty)
  • region: value is string. Overrides AWS region used by the plugin. (Default: empty)
  • config: value is string. Overrides shared config file path (for example ~/.aws/config). (Default: empty)
  • credentials: value is string. Overrides shared credentials file path (for example ~/.aws/credentials). (Default: empty)
• useAsync: value is boolean. Enables async extraction optimization. (Default: true)

The init string can be the empty string, which is treated identically to {}.

S3Interval values can be individual duration values or RFC 3339-style timestamps. In this case the interval will start at the specified time and end at the current time:

• 5d: A simple duration relative to the current time.
• 2021-03-30T18:07:17Z: A duration starting from the specified RFC 3339 formatted time.

Simple durations must be a positive integer followed by w for weeks, d for days, h for hours, m for minutes, or s for seconds. RFC 3339-style times must be formatted as a datestamp, the letter T, a timestamp with no fractional seconds, and the letter Z, e.g. 2021-03-30T18:07:17Z.

Values can also cover a range:

• 5d-2d: A simple duration interval relative to the current time.
• 2023-04-05T06:00:00Z-2023-04-05T12:00:10Z: An RFC 3339-style timestamp interval.
• 2023-04-05T06:00:00Z-5d: A combination of an RFC 3339-style timestamp and a duration.

The format of the open params string is a uri-like string with one of the following forms:

• s3://<S3 Bucket Name>[/<Optional Prefix>]
• sqs://<SQS Queue Name>
• <Some Filesystem Path>

We describe each of these below.

When using s3://<S3 Bucket Name>/[<Optional Prefix>], the plugin will scan the bucket a single time for all objects. Characters up to the first slash/end of string will be used as the S3 bucket name, and any remaining characters will be treated as a key prefix. After reading all objects, the plugin will return EOF.

All objects below the bucket, or below the bucket + prefix, will be considered cloudtrail logs. Any object ending in .json.gz will be decompressed first.

For example, if a bucket my-s3-bucket contained cloudtrail logs below a prefix AWSLogs/411571310278/CloudTrail/us-west-1/2021/09/23/, Using an open params of s3://my-s3-bucket/AWSLogs/411571310278/CloudTrail/us-west-1/2021/09/23/ would configure the plugin to read all files below AWSLogs/411571310278/CloudTrail/us-west-1/2021/09/23/ as cloudtrail logs and then return EOF. No other files in the bucket will be read.

For organization trails the files are normally stored like s3://bucket_name/prefix_name/AWSLogs/O-ID/Account ID/CloudTrail/Region/YYYY/MM/DD/file_name.json.gz. Using an open parameter of s3//my-s3-bucket/AWSLogs/o-123abc/ would configure the plugin to read all files for all account IDs in the organization o-123abc, for all regions and the entire retention time. Therefore it makes sense to combine this open parameter with S3AccountList and S3Interval parameters. S3AccountList is a comma separated string with account IDs to query.

Setting S3AccountList to 012345678912,987654321012 and S3Interval to 3d-1d with open parameter s3://my-s3-bucket/AWSLogs/o-123abc/ would get all events for account IDs 12345678912 and 987654321012 for all regions from 3 days ago up to to 1 day ago.

When using sqs://<SQS Queue Name>, the plugin will read messages from the provided SQS Queue. The messages are assumed to be SNS Notifications that announce the presence of new Cloudtrail log files in a S3 bucket. Each new file will be read from the provided s3 bucket.

In case the queue is owned by another AWS account, use the SQSOwnerAccount parameter to specify the account ID of the queue's owner. Note that the queue owner must grant you the necessary permissions to access the queue.

In this mode, the plugin polls the queue forever, waiting for new log files.

All other open params are interpreted as a filesystem path to a single cloudtrail log file. This fill will be read and parsed. When complete, the plugin returns EOF.

Here is a complete falco.yaml snippet showing valid configurations for the cloudtrail plugin:

# Cloudtrail reading from a S3 Bucket
plugins:
  - name: cloudtrail
    library_path: libcloudtrail.so
    init_config: ""
    open_params: "s3://my-s3-bucket/AWSLogs/411571310278/CloudTrail/us-west-1/2021/09/23/"

# Optional. If not specified the first entry in plugins is used.
load_plugins: [cloudtrail, json]

# Cloudtrail reading from a SQS Queue
plugins:
  - name: cloudtrail
    library_path: libcloudtrail.so
    init_config: '{"sqsDelete": true}'
    open_params: "sqs://my-sqs-queue"

# Optional. If not specified the first entry in plugins is used.
load_plugins: [cloudtrail, json]

# Cloudtrail reading from a single file
plugins:
  - name: cloudtrail
    library_path: libcloudtrail.so
    init_config: ""
    open_params: "/home/user/cloudtrail-logs/059797578166_CloudTrail_us-east-1_20210209T0130Z_65lDDH3uferZH5Br.json.gz"

# Optional. If not specified the first entry in plugins is used.
load_plugins: [cloudtrail, json]

Note that the plugin does not create any Cloudtrails, S3 Buckets, SNS Notifications, or SQS Queues. It assumes that those resources have already been created.

The general steps involve:

1. Creating a Cloudtrail with the events you would like to monitor: https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html
2. Configuring SNS Notifications for Cloudtrail: https://docs.aws.amazon.com/awscloudtrail/latest/userguide/configure-sns-notifications-for-cloudtrail.html
3. Creating a SQS Queue to receive the SNS Notifications: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-configure-subscribe-queue-sns-topic.html