filbeat config:
filebeat.inputs:
- type: aws-cloudwatch
log_group_arn: arn:aws:logs:us-east-1:REDACTED:log-group:/ecs/REDACTED:*
scan_frequency: 1m
start_position: beginning
json.keys_under_root: true
json.add_error_key: true
json.message_key: message
Cloudwatch logs:
Indexed logs:
...
"message" : """{"timestamp":"2021-06-23T07:27:36.998","level":"INFO","thread":"ForkJoinPool.commonPool-worker-3","logger":"com.mycompany.myapp.RequestHandler","message":"Processing for immediate request status: Progress={jobId=REDACTED, mediaCount=100, mediaProcessed=100, status=PROCESSING, error=null, errorCount=0, started=Wed Jun 16 05:30:52 UTC 2021, finished=null, took=null}","context":"default"}""",
...
I would have expected the json.* properties to kick in, the message field (which is all json) to be parsed as json and the resulting object keys to be put on the document. This includes overwriting the message field itself with the one in the json object.
This works as expected when using a log file input.
For confirmed bugs, please report:
- Version: 7.13.0
- Operating System: N/A (task deployed to ECS)
filbeat config:
Cloudwatch logs:
Indexed logs:
I would have expected the json.* properties to kick in, the message field (which is all json) to be parsed as json and the resulting object keys to be put on the document. This includes overwriting the message field itself with the one in the json object.
This works as expected when using a log file input.
For confirmed bugs, please report: