PUT requests in AWS S3 access logs may include a hyphen in place of an IP address, so the %{IP:aws.s3access.remote_ip} grok pattern for the message field defined in filebeat-7.13.3-aws-s3access-pipeline results in the following error: “Provided Grok expressions do not match field value”.
Modifying the pattern to (?:-|%{IP:aws.s3access.remote_ip}) resolves the issue.
- Version: 7.13.3
- Operating System: Amazon Linux
- Discuss Forum URL:
- Steps to Reproduce:
POST _ingest/pipeline/filebeat-7.13.3-aws-s3access-pipeline/_simulate
{
"docs": [
{
"_source": {
"message": """67797214d75628047d9c76b18a78cded1a4b069b71f2a9d5a53649c38da8770b flow-log-test [14/Jul/2021:18:57:31 +0000] - svc:delivery.logs.amazonaws.com MVGXZXEVN3IG9S24 REST.PUT.OBJECT AWSLogs/000000000000/vpcflowlogs/us-gov-east-1/2021/07/13/000000000000_vpcflowlogs_us-gov-east-1_fl-_20210713T1855Z_f12aa632.log.gz "PUT /AWSLogs/000000000000/vpcflowlogs/us-gov-east-1/2021/07/13/000000000000_vpcflowlogs_us-gov-east-1_fl-0e7c13bf00cf15bfe_20210713T1855Z_f12aa632.log.gz HTTP/1.1" 200 - - 773 103 13 "-" "-" - 02SxwfXpO5UysN0GsKGa3uGDQ6E/W7+Hwo/luRH8p1VEexULoe66RCM+nja0dEq2JqLrtgjocvVRRkVt4= SigV4 ECDHE-RSA-AES128-GCM-SHA256 AuthHeader flow-log-test.s3.us-gov-west-1.amazonaws.com TLSv1.2 -
"""
}
}
]
}
PUT requests in AWS S3 access logs may include a hyphen in place of an IP address, so the
%{IP:aws.s3access.remote_ip}grok pattern for themessagefield defined infilebeat-7.13.3-aws-s3access-pipelineresults in the following error: “Provided Grok expressions do not match field value”.Modifying the pattern to
(?:-|%{IP:aws.s3access.remote_ip})resolves the issue.