Version: 7.14.1
Description:
When Azure signin logs has callerIpAddress as IPv6, the event ingestion will get 400 error, only the first 4 digits is extracted and it's not a valid IP. Sample error when "callerIpAddress":"2600:0000:0000:0000:0000:0000:0000:0000".
"error": {
"type": "mapper_parsing_exception",
"reason": "failed to parse field [source.ip] of type [ip] in document with id 'yDxxx3sB0k_8PiquDAm0'. Preview of field's value: '2600'",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "'2600' is not an IP string literal."
}
}
Checking the pipeline "filebeat-7.14.1-azure-signinlogs-pipeline" (code),
- grok:
field: azure.signinlogs.callerIpAddress
patterns:
- \[%{IPORHOST:source.ip}\]:%{INT:source.port:int}
- "%{IPORHOST:source.ip}:%{INT:source.port:int}"
- "%{IPORHOST:source.ip}"
ignore_missing: true
ignore_failure: true
Workaround
Removing the 2nd pattern "%{IPORHOST:source.ip}:%{INT:source.port:int}" in the pipeline will make it work.
Testing pipeline, click to expand
```
POST _ingest/pipeline/_simulate
{
"pipeline": {
"description" : "parse multiple patterns",
"processors": [
{
"grok" : {
"field" : "azure.activitylogs.callerIpAddress",
"patterns" : [
"""\[%{IPORHOST:source.ip}\]:%{INT:source.port:int}""",
"%{IPORHOST:source.ip}:%{INT:source.port:int}", <--- remove this line will make it work
"%{IPORHOST:source.ip}"
],
"ignore_missing" : true,
"ignore_failure" : true
}
},
{
"remove" : {
"field" : "azure.activitylogs.callerIpAddress",
"if" : "ctx.source?.ip != null",
"ignore_missing" : true
}
},
{
"geoip" : {
"ignore_missing" : true,
"field" : "source.ip",
"target_field" : "geo"
}
}
]
},
"docs":[
{
"_source": {
"azure": {
"activitylogs":{
"callerIpAddress":"2600:0000:0000:0000:0000:0000:0000:0000"
}
}
}
}
]
}
Version: 7.14.1
Description:
When Azure signin logs has
callerIpAddressasIPv6, the event ingestion will get 400 error, only the first 4 digits is extracted and it's not a valid IP. Sample error when"callerIpAddress":"2600:0000:0000:0000:0000:0000:0000:0000".Checking the pipeline
"filebeat-7.14.1-azure-signinlogs-pipeline"(code),Workaround
Removing the 2nd pattern
"%{IPORHOST:source.ip}:%{INT:source.port:int}"in the pipeline will make it work.Testing pipeline, click to expand
``` POST _ingest/pipeline/_simulate { "pipeline": { "description" : "parse multiple patterns", "processors": [ { "grok" : { "field" : "azure.activitylogs.callerIpAddress", "patterns" : [ """\[%{IPORHOST:source.ip}\]:%{INT:source.port:int}""", "%{IPORHOST:source.ip}:%{INT:source.port:int}", <--- remove this line will make it work "%{IPORHOST:source.ip}" ], "ignore_missing" : true, "ignore_failure" : true } }, { "remove" : { "field" : "azure.activitylogs.callerIpAddress", "if" : "ctx.source?.ip != null", "ignore_missing" : true } }, { "geoip" : { "ignore_missing" : true, "field" : "source.ip", "target_field" : "geo" } }]
},
"docs":[
{
"_source": {
"azure": {
"activitylogs":{
"callerIpAddress":"2600:0000:0000:0000:0000:0000:0000:0000"
}
}
}
}
]
}