I have been looking at the Windows integration and found an error in the mappings. Full details at the link below of my report and a suggested fix.
Discuss post
We have an environment where all Windows event logs are forwarded to a central log collector. This device has been running winlogbeat to collect logs from the 'Forwarded Events' channel to Elastic.
Today I tried running this via the Windows integration which has options to collect various log events, however there is an error on ingest.
Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0x2379e348, ext:63767206896, loc:(*time.Location)(nil)}, Meta:{"raw_index":"logs-windows.forwarded-default"}, Fields:{"agent":{"ephemeral_id":"b512edb8-9998-42bd-8941-e1e53d750cc9","hostname":"logcollector2","id":"ef67a54e-002d-4c31-a1c8-931a1a56bea4","name":"logcollector2","type":"filebeat","version":"7.14.1"},"data_stream":{"dataset":"windows.forwarded","namespace":"default","type":"logs"},"ecs":{"version":"1.10.0"},"elastic_agent":{"id":"ef67a54e-002d-4c31-a1c8-931a1a56bea4","snapshot":false,"version":"7.14.1"},"event":{"code":"22","created":"2021-09-14T09:20:46.257Z","dataset":"windows.forwarded","kind":"event","provider":"Microsoft-Windows-Sysmon"},"host":{"name":"DEVICENAME"},"input":{"type":"winlog"},"log":{"level":"information"},"tags":["forwarded"],"winlog":{"api":"wineventlog","channel":"Microsoft-Windows-Sysmon/Operational","computer_name":"DEVICENAME","event_data":{"Image":"\u003cunknown process\u003e","ProcessGuid":"{00000000-0000-0000-0000-000000000000}","ProcessId":"6968","QueryName":"enterpriseregistration.windows.net","QueryResults":"type: 5 adrs.privatelink.msidentity.com;type: 5 www.tm.prd.adrs.akadns.net;::ffff:20.190.129.168;","QueryStatus":"0","RuleName":"-","UtcTime":"2021-09-14 09:01:34.006"},"event_id":"22","process":{"pid":2412,"thread":{"id":3596}},"provider_guid":"{5770385f-c22a-43e0-bf4c-06f5698ffbd9}","provider_name":"Microsoft-Windows-Sysmon","record_id":12337,"user":{"domain":"NT AUTHORITY","identifier":"S-1-5-18","name":"SYSTEM","type":"User"},"version":5}}, Private:(*cursor.updateOp)(0xc0027a8980), TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [related.ip] of type [ip] in document with id 'Hz-b43sBuVXDW4lt2EVA'. Preview of field's value: '_ingest._value'","caused_by":{"type":"illegal_argument_exception","reason":"'_ingest._value' is not an IP string literal."}}
A response from @andrewkroh suggests that:
Looking at the pipeline for the forwarded events vs the pipeline for the sysmon dataset, I think the two got out of sync. The forwarded one needs the {{ }} to make it a templated value.
He gives a full explanation on the Discuss forum so it may be easier to refer to that.
I have been looking at the Windows integration and found an error in the mappings. Full details at the link below of my report and a suggested fix.
Discuss post
We have an environment where all Windows event logs are forwarded to a central log collector. This device has been running winlogbeat to collect logs from the 'Forwarded Events' channel to Elastic.
Today I tried running this via the Windows integration which has options to collect various log events, however there is an error on ingest.
Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0x2379e348, ext:63767206896, loc:(*time.Location)(nil)}, Meta:{"raw_index":"logs-windows.forwarded-default"}, Fields:{"agent":{"ephemeral_id":"b512edb8-9998-42bd-8941-e1e53d750cc9","hostname":"logcollector2","id":"ef67a54e-002d-4c31-a1c8-931a1a56bea4","name":"logcollector2","type":"filebeat","version":"7.14.1"},"data_stream":{"dataset":"windows.forwarded","namespace":"default","type":"logs"},"ecs":{"version":"1.10.0"},"elastic_agent":{"id":"ef67a54e-002d-4c31-a1c8-931a1a56bea4","snapshot":false,"version":"7.14.1"},"event":{"code":"22","created":"2021-09-14T09:20:46.257Z","dataset":"windows.forwarded","kind":"event","provider":"Microsoft-Windows-Sysmon"},"host":{"name":"DEVICENAME"},"input":{"type":"winlog"},"log":{"level":"information"},"tags":["forwarded"],"winlog":{"api":"wineventlog","channel":"Microsoft-Windows-Sysmon/Operational","computer_name":"DEVICENAME","event_data":{"Image":"\u003cunknown process\u003e","ProcessGuid":"{00000000-0000-0000-0000-000000000000}","ProcessId":"6968","QueryName":"enterpriseregistration.windows.net","QueryResults":"type: 5 adrs.privatelink.msidentity.com;type: 5 www.tm.prd.adrs.akadns.net;::ffff:20.190.129.168;","QueryStatus":"0","RuleName":"-","UtcTime":"2021-09-14 09:01:34.006"},"event_id":"22","process":{"pid":2412,"thread":{"id":3596}},"provider_guid":"{5770385f-c22a-43e0-bf4c-06f5698ffbd9}","provider_name":"Microsoft-Windows-Sysmon","record_id":12337,"user":{"domain":"NT AUTHORITY","identifier":"S-1-5-18","name":"SYSTEM","type":"User"},"version":5}}, Private:(*cursor.updateOp)(0xc0027a8980), TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [related.ip] of type [ip] in document with id 'Hz-b43sBuVXDW4lt2EVA'. Preview of field's value: '_ingest._value'","caused_by":{"type":"illegal_argument_exception","reason":"'_ingest._value' is not an IP string literal."}}A response from @andrewkroh suggests that:
Looking at the pipeline for the forwarded events vs the pipeline for the sysmon dataset, I think the two got out of sync. The forwarded one needs the {{ }} to make it a templated value.
He gives a full explanation on the Discuss forum so it may be easier to refer to that.