"Within the Windows integration in the ingest pipeline, we have several processors, and one of them deletes certain non-ECS fields from the event. One of these deleted fields is actually useful.
In our installations, we modify the ingest pipeline 'logs-windows.sysmon_operational-XX.YY.ZZ' after each update. We only need to add a single SET processor before the Delete processor. Here's the required addition to the following file:
/packages/windows/data_stream/sysmon_operational/elasticsearch/ingest_pipeline/default.yml
{
"set": {
"field": "winlog.event_data.Detail",
"copy_from": "winlog.event_data.Details",
"ignore_failure": true
}
}Before processor:
{
"remove": {
"field": [
"_temp",
"winlog.event_data.ProcessId",
"winlog.event_data.ParentProcessId",
"winlog.event_data.SourceProcessId",
"winlog.event_data.SourceThreadId",
"winlog.event_data.SourceIp",
"winlog.event_data.SourcePort",
"winlog.event_data.SourcePortName",
"winlog.event_data.DestinationIp",
"winlog.event_data.DestinationPort",
"winlog.event_data.DestinationPortName",
"winlog.event_data.RuleName",
"winlog.event_data.User",
"winlog.event_data.Initiated",
"winlog.event_data.SourceIsIpv6",
"winlog.event_data.DestinationIsIpv6",
"winlog.event_data.QueryStatus",
"winlog.event_data.Archived",
"winlog.event_data.IsExecutable",
"winlog.event_data.QueryResults",
"winlog.event_data.UtcTime",
"winlog.event_data.Hash",
"winlog.event_data.Hashes",
"winlog.event_data.TargetObject",
"winlog.event_data.Details",
"winlog.time_created",
"winlog.level"
],
"ignore_failure": true,
"ignore_missing": true
}
},
Unfortunately, we cannot use a custom ingest pipeline, as the data is completely removed within the managed ingest pipeline."
"Within the Windows integration in the ingest pipeline, we have several processors, and one of them deletes certain non-ECS fields from the event. One of these deleted fields is actually useful.
In our installations, we modify the ingest pipeline 'logs-windows.sysmon_operational-XX.YY.ZZ' after each update. We only need to add a single SET processor before the Delete processor. Here's the required addition to the following file:
/packages/windows/data_stream/sysmon_operational/elasticsearch/ingest_pipeline/default.yml
{ "set": { "field": "winlog.event_data.Detail", "copy_from": "winlog.event_data.Details", "ignore_failure": true } }Before processor:
{ "remove": { "field": [ "_temp", "winlog.event_data.ProcessId", "winlog.event_data.ParentProcessId", "winlog.event_data.SourceProcessId", "winlog.event_data.SourceThreadId", "winlog.event_data.SourceIp", "winlog.event_data.SourcePort", "winlog.event_data.SourcePortName", "winlog.event_data.DestinationIp", "winlog.event_data.DestinationPort", "winlog.event_data.DestinationPortName", "winlog.event_data.RuleName", "winlog.event_data.User", "winlog.event_data.Initiated", "winlog.event_data.SourceIsIpv6", "winlog.event_data.DestinationIsIpv6", "winlog.event_data.QueryStatus", "winlog.event_data.Archived", "winlog.event_data.IsExecutable", "winlog.event_data.QueryResults", "winlog.event_data.UtcTime", "winlog.event_data.Hash", "winlog.event_data.Hashes", "winlog.event_data.TargetObject", "winlog.event_data.Details", "winlog.time_created", "winlog.level" ], "ignore_failure": true, "ignore_missing": true } },Unfortunately, we cannot use a custom ingest pipeline, as the data is completely removed within the managed ingest pipeline."