Describe the bug:
The ingest pipelines windows.powershell* cannot parse logs with german content correctly.
Pipeline version:
1.44.5
Steps to reproduce:
Ingest logs with a message in german language. Example:
CommandInvocation(Select-Object): "Select-Object"
ParameterBinding(Select-Object): Name="Property"; Wert="Date"
ParameterBinding(Select-Object): Name="InputObject"; Wert="System.__ComObject"
Kontext:
Schweregrad: Informational
Hostname: Default Host
Hostversion: 5.1.17763.XXXX
Host-ID: 8c28d572-XXX-XXXX-XXXX-141be1a8b83d
Hostanwendung = C:\Program Files (x86)\[...]
Modulversion: 5.1.17763.XXXX
Runspace-ID: fe6a6be4-XXXX-XXXX-XXXX-aff0870ddccc
Pipeline-ID: 5
Befehlsname: Select-Object
Befehlstyp: Cmdlet
Skriptname:
Befehlspfad:
Sequenznummer: 50
Benutzer: WORKGROUP\SYSTEM
Verbundener Benutzer =
Shell-ID: Microsoft.PowerShell
Benutzerdaten:
Current behavior:
- The ingest pipeline failes on the first KV-processor (on event.code 4103) with
field [winlog.event_data.ContextInfo] does not contain value_split [=].
- The ingest pipeline does not parse the
message field correctly.
Expected behavior:
- Correct parsing of all elements of the event, regardless of the system language.
Suggestion:
To solve the problem in the short term, I suggest the following adjustments to the ingest pipeliens. in the future, however, it might be better to provide dedicated pipeliens for different system languages.
logs-windows.powershell
- Adaptation of the script processor "Parses all command invocation detail raw lines [...]" and replace the regex pattern.
- Original:
Pattern parameterBindingRegex = /name\=(.+);\s*value\=(.+)$/;
- Adaptation:
Pattern parameterBindingRegex = /(?:n|N)ame\\=(.+);\\s*(?:value|value)\\=(.+)$/;
logs-windows.powershell_operational
-
Adaptation of the script processor "Parses all command invocation detail raw lines [...]" and replace the regex pattern.
- Original:
Pattern parameterBindingRegex = /name\=(.+);\s*value\=(.+)$/;
- Adaptation:
Pattern parameterBindingRegex = /(?:n|N)ame\\=(.+);\\s*(?:value|value)\\=(.+)$/;
-
Add a failure handler to the KV-Value processor "Split Event 4103 event data fields".
- Additional KV-Value processor for splitting the German-language logs
"kv": {
"field": "winlog.event_data.ContextInfo",
"field_split": "\\n",
"value_split": "(:|=)",
"target_field": "winlog.event_data",
"trim_key": " \"",
"trim_value": " \"",
"if": "ctx?.winlog?.event_id == \"4103\"",
"ignore_failure": true
}
- Rename processors to replace the German names of the keys into English
"rename": {
"field": "winlog.event_data.Schweregrad",
"target_field": "winlog.event_data.Severity",
"ignore_missing": true,
"if": "ctx?.winlog?.event_id == \"4103\"",
"ignore_failure": true
}
[...]
rename: Hostname, Hostversion, Host-ID, Hostanwendung, ...
Thank you.
Describe the bug:
The ingest pipelines
windows.powershell*cannot parse logs with german content correctly.Pipeline version:
1.44.5
Steps to reproduce:
Ingest logs with a
messagein german language. Example:Current behavior:
field [winlog.event_data.ContextInfo] does not contain value_split [=].messagefield correctly.Expected behavior:
Suggestion:
To solve the problem in the short term, I suggest the following adjustments to the ingest pipeliens. in the future, however, it might be better to provide dedicated pipeliens for different system languages.
logs-windows.powershell
Pattern parameterBindingRegex = /name\=(.+);\s*value\=(.+)$/;Pattern parameterBindingRegex = /(?:n|N)ame\\=(.+);\\s*(?:value|value)\\=(.+)$/;logs-windows.powershell_operational
Adaptation of the script processor "Parses all command invocation detail raw lines [...]" and replace the regex pattern.
Pattern parameterBindingRegex = /name\=(.+);\s*value\=(.+)$/;Pattern parameterBindingRegex = /(?:n|N)ame\\=(.+);\\s*(?:value|value)\\=(.+)$/;Add a failure handler to the KV-Value processor "Split Event 4103 event data fields".
Thank you.