Skip to content

[windows] [powershell] ingest-pipeline failes on german logs #9672

@saiiman

Description

@saiiman

Describe the bug:
The ingest pipelines windows.powershell* cannot parse logs with german content correctly.

Pipeline version:
1.44.5

Steps to reproduce:
Ingest logs with a message in german language. Example:

CommandInvocation(Select-Object): "Select-Object"
ParameterBinding(Select-Object): Name="Property"; Wert="Date"
ParameterBinding(Select-Object): Name="InputObject"; Wert="System.__ComObject"

Kontext:
        Schweregrad: Informational
        Hostname: Default Host
        Hostversion: 5.1.17763.XXXX
        Host-ID: 8c28d572-XXX-XXXX-XXXX-141be1a8b83d
        Hostanwendung = C:\Program Files (x86)\[...]
        Modulversion: 5.1.17763.XXXX
        Runspace-ID: fe6a6be4-XXXX-XXXX-XXXX-aff0870ddccc
        Pipeline-ID: 5
        Befehlsname: Select-Object
        Befehlstyp: Cmdlet
        Skriptname: 
        Befehlspfad: 
        Sequenznummer: 50
        Benutzer: WORKGROUP\SYSTEM
        Verbundener Benutzer = 
        Shell-ID: Microsoft.PowerShell

Benutzerdaten:

Current behavior:

  • The ingest pipeline failes on the first KV-processor (on event.code 4103) with field [winlog.event_data.ContextInfo] does not contain value_split [=].
  • The ingest pipeline does not parse the message field correctly.

Expected behavior:

  • Correct parsing of all elements of the event, regardless of the system language.

Suggestion:
To solve the problem in the short term, I suggest the following adjustments to the ingest pipeliens. in the future, however, it might be better to provide dedicated pipeliens for different system languages.

logs-windows.powershell

  • Adaptation of the script processor "Parses all command invocation detail raw lines [...]" and replace the regex pattern.
    • Original: Pattern parameterBindingRegex = /name\=(.+);\s*value\=(.+)$/;
    • Adaptation: Pattern parameterBindingRegex = /(?:n|N)ame\\=(.+);\\s*(?:value|value)\\=(.+)$/;

logs-windows.powershell_operational

  • Adaptation of the script processor "Parses all command invocation detail raw lines [...]" and replace the regex pattern.

    • Original: Pattern parameterBindingRegex = /name\=(.+);\s*value\=(.+)$/;
    • Adaptation: Pattern parameterBindingRegex = /(?:n|N)ame\\=(.+);\\s*(?:value|value)\\=(.+)$/;
  • Add a failure handler to the KV-Value processor "Split Event 4103 event data fields".

    • Additional KV-Value processor for splitting the German-language logs
"kv": {
      "field": "winlog.event_data.ContextInfo",
      "field_split": "\\n",
      "value_split": "(:|=)",
      "target_field": "winlog.event_data",
      "trim_key": " \"",
      "trim_value": " \"",
      "if": "ctx?.winlog?.event_id == \"4103\"",
      "ignore_failure": true
}
  • Rename processors to replace the German names of the keys into English
"rename": {
      "field": "winlog.event_data.Schweregrad",
      "target_field": "winlog.event_data.Severity",
      "ignore_missing": true,
      "if": "ctx?.winlog?.event_id == \"4103\"",
      "ignore_failure": true
}
[...]
rename: Hostname, Hostversion, Host-ID, Hostanwendung, ...

Thank you.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions