Skip to content

Remove event.original removal processors from ingest pipelines #10072

@Alphayeeeet

Description

@Alphayeeeet

It seems that most of the checked integrations have a removal processor for event.original (except if it contains the preserve tag) in their pipelines.
In the other way, the .fleet-final-pipeline also has this processor and runs after all integration pipelines (incl. the @Custom pipelines) have finished. If the integration has this removal, the @Custom pipeline cannot parse the original event anymore as it is not present in the event anymore.

I would suggest removing the removal processors from all integration ingest pipelines and forward the removal functionality into the fleet-final-pipeline where it belongs. In that case @Custom pipelines may parse the original event and can extract additional data or correct malformed parsing in the generic integrations (e.g. catalina.out in the Apache Tomcat integration):

Metadata

Metadata

Assignees

No one assigned

    Labels

    Integration:AllApplies to all integrations [Integration not found in source]enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions