Context

The Azure Logs v2 routing pipeline infers the dataset by inspecting the category field in the log event.

If the category is unknown, it routes the log event to the azure.platformlogs dataset by default.

However, users may want to define special cases for this routing—for example, sending unrecognized log categories to a specific data stream.

Goal

Allow users to override log event routing via custom pipelines.

The default pipeline must extract the log category, and make it available as field for the custom pipeline.

For example, if the default pipeline extract routing.category, we can use a reroute processor to route the category to a custom dataset:

PUT _ingest/pipeline/logs-azure.events@custom
{
  "processors": [
    {
      "reroute": {
        "if": "ctx?.routing?.category == \"Whatever\"",
        "dataset": "azure.whatever"
      }
    }
  ]
}

Impact

Implementing routing overrides will allow users to add exceptions to standard routing rules, providing greater flexibility for non-standard log formats.