Skip to content

Microsoft Exchange Message Trace httpjson template broken cursor usage #5180

@colin-stubbs

Description

@colin-stubbs

The httpjson input template irrationally/unnecessarily attempts to use parseDuration with no arguments, which filebeat does not accept.

e.g. template snippet,

cursor:
  last_execution_datetime:
    value: '[[(formatDate (now (parseDuration)) "RFC3339")]]'

Error,

Screenshot 2023-02-04 at 6 07 50 pm

This means that the request.transforms reference to the cursor will fail, and the default value which applies the parseDuration minus {{initial_interval}} gets used.

Hence the integration continually requests historic logs and never really updates the cursor to only request logs from the last time it ran. e.g.

request.transforms:
  - set:
      target: url.params.$filter
      value: StartDate eq datetime'[[.cursor.last_execution_datetime]]' and EndDate eq datetime'[[(formatDate (now) "RFC3339")]]'
      default: StartDate eq datetime'[[(formatDate (now (parseDuration "-{{initial_interval}}")) "RFC3339")]]' and EndDate eq datetime'[[(formatDate (now) "RFC3339")]]'

PR inbound shortly.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions