Skip to content

Elastic Agent shouldn't log httpjson pagination completion as an Agent Error #38291

@agmic

Description

@agmic

I am not sure if this should be raised against Elastic Agent or the integration. Raising it here, but please move as necessary.

The Cisco Secure Endpoint integration (v2.24.2) collects event data via httpjson. The integration throws an error when it hits the last page, and this gets logged in the Elastic Agent error log with something similar to:
error processing response: template: :1:17: executing "" at <.last_response.body.metadata.links.next>: map has no entry for key "next"
This fills up the Elastic Agent dashboard with agent errors, as it is raised every time the Secure Endpoint data is polled (every 2 minutes/default) and the noise makes it more difficult to find actual errors.

I believe that this is occurring because fail_on_template_error is set to true.

The fail condition prompting this is at
integrations/packages/cisco_secure_endpoint/data_stream/event/agent/stream/httpjson.yml.hbs at main · elastic/integrations (github.com) line 36-40

response.pagination:
- set:
    target: url.value
    value: '[[ .last_response.body.metadata.links.next ]]'
    fail_on_template_error: true

I'm not sure how this should be fixed, because AIUI fail_on_template_error: true is the recommended configuration for pagination, but raising an error should maybe be reconsidered?

For confirmed bugs, please report:

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions