Skip to content

awscloudwatch input drops data #38918

@faec

Description

@faec

The awscloudwatch input can skip data in its target log groups, with severity depending on configuration and log size. This seems to apply to all platforms and all versions at least since 8.0.

Easy reproduction:

  • Set number_of_workers to 1 (the default)
  • Set start_position to beginning (the default)
  • Set log_group_name_prefix to a value matching 2 or more log groups

The first matching log group will ingest data starting from the beginning, but all other log groups will only include data from after ingestion began.

This loss continues during ingestion: events from any time span will only include data from at most one log group at a time.

More finicky reproduction with a single log group:

  • Set number_of_workers to 1 (the default)
  • Set start_position to beginning (the default)
  • Target a single log group with a significant amount of past data (enough to require significantly longer than the scan_frequency to ingest -- optionally set scan_frequency to 1s to make this easier)

Data added to the log group between the start of ingestion and the completion of the first scan will be skipped.

Metadata

Metadata

Assignees

Labels

Team:Elastic-AgentLabel for the Agent teamTeam:Obs-InfraObsLabel for the Observability Infrastructure Monitoring teambug

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions