When aws-cloudwatch input is running, user found out there are several logs missing in between timestamp 09:53:56 to 09:54:07.
The previous collection had a 1-minute sleep from 09:53:08 to 09:54:08. Then Filebeat wakes up, starts querying for logs from startTime=09:52:57 to endTime=09:54:08. It reported logs from 09:53:04 to 09:53:55. Then it went to sleep for 1 minute from 09:54:18 to 09:55:18.
Based on the log, FilterLogEvents returned 1 log entry first with a token for pagination, and then 3 log entries, 8 log entries, 5 log entries, and etc. I think there is something wrong with the pagination to cause losing data here. Will need more investigation and needs to try reproduce it locally.
When aws-cloudwatch input is running, user found out there are several logs missing in between timestamp 09:53:56 to 09:54:07.
The previous collection had a 1-minute sleep from 09:53:08 to 09:54:08. Then Filebeat wakes up, starts querying for logs from
startTime=09:52:57toendTime=09:54:08. It reported logs from 09:53:04 to 09:53:55. Then it went to sleep for 1 minute from 09:54:18 to 09:55:18.Based on the log,
FilterLogEventsreturned 1 log entry first with a token for pagination, and then 3 log entries, 8 log entries, 5 log entries, and etc. I think there is something wrong with the pagination to cause losing data here. Will need more investigation and needs to try reproduce it locally.