Version: 5.2.2
OS: CentOS 7
Due to an old OS on another machine, under which filebeat doesn't run, i rsync the logs to my elastic server on which a filebeat reads them.
Through this setup i found a bug in the clean_removed Routine, which creates a situation, in combination with the inode reuse under linux, in which there are two entrys in the registry file for the same source path with different inodes.
{ "source":"/root/copied_logs/<path to the same log>", "offset":5243941, "FileStateOS":{ "inode":1433391, "device":2049 }, "timestamp":"2017-03-22T12:59:31.413988399+01:00", "ttl":-1 }, { "source":"/root/copied_logs/<path to the same log>", "offset":5246116, "FileStateOS":{ "inode":1433400, "device":2049 }, "timestamp":"2017-03-22T13:02:32.749620477+01:00", "ttl":-1 }
The second entry is the one that doesn't exist anymore and when the new log file is rsynced it gets the inode from this entry and filebeat trys to continue reading it.
As a solution for this Bug:
The cleanup method should not only check if it can get os.Stat for the source but also if this stat is the same as the one in the registry entry which is searched. Otherwise it should be deleted.
|
_, err := os.Stat(state.Source) |
Version: 5.2.2
OS: CentOS 7
Due to an old OS on another machine, under which filebeat doesn't run, i rsync the logs to my elastic server on which a filebeat reads them.
Through this setup i found a bug in the clean_removed Routine, which creates a situation, in combination with the inode reuse under linux, in which there are two entrys in the registry file for the same source path with different inodes.
{ "source":"/root/copied_logs/<path to the same log>", "offset":5243941, "FileStateOS":{ "inode":1433391, "device":2049 }, "timestamp":"2017-03-22T12:59:31.413988399+01:00", "ttl":-1 }, { "source":"/root/copied_logs/<path to the same log>", "offset":5246116, "FileStateOS":{ "inode":1433400, "device":2049 }, "timestamp":"2017-03-22T13:02:32.749620477+01:00", "ttl":-1 }The second entry is the one that doesn't exist anymore and when the new log file is rsynced it gets the inode from this entry and filebeat trys to continue reading it.
As a solution for this Bug:
The cleanup method should not only check if it can get os.Stat for the source but also if this stat is the same as the one in the registry entry which is searched. Otherwise it should be deleted.
beats/filebeat/prospector/prospector_log.go
Line 94 in 4527162