fluentd: 0.14.17
OS: macOS Sierra 10.12.5
in_tail configuration:
<source>
@type tail
path /user/log/access.log*
tag access.*
format none
refresh_interval 1
pos_file /user/log/pos/access.pos
limit_recently_modified 5
</source>
output plugin configuration:
<match access.**>
@type stdout
@id stdout_output
</match>
Log4j configuration:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/xml/doc-files/log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="ACCESS" class="org.apache.log4j.RollingFileAppender">
<param name="file" value="/user/log/access.log" />
<param name="maxFileSize" value="1KB"/>
<param name="maxBackupIndex" value="1"/>
<param name="immediateFlush" value="true" />
<param name="threshold" value="debug" />
<layout class="org.apache.log4j.EnhancedPatternLayout">
<param name="ConversionPattern" value="%d{yy-MM-dd HH:mm:ss.SSS} %p [%t] %c - %m%n" />
</layout>
</appender>
<logger name="com.example.log.rotation">
<level value="INFO" />
</logger>
<root>
<level value="WARN" />
<appender-ref ref="ACCESS" />
</root>
</log4j:configuration>
Problem description:
I have an application that writes log lines to access.log file. Once file reached of size 1KB it is renamed to access.log.1(inode is not changing at this moment) and new logs are written to newly created access.log file. In my log4j.xml configuration I set up to keep at most 1 backup log file, so second round of rotation occurs following things happening:
access.log.1 is deletedaccess.log is renamed to access.log.1- new
access.log file created and new logs will be written here
Here is fluentd log output:
2017-07-04 10:45:11 +0300 [info]: #0 following tail of /user/log/access.log
<access.log content>
2017-07-04 10:46:31 +0300 [info]: #0 detected rotation of /user/log/access.log; waiting 5 seconds
2017-07-04 10:11:36 +0300 [info]: #0 following tail of /user/log/access.log
2017-07-04 10:11:36 +0300 [info]: #0 following tail of /user/log/access.log.1
<access.log content>
2017-07-04 10:48:01 +0300 [info]: #0 detected rotation of /user/log/access.log.1; waiting 5 seconds
2017-07-04 10:48:01 +0300 [info]: #0 following tail of /user/log/access.log.1
<access.log.1 content -- shouldn't be here>
2017-07-04 10:48:01 +0300 [info]: #0 detected rotation of /user/log/access.log; waiting 5 seconds
2017-07-04 10:48:01 +0300 [info]: #0 following tail of /user/log/access.log
<access.log content>
So after second round of rotation (when oldest file is deleted) for some reason in_tail plugin rereads full content of file (access.log -> access.log.1) even though inode is not changed - just file renaming
pos.file before rotation:
/user/log/access.log 0000000000000413 000000000066c826
/user/log/access.log.1 0000000000000417 000000000066c801
pos.file after rotation
/user/log/access.log 0000000000000000 000000000066c8c2
/user/log/access.log.1 0000000000000413 000000000066c826
fluentd: 0.14.17
OS: macOS Sierra 10.12.5
in_tail configuration:
output plugin configuration:
Log4j configuration:
Problem description:
I have an application that writes log lines to
access.logfile. Once file reached of size 1KB it is renamed toaccess.log.1(inode is not changing at this moment) and new logs are written to newly createdaccess.logfile. In my log4j.xml configuration I set up to keep at most 1 backup log file, so second round of rotation occurs following things happening:access.log.1is deletedaccess.logis renamed toaccess.log.1access.logfile created and new logs will be written hereHere is fluentd log output:
So after second round of rotation (when oldest file is deleted) for some reason in_tail plugin rereads full content of file (
access.log->access.log.1) even though inode is not changed - just file renamingpos.file before rotation:
pos.file after rotation