Cherry-pick #21450 to 7.x: Add new log file reader for filestream input#21470
Cherry-pick #21450 to 7.x: Add new log file reader for filestream input#21470kvch merged 1 commit intoelastic:7.xfrom
Conversation
|
Pinging @elastic/integrations-services (Team:Services) |
💔 Tests FailedExpand to view the summary
Build stats
Test stats 🧪
Test errorsExpand to view the tests failures
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
## What does this PR do? This PR adds the new refactored version of the previous `Log` reader of the `log` input called `logFile`. The differences between the two readers are the following: * `logFile` calls `Stat` only once to avoid too many system calls (`Log` calls `Stat` 3 times after every read) * `logFile` starts separate go routines to check if `close.after_interval` or `close.inactive` has elapsed. `Log` checks `close_inactive` after every `Read`. Thus, if the output is blocked, it cannot stop the reader. * `logFile` does not check if the file has been removed or renamed if `close_removed` or `close_renamed` are enabled. Instead it is checked separately in the prospector, so if the output blocks, the reader can be closed. This prevents Filebeat keeping too many open files if the output is blocked. (The code mentioned is not yet included in any PR.) ## Why is it important? This is the improved version of the previous `log` reader. (cherry picked from commit 1945373)
d979cdf to
ad62274
Compare
💔 Tests FailedExpand to view the summary
Build stats
Test stats 🧪
Test errorsExpand to view the tests failures
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
Cherry-pick of PR #21450 to 7.x branch. Original message:
What does this PR do?
This PR adds the new refactored version of the previous
Logreader of theloginput calledlogFile. The differences between the two readers are the following:logFilecallsStatonly once to avoid too many system calls (LogcallsStat3 times after every read)logFilestarts separate go routines to check ifclose.after_intervalorclose.inactivehas elapsed.Logchecksclose_inactiveafter everyRead. Thus, if the output is blocked, it cannot stop the reader.logFiledoes not check if the file has been removed or renamed ifclose_removedorclose_renamedare enabled. Instead it is checked separately in the prospector, so if the output blocks, the reader can be closed. This prevents Filebeat keeping too many open files if the output is blocked. (The code mentioned is not yet included in any PR.)Why is it important?
This is the improved version of the previous
logreader.Checklist
- [ ] I have made corresponding changes to the documentation- [ ] I have made corresponding change to the default configuration files- [ ] I have added tests that prove my fix is effective or that my feature works- [ ] I have added an entry inCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Related issues
Rebased on #21444