Today when the log input does not have permission to open a file, it logs an error.
{"log.level":"error","@timestamp":"2024-05-06T06:52:38.843Z","message":"Harvester could not be started on new file: /var/log/system.log, Err: error setting up harvester: Harvester setup failed. Unexpected file opening error: Failed opening /var/log/system.log: open /var/log/system.log: permission denied","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"log-default","type":"log"},"log":{"source":"log-default"},"finished":false,"ecs.version":"1.6.0","log.origin":{"file.line":561,"file.name":"log/input.go","function":"github.com/elastic/beats/v7/filebeat/input/log.(*Input).scan"},"service.name":"filebeat","state_id":"native::9418910-16777229","log.logger":"input","input_id":"476b19f2-632a-4647-8700-78e244554690","source_file":"/var/log/system.log","os_id":"9418910-16777229","ecs.version":"1.6.0"}When the log input is run as part of the system integration installed in an unprivileged agent, this can lead to missing data with no obvious signal to the user in the output of elastic-agent status or in the Fleet UI. This is a poor user experience as users will install the system integration and data will be missing with no obvious root cause.
Modify the log input to set itself as Degraded using the Elastic Agent control protocol when it encounters a permissions error attempting to read a file. Support for reporting status using the control protocol was added in #39209 using the CEL input as a reference implementation.
--unprivilegedflag. elastic-agent#4112system.syslognot available for mac agent installed with unprivileged flag. elastic-agent#4675Today when the log input does not have permission to open a file, it logs an error.
{"log.level":"error","@timestamp":"2024-05-06T06:52:38.843Z","message":"Harvester could not be started on new file: /var/log/system.log, Err: error setting up harvester: Harvester setup failed. Unexpected file opening error: Failed opening /var/log/system.log: open /var/log/system.log: permission denied","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"log-default","type":"log"},"log":{"source":"log-default"},"finished":false,"ecs.version":"1.6.0","log.origin":{"file.line":561,"file.name":"log/input.go","function":"github.com/elastic/beats/v7/filebeat/input/log.(*Input).scan"},"service.name":"filebeat","state_id":"native::9418910-16777229","log.logger":"input","input_id":"476b19f2-632a-4647-8700-78e244554690","source_file":"/var/log/system.log","os_id":"9418910-16777229","ecs.version":"1.6.0"}When the log input is run as part of the system integration installed in an unprivileged agent, this can lead to missing data with no obvious signal to the user in the output of
elastic-agent statusor in the Fleet UI. This is a poor user experience as users will install the system integration and data will be missing with no obvious root cause.Modify the log input to set itself as Degraded using the Elastic Agent control protocol when it encounters a permissions error attempting to read a file. Support for reporting status using the control protocol was added in #39209 using the CEL input as a reference implementation.