-
Notifications
You must be signed in to change notification settings - Fork 5k
Add timezone metadata #2887
Description
There are a few log formats that I am parsing with Filebeat + Logstash in which the logs contain a timestamp but no timezone. For example, here is a log event from nginx error logs:
2016/10/30 19:40:06 [error] 10522#10522: *42075309 broken header: "" while reading PROXY protocol, client: x.x.x.x, server: 0.0.0.0:443
In all of the cases I have seen, the timestamp is logged in the local timezone (that is, the timezone of the host on which Filebeat is running). The problem, however, is that Logstash doesn't have enough information from which the host's timezone can be inferred. Consequently, Logstash ingests the log event using the local timezone for the host that is running Logstash (which may or may not be the same as the timezone of the host which is running Filebeat).
Obviously, it would be great if all logs could contain enough information such that the timestamp is unambigious, but this is overly ambitious. Instead, it would be great if Filebeat could add a [beat][timezone] field to all log events processed. This field could then be used in Logstash with the date filter.