Relevant telegraf.conf:
[[outputs.influxdb]]
urls = ["http://localhost:8086"]
database = "AAAAAAAAAAAAA"
retention_policy = ""
write_consistency = "any"
timeout = "5s"
username = "DDDDDDDDDDD"
password = "FFFFFFFFFFFFF"
user_agent = "telegraf"
[[inputs.kafka_consumer]]
brokers = ["XXXXX:9092","XXXXX:9092","XXXXX:9092"]
topics = ["elb_logs"]
consumer_group = "telegraf_consumer_elb_0"
offset = "oldest"
data_format ="json"
name_override = "elb"
max_message_len = 0
json_time_format = "RFC3339" // or json_time_format = "2018-10-26T13:30:33Z"
tag_keys = ["Elb"]
json_string_fields = [
"Elb",
"ClientPort",
"BackendPort",
"Request_processing_time",
"Backend_processing_time",
"Response_processing_time",
"Elb_status_code",
"Backend_status_code",
"Received_bytes",
"Sent_bytes",
"Request",
"User_agent",
"Ssl_cipher",
"Ssl_protocol"
]
json_time_key = "Timestamp"
System info:
Telegraf 1.8.1
Linux ip-172-40-16-175 4.15.0-1021-aws #21-Ubuntu SMP Tue Aug 28 10:23:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Ubuntu 16.04
Expected behavior:
I add to kafka json message , with golang
These timsteamp are formated like that:
timestring.Format(time.RFC3339) I use time library in golang
Actual behavior:
for json_time_format = "2018-10-26T13:30:33Z"
I got :
message: {"Timestamp":"2018-10-26T14:00:52Z","Elb":"blabla-prod","ClientPort":"213.152.17.19:54314","BackendPort":"-","Request_processing_time":"-1","Backend_processing_time":"-1","Response_processing_time":"-1","Elb_status_code":"503","Backend_status_code":"0","Received_bytes":"0","Sent_bytes":"0","Request":"GET https://YYYYY:443/ HTTP/1.1","User_agent":"curl/7.47.0","Ssl_cipher":"ECDHE-RSA-AES128-GCM-SHA256","Ssl_protocol":"TLSv1.2"}
error: parsing time "2018-10-26T14:00:52Z": month out of range
for json_time_format = "RFC3339"
error: parsing time "2018-10-26T13:30:33Z" as "RFC3339": cannot parse "2018-10-26T13:30:33Z" as "RFC"
Additional info:
telegraf parser json drove me nuts
Relevant telegraf.conf:
System info:
Telegraf 1.8.1
Linux ip-172-40-16-175 4.15.0-1021-aws #21-Ubuntu SMP Tue Aug 28 10:23:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Ubuntu 16.04
Expected behavior:
I add to kafka json message , with golang
These timsteamp are formated like that:
timestring.Format(time.RFC3339)I use time library in golangActual behavior:
for
json_time_format = "2018-10-26T13:30:33Z"I got :
for
json_time_format = "RFC3339"error: parsing time "2018-10-26T13:30:33Z" as "RFC3339": cannot parse "2018-10-26T13:30:33Z" as "RFC"Additional info:
telegraf parser json drove me nuts