Skip to content

csv-data omitting csv_comment with input.tail #5338

@joriws

Description

@joriws

Relevant telegraf.conf:

Slightly modified from #5334 by adding csv_comment="#" and changing test input with starting line "#version=13.0". Your #5336 has the example code to produce.

System info:

telegraf --version

Telegraf unknown (git: master efbc83c)

Steps to reproduce:

I experimented more with input.tail and tried csv_comment:

  • added configuration to existing
    csv_comment = "#"

  • changed input file
    #version=13.0
    date,instance,Incoming Requests,Outgoing Requests,Incoming Answers 2xxx,Outgoing Answers 2xxx,Incoming Answers UTD,Outgoing Answers UTD,Incoming Answers Redirect,Outgoing Answers Redirec...

Expected behavior:

First line to be skipped because it is CSV comment, 2nd line used as headers and from 3rd line onwards all data

Actual behavior:

2019-01-24T07:25:35Z D! [agent] Starting service inputs
2019-01-24T07:25:35Z D! [inputs.tail] tail added for file: testlog
2019-01-24T07:25:35Z E! [inputs.tail]: Error in plugin: E! Malformed log line in testlog: [#version=13.0], Error: EOF
2019-01-24T07:25:35Z E! [inputs.tail]: Error in plugin: E! Malformed log line in testlog: [date,instance,In coming Requests,Outgoing Requests,Incoming Answers 2xxx,Outgoing Answers 2xxx,Incoming Answers UTD,Outgoing Answers UTD,Incoming Answers Redirect,Outgoing Answers Redirect,Incoming Answers Other,Outgoing Answers Other,Retransmit Requests,Rejected Requests Filtering,Rejected Requests In-gress Filtering,Rejected Requests Other,Timeout Requests,Discarded Answers], Error: [parsers.csv] data columns must be specified

Additional info:

I converted input.tail to input.file and ..

csv_comment works as expected - outputs CSV data starting from line 3
csv_skip_rows works different (ref #5336), not more parsing error on headers but...

2019-01-24T07:33:35Z D! [agent] Starting service inputs
2019-01-24T07:33:40Z E! [inputs.file]: Error in plugin: timestamp column: date could not be found
2019-01-24T07:33:50Z E! [inputs.file]: Error in plugin: timestamp column: date could not be found

Metadata

Metadata

Assignees

Labels

area/tailbugunexpected problem or unintended behavior

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions