Skip to content

inputs.tail does not support csv_skip_rows>0 and csv_header_row_count>=1 #9903

@etycomputer

Description

@etycomputer

I have noticed that the inputs.tail does not allow support for csv_skip_rows>0 and csv_header_row_count>=1.
This is an issue that was raised and fixed for inputs.directory_monitor.

At this stage, I could like help from the community to indicate if it would be helpful to have the same fix apply for inputs.tail.
My reservation not to make this change is that we have other input types like inputs.file that could be used as an alternative. Also, the inputs.tail is commonly designed for tailing log files. In the case of telling a CSV file, the headers should already be provided.

Relevant telegraf.conf:

[[inputs.tail]]
  files = ["/path/to/metrics.csv"]
  from_beginning = true
  file_tag = "filename"
  data_format = "csv"
  csv_header_row_count = 1
  csv_column_names = []
  csv_column_types = []
  csv_skip_rows = 2
  csv_skip_columns = 0
  csv_delimiter = ","
  csv_comment = ""
  csv_trim_space = false
  csv_tag_columns = []
  csv_measurement_column = ""
  csv_timestamp_column = ""
  csv_timestamp_format = ""
  csv_timezone = ""
  csv_skip_values = []

System info:

Telegraf 1.20.0

  1. Create a metrics.csv file with the following information in the /path/to/ folder.
version=1.0
file created: 2021-10-08T12:34:18+10:00
timestamp,type,name,status
2020-11-23T08:19:27+10:00,Reader,R002,1
2020-11-04T13:23:04+10:00,Reader,R031,0
2020-11-04T13:29:47+10:00,Coordinator,C001,1

Expected behaviour:

The expected behaviour would be that the parser would skip the first two lines and parse the rest of the file as normal.

Actual behaviour:

The first two metadata lines are not ignored and it results in an error while reading the file.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/tailbugunexpected problem or unintended behavior

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions