-
Notifications
You must be signed in to change notification settings - Fork 5.8k
inputs.tail does not support csv_skip_rows>0 and csv_header_row_count>=1 #9903
Description
I have noticed that the inputs.tail does not allow support for csv_skip_rows>0 and csv_header_row_count>=1.
This is an issue that was raised and fixed for inputs.directory_monitor.
At this stage, I could like help from the community to indicate if it would be helpful to have the same fix apply for inputs.tail.
My reservation not to make this change is that we have other input types like inputs.file that could be used as an alternative. Also, the inputs.tail is commonly designed for tailing log files. In the case of telling a CSV file, the headers should already be provided.
Relevant telegraf.conf:
[[inputs.tail]]
files = ["/path/to/metrics.csv"]
from_beginning = true
file_tag = "filename"
data_format = "csv"
csv_header_row_count = 1
csv_column_names = []
csv_column_types = []
csv_skip_rows = 2
csv_skip_columns = 0
csv_delimiter = ","
csv_comment = ""
csv_trim_space = false
csv_tag_columns = []
csv_measurement_column = ""
csv_timestamp_column = ""
csv_timestamp_format = ""
csv_timezone = ""
csv_skip_values = []System info:
Telegraf 1.20.0
- Create a
metrics.csvfile with the following information in the/path/to/folder.
version=1.0
file created: 2021-10-08T12:34:18+10:00
timestamp,type,name,status
2020-11-23T08:19:27+10:00,Reader,R002,1
2020-11-04T13:23:04+10:00,Reader,R031,0
2020-11-04T13:29:47+10:00,Coordinator,C001,1
Expected behaviour:
The expected behaviour would be that the parser would skip the first two lines and parse the rest of the file as normal.
Actual behaviour:
The first two metadata lines are not ignored and it results in an error while reading the file.