[Ingest Pipeline] Date processor ignores timezone offset XXX #63458
Description
Currently tested in Dev Console on 7.9.2 running on ESS.
Steps to reproduce:
It seems that timezone formats that work in pure Java is being ignored when using the date processor in ingest pipelines. Example:
POST /_ingest/pipeline/_simulate
{
"pipeline": {
"description": "_description",
"processors": [
{
"date": {
"field": "message",
"target_field": "@timestamp",
"formats": ["EEE MMM d HH:mm:ss YYYY XXX"],
"timezone": "UTC"
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "id",
"_source": {
"message": """Wed Oct 21 11:19:17 2020 -04:00"""
}
}
]
}
This will output:
{
"docs" : [
{
"doc" : {
"_index" : "index",
"_type" : "_doc",
"_id" : "id",
"_source" : {
"@timestamp" : "2020-10-21T11:19:17.000Z",
"message" : "Wed Oct 21 11:19:17 2020 -04:00"
},
"_ingest" : {
"timestamp" : "2020-10-08T09:21:11.48095Z"
}
}
}
]
}
As you can see even though the offset is -04:00, and date processors defaults to UTC; it does not calculate any offset?
The same behavior is seen with other offset examples like Z, z, VV etc.
The same format in Java will work (sample code provided by @marc-gr ):
import java.util.*;
import java.util.Date;
import java.text.SimpleDateFormat;
public class Main {
public static void main(String[] args) throws Exception {
SimpleDateFormat format = new SimpleDateFormat("E MMM d HH:mm:ss YYYY XXX");
String dateString = format.format( new Date() );
Date date = format.parse ( "Wed Oct 21 11:19:17 2020 -04:00" );
System.out.println(date);
}
}
Anyone know if this is intended behavior?