Skip to content

Ingest processor ISO8601 format parsing is not ISO compatible #38345

@spinscale

Description

@spinscale

Elasticsearch version (bin/elasticsearch --version): master branch after java-time switch

Description of the problem including expected versus actual behavior: With the switch to java-time the ISO8601 format is no longer able to parse dates that include a comma, where as the joda time implementation was able to do so.

Steps to reproduce:

    public void testParseISO8601Commas() {
        assertThat(DateFormat.Iso8601.getFunction(null, ZoneOffset.UTC, null).apply("2001-01-01T00:00:00,123").toInstant().toEpochMilli(),
                equalTo(978336000123L));
    }

the ISO 8601 spec is a bit complex (allowing you to cut the date, and use fractionals everywhere, but most importantly those fractionals may be using either a dot or a comma), I think the easiest solution might be to just add another parser that supports commas at the millisecond stage, as this is a format we are also using in our own logs, it might be worth to add.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions