Skip to content

ESQL: unmapped_fields="load" and "nullify" do not keep column ordering stable #141923

@alex-spies

Description

@alex-spies

A small bug? (maybe?) in SET unmapped_fields="..." where an unmapped or missing field can show up in a different position compared to when the field is mapped.

This could affect use cases where the query is used for a dashboard and users would expect the visualization not change when pointing at an index that happens not to have a given field.

In any case, it breaks the assumption that the output columns will be the same even if a field is missing or unmapped.

Reproducer using the csv tests

Un-map the name field from the apps index by adjusting mapping-apps.json:

{
    "dynamic": false,
    "properties" : {
        "id" : {
            "type" : "integer"
        },
        "version" : {
            "type" : "version"
        }
    }
}

Then run

 ./gradlew ":x-pack:plugin:esql:qa:server:single-node:javaRestTest" --tests "org.elasticsearch.xpack.esql.qa.single_node.EsqlSpecIT" -Dtests.method="test {csv-spec:union_types.MultiIndexInlinestatsOfMultiTypedField}"

This runs

FROM apps, apps_short
| INLINE STATS s = sum(id::integer)
| SORT name, version
| LIMIT 5

and returns

Different column name [1]
Expected :name
Actual   :version

The name and version columns are now swapped.

It seems like the columns in our EsRelation are actually sorted, maybe already in the field caps response. It'd be nice to keep that sorting intact.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions