Skip to content

ES|QL: Return columns for unsupported field types in FORK#128508

Merged
ioanatia merged 1 commit intoelastic:mainfrom
ioanatia:fork_unsupported_attributes
May 28, 2025
Merged

ES|QL: Return columns for unsupported field types in FORK#128508
ioanatia merged 1 commit intoelastic:mainfrom
ioanatia:fork_unsupported_attributes

Conversation

@ioanatia
Copy link
Copy Markdown
Member

@ioanatia ioanatia commented May 27, 2025 •
edited
Loading

When querying indices that have field types that are unsupported in ES|QL (e.g. ip_range, sparse_vector etc) with FORK, the query returns an error.

Steps to replicate:

Create an index with an unsupported field type

PUT my-index
{
    "mappings": {
        "properties": {
            "abc": {
                "type": "date_range"
            },
            "x": {
                "type": "keyword"
            }
        }
    }
}

Issue a query using FORK

POST /_query
{
    "query": """
        FROM my-index
        | FORK (limit 1) (limit 1)
        """
}

See error response:

{
  "error": {
    "root_cause": [
      {
        "type": "esql_illegal_argument_exception",
        "reason": "unexpected projection: !abc"
      }
    ],
    "type": "esql_illegal_argument_exception",
    "reason": "unexpected projection: !abc"
  },
  "status": 500
}

Stacktrace:

[2025-05-27T14:19:56,091][WARN ][r.suppressed             ] [runTask-0] path: /_query, params: {pretty=true}, status: 500 org.elasticsearch.xpack.esql.EsqlIllegalArgumentException: unexpected projection: !abc
        at org.elasticsearch.xpack.esql.analysis.Analyzer$ResolveRefs.resolveKeep(Analyzer.java:1063)
        at org.elasticsearch.xpack.esql.analysis.Analyzer$ResolveRefs.rule(Analyzer.java:502)
        at org.elasticsearch.xpack.esql.analysis.Analyzer$ResolveRefs.rule(Analyzer.java:470)
        at org.elasticsearch.xpack.esql.analysis.AnalyzerRules$ParameterizedAnalyzerRule.lambda$apply$0(AnalyzerRules.java:49)
        at org.elasticsearch.xpack.esql.core.tree.Node.lambda$transformUp$10(Node.java:209)
        at org.elasticsearch.xpack.esql.core.tree.Node.transformUp(Node.java:204)
        at org.elasticsearch.xpack.esql.core.tree.Node.lambda$transformUp$9(Node.java:202)
        at org.elasticsearch.xpack.esql.core.tree.Node.transformChildren(Node.java:227)
        at org.elasticsearch.xpack.esql.core.tree.Node.transformUp(Node.java:202)
        at org.elasticsearch.xpack.esql.core.tree.Node.transformUp(Node.java:209)
        at org.elasticsearch.xpack.esql.analysis.AnalyzerRules$ParameterizedAnalyzerRule.apply(AnalyzerRules.java:49)
        at org.elasticsearch.xpack.esql.analysis.AnalyzerRules$ParameterizedAnalyzerRule.apply(AnalyzerRules.java:41)
        at org.elasticsearch.xpack.esql.rule.ParameterizedRuleExecutor.lambda$transform$0(ParameterizedRuleExecutor.java:29)
        at org.elasticsearch.xpack.esql.rule.RuleExecutor$Transformation.<init>(RuleExecutor.java:92)
        at org.elasticsearch.xpack.esql.rule.RuleExecutor.executeWithInfo(RuleExecutor.java:171)
        at org.elasticsearch.xpack.esql.rule.RuleExecutor.execute(RuleExecutor.java:140)
        at org.elasticsearch.xpack.esql.analysis.Analyzer.analyze(Analyzer.java:192)
        at org.elasticsearch.xpack.esql.session.EsqlSession.lambda$analyzedPlan$7(EsqlSession.java:325)
        at org.elasticsearch.xpack.esql.session.EsqlSession.analyzeAndMaybeRetry(EsqlSession.java:524)
        at org.elasticsearch.xpack.esql.session.EsqlSession.lambda$analyzedPlan$15(EsqlSession.java:367)

Cause

The problem is that when we resolve FORK during analysis, we add KEEP plans to align the output of FORK branches.
If the output contains UnsupportedAttribute, these are not correctly handled when we resolve the Keep projections (in Analyzer.resolveKeep).

@ioanatia ioanatia added >non-issue :Analytics/ES|QL AKA ESQL v9.1.0 Team:Search - Relevance The Search organization Search Relevance team labels May 27, 2025
@ioanatia ioanatia marked this pull request as ready for review May 27, 2025 14:06
@elasticsearchmachine elasticsearchmachine added Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) and removed Team:Search - Relevance The Search organization Search Relevance team labels May 27, 2025
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

elasticsearchmachine commented May 27, 2025

Pinging @elastic/es-analytical-engine (Team:Analytics)

@ioanatia ioanatia merged commit 392777d into elastic:main May 28, 2025
18 checks passed
@ioanatia ioanatia mentioned this pull request May 28, 2025
Closed
23 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@carlosdelest carlosdelest carlosdelest approved these changes

@dnhatn dnhatn dnhatn approved these changes

@ChrisHegarty ChrisHegarty Awaiting requested review from ChrisHegarty

@bpintea bpintea Awaiting requested review from bpintea

Assignees

No one assigned

Labels

:Analytics/ES|QL AKA ESQL >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.1.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ioanatia @elasticsearchmachine @carlosdelest @dnhatn