Skip to content

[FEATURE] Support date field comparsion operation pushdown #3710

@penghuo

Description

@penghuo

Is your feature request related to a problem?

  • when plugins.calcite.enabled is enabled, date field comparsion is not pushdown to DSL
###
POST {{baseUrl}}/_plugins/_ppl/_explain
Content-Type: application/x-ndjson

{
    "query": "source=testindex | where timestamp > '2025-05-01 05:40:00'"
}

{
  "calcite": {
    "logical": "LogicalProject(timestamp=[$0])\n  LogicalFilter(condition=[>(TIMESTAMP($0), TIMESTAMP('2025-05-01 05:40:00':VARCHAR))])\n    CalciteLogicalIndexScan(table=[[OpenSearch, testindex]])\n",
    "physical": "EnumerableCalc(expr#0=[{inputs}], expr#1=[TIMESTAMP($t0)], expr#2=['2025-05-01 05:40:00':EXPR_TIMESTAMP VARCHAR], expr#3=[>($t1, $t2)], timestamp=[$t0], $condition=[$t3])\n  CalciteEnumerableIndexScan(table=[[OpenSearch, testindex]], PushDownContext=[[PROJECT->[timestamp]], OpenSearchRequestBuilder(sourceBuilder={\"from\":0,\"timeout\":\"1m\",\"_source\":{\"includes\":[\"timestamp\"],\"excludes\":[]}}, requestedTotalSize=10000, pageSize=null, startFrom=0)])\n"
  }
}

What solution would you like?
date field comparsion should pushdown to DSL

What alternatives have you considered?
n/a

Do you have any additional context?

  • prepare data
###
PUT {{baseUrl}}/testindex
Content-Type: application/x-ndjson

{
  "mappings": {
    "properties": {
      "timestamp" : {
        "type" : "date",
        "format" : "yyyy-MM-dd HH:mm:ss" 
      }
    }
  }
}

###
POST {{baseUrl}}/testindex/_doc
Content-Type: application/x-ndjson

{
  "timestamp" : "2025-05-01 05:49:00"
}

Metadata

Metadata

Assignees

Labels

PPLPiped processing languagebugSomething isn't workingpushdownpushdown related issues

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions