-
Notifications
You must be signed in to change notification settings - Fork 190
[FEATURE] Support comparing IP values with Calcite #3776
Description
Is your feature request related to a problem?
Current comparison operators are mapped to Calcite's built-in comparisons, but the IP types are implemented as user-defined types (UDT). As a result, comparison operators does not support comparison between IP types.
This is not a problem when pushdown is enabled, since such comparison will be pushed down and can be properly executed. For example, source=weblogs | where host > '1.2.3.4' | fields host will be executed in the following way (weblogs dataset):
{
"calcite": {
"logical": "LogicalProject(host=[$0])\n LogicalFilter(condition=[>($0, '1.2.3.4':VARCHAR)])\n CalciteLogicalIndexScan(table=[[OpenSearch, weblogs]])\n",
"physical": "CalciteEnumerableIndexScan(table=[[OpenSearch, weblogs]], PushDownContext=[[PROJECT->[host], FILTER->>($0, '1.2.3.4':VARCHAR)], OpenSearchRequestBuilder(sourceBuilder={\"from\":0,\"timeout\":\"1m\",\"query\":{\"range\":{\"host\":{\"from\":\"1.2.3.4\",\"to\":null,\"include_lower\":false,\"include_upper\":true,\"boost\":1.0}}},\"_source\":{\"includes\":[\"host\"],\"excludes\":[]},\"sort\":[{\"_doc\":{\"order\":\"asc\"}}]}, requestedTotalSize=2147483647, pageSize=null, startFrom=0)])\n"
}
}However, when pushdown is disabled, it raises the following error:
{
"error": {
"reason": "Invalid Query",
"details": "No assign rules for OTHER defined",
"type": "CalciteUnsupportedException"
},
"status": 400
}
The mysterious error arises from the process of coercing operand types of the built-in greater than operator of Calcite, since it does not handle UDTs properly.
What solution would you like?
Implement user-defined function for IP comparison operators.
What alternatives have you considered?
Convert IP UDT to a Calcite's built-in type, so that the comparison can be handled properly. However, there isn't a direct correspondence of IP.
I'm also thinking of converting it to a tuple of numbers, maybe Calcite comparison operators can recognize them.
Do you have any additional context?
#3229