ESQL: Push CIDR_MATCH to Lucene if possible#105061
Conversation
9321ce7 to
044fe41
Compare
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
|
Hi @alex-spies, I've created a changelog YAML for you. |
costin
left a comment
There was a problem hiding this comment.
Almost there.
A follow-up is to optimize different CIDR_Match functions on the same field into one - so
WHERE CIDR_MATCH(field, "123.0.0.0/24") OR CIDR_MATCH(field, "123.1.2.4.5/24") OR CIDR_MATCH(another_field, "172.0.0.1/16") AND CIDR_MATCH(extra_field, "5.4.2.1/8")
gets optimized into
WHERE CIDR_MATCH(field, "123.0.0.0/24", "123.1.2.4.5/24") OR CIDR_MATCH(another_field, "172.0.0.1/16") AND CIDR_MATCH(extra_field, "5.4.2.1/8")
...ck/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/EsqlTranslatorHandler.java
Outdated
Show resolved
Hide resolved
| * "cidr_match(ip, \"127.0.0.0/24\")@1:19"}}][_doc{f}#21], limit[500], sort[] estimatedRowSize[389] | ||
| */ | ||
| public void testCidrMatchPushdownFilter() { | ||
| var allTypeMappingAnalyzer = makeAnalyzer("mapping-all-types.json", new EnrichResolution()); |
There was a problem hiding this comment.
Why not use the default analyzer?
There was a problem hiding this comment.
The default mapping is missing a field of type ip. I'll use mapping-ip.json, though, to be more specific.
| int cidrBlockCount = randomIntBetween(1, 10); | ||
| ArrayList<String> cidrBlocks = new ArrayList<>(); | ||
| for (int i = 0; i < cidrBlockCount; i++) { | ||
| cidrBlocks.add("127.0." + i + ".0/24"); |
There was a problem hiding this comment.
See whether there's another randomization function to have a larger spectrum of IPs including IPv6.
|
Thanks for reviews, Bogdan and Costin! @costin, I looked into the optimization where we combine disjunctions of |
|
This is a bug so make sure to backport it in 8.12 - use the automatic way through auto-backport-and-merge label + v8.12.2. |
💔 Backport failed
You can use sqren/backport to manually backport by running |
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
(cherry picked from commit b5f4c5e) # Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/EsqlTranslatorHandler.java # x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/LocalPhysicalPlanOptimizerTests.java
Fix #105042