ESQL: Add parsing for a LOOKUP command#109040
ESQL: Add parsing for a LOOKUP command#109040elasticsearchmachine merged 7 commits intoelastic:mainfrom
Conversation
This command will serve as a sort of "inline" enrich. This commit itself is mostly antlr generated code and paranoid tests that the new `LOOKUP` keyword doesn't clash with any variables named `lookup`.
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
| x-pack/plugin/esql/src/main/antlr/*.tokens linguist-generated=true | ||
| x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/*.interp linguist-generated=true | ||
| x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseLexer*.java linguist-generated=true | ||
| x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParser*.java linguist-generated=true |
There was a problem hiding this comment.
yeah! I saw kibana doing it for something so I stole their trick.
There was a problem hiding this comment.
ha! interesting. what does the IDE do, if anything, with this ?
There was a problem hiding this comment.
So far as I can tell, nothing.
|
BWC failure is real - LOOKUP can't be used there. Easy enough. |
|
|
||
|
|
||
| // | ||
| // Make sure that the new LOOKUP syntax doesn't clash with any existing things |
There was a problem hiding this comment.
++ to these tests, very nice!
| ; | ||
|
|
||
| lookupCommand | ||
| : LOOKUP tableName=qualifiedNamePattern ON matchFields=qualifiedNamePatterns |
There was a problem hiding this comment.
I'm really not sure if this should use a qualified name pattern for the table name. IMHO the table name should be somewhat in line with FROM's index name and ENRICH's policy name. Qualified name patterns are usually used for field names, so we can do stuff like
KEEP `some-thing`.`something-else`.yet_something_else
FROM uses an INDEX_UNQUOTED_IDENTIFIER, ENRICH a similar ENRICH_POLICY_NAME_BODY.
Maybe we just align with FROM here and use INDEX_UNQUOTED_IDENTIFIER as well, unless table names need to be more general than that?
This removes the shim layers for `nested` that we inherited from QL. ESQL doesn't support nested yet and when it does support nested its not going to use the same tricks that QL uses.
This reverts commit e50d09d.
This adds support for `LOOKUP`, a command that implements a sort of
inline `ENRICH`, using data that is passed in the request:
```
$ curl -uelastic:password -HContent-Type:application/json -XPOST \
'localhost:9200/_query?error_trace&pretty&format=txt' \
-d'{
"query": "ROW a=1::LONG | LOOKUP t ON a",
"tables": {
"t": {
"a:long": [ 1, 4, 2],
"v1:integer": [ 10, 11, 12],
"v2:keyword": ["cat", "dog", "wow"]
}
},
"version": "2024.04.01"
}'
v1 | v2 | a
---------------+---------------+---------------
10 |cat |1
```
This required these PRs: * #107624 * #107634 * #107701 * #107762 *
#107923 * #107894 * #107982 * #108012 * #108020 * #108169 * #108191 *
#108334 * #108482 * #108696 * #109040 * #109045
Closes #107306
This command will serve as a sort of "inline" enrich. This commit itself is mostly antlr generated code and paranoid tests that the new
LOOKUPkeyword doesn't clash with any variables namedlookup.I've also marked our ANTLR generated files as
linguist-generatedwhich causes them to be hidden by default in github's UI. You can still click a button to see them if you like. See https://docs.github.com/en/repositories/working-with-files/managing-files/customizing-how-changed-files-appear-on-github