Skip to content

Add support for aggregations, GROK and DISSECT for semantic_text#117337

Merged
ioanatia merged 4 commits intoelastic:mainfrom
ioanatia:semantic_text_commands_support
Nov 25, 2024
Merged

Add support for aggregations, GROK and DISSECT for semantic_text#117337
ioanatia merged 4 commits intoelastic:mainfrom
ioanatia:semantic_text_commands_support

Conversation

@ioanatia
Copy link
Copy Markdown
Member

@ioanatia ioanatia commented Nov 22, 2024

Tracked in #115103

Follow up from #115243 where we the main focus was support for functions.

I still needed to go through the commands list and figure out which ones should support semantic_text, which ones are tested already and which ones need a fix.

This left us with:

  • GROK - was already working, but not tested - added a CSV test
  • DISSECT - was already working, but not tested - added a CSV test
  • STATS BY - was not working when applying COUNT/COUNT_DISTINCT/MIN/MAX/TOP over semantic_text fields. I fixed this and for bwc I added an ESQL capability semantic_text_aggregations (similar to [ESQL] Enable "any type" aggregations on Date Nanos #114438)

The rest of the documented commands (DROP, KEEP, MV_EXPAND, SORT etc) should already be tested.

I hope that with this we can be close to conclude support for semantic_text as a field type in ES|QL and to focus on semantic_text support in match.

@ioanatia ioanatia added >non-issue v8.18.0 :Analytics/ES|QL AKA ESQL auto-backport Automatically create backport pull requests when merged labels Nov 22, 2024
@ioanatia ioanatia mentioned this pull request Nov 22, 2024
16 tasks
@ioanatia ioanatia marked this pull request as ready for review November 22, 2024 13:45
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Nov 22, 2024
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Copy link
Copy Markdown
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯 Thorough work, thank you Ioana

@ioanatia ioanatia merged commit 1402e68 into elastic:main Nov 25, 2024
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

💔 Backport failed

Status Branch Result
8.x Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 117337

alexey-ivanov-es pushed a commit to alexey-ivanov-es/elasticsearch that referenced this pull request Nov 28, 2024
…stic#117337)

* Add support for aggregations for semantic_text

* Add capability to csv tests for grok and dissect

* Sort values to avoid flaky tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL auto-backport Automatically create backport pull requests when merged backport pending >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.18.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants