Skip to content

Add 'filter' to 'knn' search section#88217

Merged
jtibshirani merged 3 commits intoelastic:knn-searchfrom
jtibshirani:knn-filter
Jul 13, 2022
Merged

Add 'filter' to 'knn' search section#88217
jtibshirani merged 3 commits intoelastic:knn-searchfrom
jtibshirani:knn-filter

Conversation

@jtibshirani
Copy link
Copy Markdown
Contributor

@jtibshirani jtibshirani commented Jun 30, 2022 •
edited
Loading

This PR introduces a 'filter' subsection where filters can be provided to the
kNN search:

POST index/_search
{
  "knn": {
    "field": "image_vector",
    "query_vector": [0.3f, 0.1f, ...],
    "k": 10,
    "num_candidates": 100,
    "filter": {
      "term": {
       "file_type": "jpeg"
      }
    }
  }
}

Like all other queries, the filters are rewritten on the coordinating nodes as
well as data nodes.

Addresses #87625.

@jtibshirani jtibshirani added >non-issue :Search/Search Search-related issues that do not fall into other categories Team:Clients Meta label for clients team labels Jun 30, 2022
@elasticmachine
Copy link
Copy Markdown
Collaborator

elasticmachine commented Jun 30, 2022

Pinging @elastic/clients-team (Team:Clients)

@elasticmachine elasticmachine added the Team:Search Meta label for search team label Jun 30, 2022
@elasticmachine
Copy link
Copy Markdown
Collaborator

elasticmachine commented Jun 30, 2022

Pinging @elastic/es-search (Team:Search)

@sethmlarson sethmlarson mentioned this pull request Jun 30, 2022
Closed
@jtibshirani jtibshirani mentioned this pull request Jun 30, 2022
Closed
8 tasks
mayya-sharipova
mayya-sharipova approved these changes Jul 5, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@mayya-sharipova mayya-sharipova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jtibshirani Thanks, great work! I left a couple of comments.

server/src/main/java/org/elasticsearch/search/vectors/KnnSearchBuilder.java Outdated
Copy link
Copy Markdown
Contributor

@mayya-sharipova mayya-sharipova Jul 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add filterQueries to hashCode() as well?

Copy link
Copy Markdown
Contributor Author

@jtibshirani jtibshirani Jul 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, thanks for catching that!

server/src/main/java/org/elasticsearch/search/vectors/KnnSearchBuilder.java Outdated
Copy link
Copy Markdown
Contributor

@mayya-sharipova mayya-sharipova Jul 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we write filterQueries the same way we read them:
out.writeNamedWriteableList(filterQueries);?

Copy link
Copy Markdown
Contributor Author

@jtibshirani jtibshirani Jul 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, not sure why I did this manually.

@jtibshirani
Add 'filter' to 'knn' search section
25a230a 
This PR introduces a 'filter' subsection where filters can be provided to the
kNN search.

Like all other queries, the filters are rewritten on the coordinating nodes as
well as data nodes.
@jtibshirani
Fix checkstyle
2a54460
@jtibshirani
Address code review comments
8220e70
@jtibshirani
Copy link
Copy Markdown
Contributor Author

jtibshirani commented Jul 13, 2022

Thanks @mayya-sharipova for the review. I addressed your comments and also updated the feature branch knn-search with the latest commits from main (that's why there are force pushes).

@jtibshirani jtibshirani merged commit a135feb into elastic:knn-search Jul 13, 2022
@jtibshirani jtibshirani deleted the knn-filter branch July 13, 2022 09:10
@jtibshirani jtibshirani added :Search Relevance/Vectors Vector search and removed :Search/Search Search-related issues that do not fall into other categories labels Jul 21, 2022
@jtibshirani jtibshirani mentioned this pull request Jul 21, 2022
Merged
@sethmlarson sethmlarson mentioned this pull request Jul 26, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mayya-sharipova mayya-sharipova mayya-sharipova approved these changes

Assignees

No one assigned

Labels

>non-issue :Search Relevance/Vectors Vector search Team:Clients Meta label for clients team Team:Search Meta label for search team v8.4.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jtibshirani @elasticmachine @mayya-sharipova