-
Notifications
You must be signed in to change notification settings - Fork 32
Fast Post-Filtering Library for Vector Search #282
Description
Is your feature request related to a problem? Please describe.
A filtered vector search consists of a query vector and a filter for a specific field. Current design of VDMS uses the descriptor indices for doing the vector search (KNN) and the filtering is done using the graph store (PMGD, neo4j)
Fast post-filtering library introduces a new layer of filtering that can be used after the vector search and before hitting the graph store to narrow down the candidate list considerably that satisfy both KNN and filter (to avoid sending too many queries to the graph store), and as a result improve the filtered-search throughput
Describe the solution you'd like
1- A new library with an new API to:
(a) create "summaries" which are vectors that matches a certain attribute that the user can apply a fast filter on.
(b) query these "summaries" to return a set of K vectors that satisfy being K-nearest neighbors and satisfy the attribute for filtering
2- Integrate the new library with the KNN flow and graph store search