Skip to content

Add new "lookup" for query_vector_builder #141069

@benwtrent

Description

@benwtrent

Description

The query vector builder interface allows for vectors to be created via any mechanism. Right now, we have focused on inference, but a reasonable use-case is using it like "term lookup".

While this isn't a full "more like this" end-to-end experience, its reasonable for folks to have vectors statically stored and desire to fetch them and use those vectors to query.

See: https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-terms-query#query-dsl-terms-lookup-params

I expect this type of lookup for query_vector_builder to be similarly restrictive, with the caveate that we don't allow multiple vectors to match. This restriction may be lifted in the future (e.g. multi-vector search, some field indicating aggregative results, or we just do an or), but for now restricting to a single vector seems safe.

API would be something like:

(...)
{
  "knn": {
    "field": "dense-vector-field",
    "k": 10,
    "num_candidates": 100,
    "query_vector_builder": {
      "lookup": {
        "index": "vectors",
        "id": "foobarbaz",
        "path": "vector_field"
      }
    }
  }
}
(...)

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions