Skip to content

Added dist functions from usearch#46

Merged
Ngalstyan4 merged 10 commits intolanterndata:mainfrom
var77:feature/add-dist-funcs
Aug 18, 2023
Merged

Added dist functions from usearch#46
Ngalstyan4 merged 10 commits intolanterndata:mainfrom
var77:feature/add-dist-funcs

Conversation

@var77
Copy link
Copy Markdown
Collaborator

@var77 var77 commented Aug 12, 2023

Description

Added new dist functions cos_dist and hamming_dist which will use usearch implementations you can refer to this PR for more information.
l2sq_dist function now will also use usearch implementation.

TODO

  • Create new update file for sql
  • Detect dist function when used with index, so we can set corresponding metric type on usearch options without creating several operators
  • Add generic implementation of dist_func to reject usage outside of index
  • Add tests to cover functionality

Issue

closes #34

@var77 var77 added the WIP label Aug 12, 2023
@var77 var77 requested a review from Ngalstyan4 August 12, 2023 05:31
@var77 var77 marked this pull request as ready for review August 12, 2023 09:13
@var77 var77 removed the WIP label Aug 12, 2023
Copy link
Copy Markdown
Contributor

@Ngalstyan4 Ngalstyan4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@var77 var77 requested a review from Ngalstyan4 August 13, 2023 14:14
Copy link
Copy Markdown
Contributor

@Ngalstyan4 Ngalstyan4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a test file with usage tests

  • Create index with each op class and add some rows
  • Use the wrong distance function in an order by and assure the index is not used (via explain)

@var77 var77 requested a review from Ngalstyan4 August 14, 2023 15:01
@var77 var77 force-pushed the feature/add-dist-funcs branch from f195bd5 to 6fc2dd2 Compare August 16, 2023 11:26
Copy link
Copy Markdown
Contributor

@Ngalstyan4 Ngalstyan4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing!

Copy link
Copy Markdown
Contributor

@Ngalstyan4 Ngalstyan4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did another pass over this.


CREATE TABLE small_world_l2_vec (
id varchar(3),
vector vector(3)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to vactor this into a separate file (e.g. pgvector_compat.sql)
We will need to then move all the other tests that require pgvector into this file so we could make pgvector dependency optional for running tests as well.

@Ngalstyan4 Ngalstyan4 merged commit e823294 into lanterndata:main Aug 18, 2023
var77 added a commit that referenced this pull request Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a new distance function to LanternDB index

2 participants