Skip to content

Add a SIMD (Neon) optimised vector distance function for int8#106133

Merged
ChrisHegarty merged 79 commits intoelastic:mainfrom
ChrisHegarty:native_vec
Apr 12, 2024
Merged

Add a SIMD (Neon) optimised vector distance function for int8#106133
ChrisHegarty merged 79 commits intoelastic:mainfrom
ChrisHegarty:native_vec

Conversation

@ChrisHegarty
Copy link
Copy Markdown
Contributor

@ChrisHegarty ChrisHegarty commented Mar 8, 2024 •
edited
Loading

This commit adds an optimised int8 vector distance implementation for aarch64. Additional platforms like, say, x64, will be added as a follow-up.

The vector distance implementation outperforms Lucene's Pamana Vector implementation for binary comparisons by approx 5x (depending on the number of dimensions). It does so by means of SIMD (Neon) intrinsics compiled into a separate native library and link by Panama's FFI. Comparisons are performed on off-heap mmap'ed vector data.

The implementation is currently only used during merging of scalar quantized segments, through a custom format ES814HnswScalarQuantizedVectorsFormat, but its usage will likely be expanded over time.

https://www.elastic.co/search-labs/blog/vector-similarity-computations-ludicrous-speed

@ChrisHegarty ChrisHegarty changed the title [WIP] [WIP] Add an optimised vector distance function for aarch64. Mar 12, 2024
@ChrisHegarty ChrisHegarty mentioned this pull request Mar 27, 2024
Merged
@ChrisHegarty ChrisHegarty added the test-arm Pull Requests that should be tested against arm agents label Mar 27, 2024
@ldematte ldematte mentioned this pull request Apr 10, 2024
Merged
@ChrisHegarty
Copy link
Copy Markdown
Contributor Author

ChrisHegarty commented Apr 10, 2024

@elasticmachine update branch

@ChrisHegarty
Copy link
Copy Markdown
Contributor Author

ChrisHegarty commented Apr 11, 2024

@elasticmachine update branch

@ChrisHegarty
Copy link
Copy Markdown
Contributor Author

ChrisHegarty commented Apr 11, 2024

@elasticmachine update branch

@ChrisHegarty
Copy link
Copy Markdown
Contributor Author

ChrisHegarty commented Apr 11, 2024

@elasticmachine update branch

@ChrisHegarty ChrisHegarty changed the title Add an optimised vector distance function for aarch64. Add an optimised int8 vector distance function for aarch64. Apr 12, 2024
@ChrisHegarty ChrisHegarty merged commit 6b52d78 into elastic:main Apr 12, 2024
@ChrisHegarty ChrisHegarty changed the title Add an optimised int8 vector distance function for aarch64. Add a SIMD (Neon) optimised vector distance function for int8 Apr 12, 2024
@ldematte ldematte mentioned this pull request Apr 12, 2024
Closed
@rjernst rjernst mentioned this pull request Apr 12, 2024
Closed
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

elasticsearchmachine commented Apr 12, 2024

@ChrisHegarty according to this PR's labels, I need to update the changelog YAML, but I can't because the PR is closed. Please either update the changelog yourself on the appropriate branch, or adjust the labels. Specifically:

  • The PR is labelled release highlight but the changelog has no highlight section

1 similar comment
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

elasticsearchmachine commented Apr 15, 2024

@ChrisHegarty according to this PR's labels, I need to update the changelog YAML, but I can't because the PR is closed. Please either update the changelog yourself on the appropriate branch, or adjust the labels. Specifically:

  • The PR is labelled release highlight but the changelog has no highlight section

@ChrisHegarty ChrisHegarty mentioned this pull request Apr 15, 2024
Merged
ChrisHegarty added a commit that referenced this pull request Apr 16, 2024
@ChrisHegarty
Add release highlight for #106133 (#107487)
e6d421c 
This commit adds a release highlight for #106133.
This was referenced Apr 23, 2024
Merged
Merged
@ChrisHegarty
Copy link
Copy Markdown
Contributor Author

ChrisHegarty commented May 1, 2024

Pasting some benchmark results here, for reference:

$ export RUNTIME_JAVA_HOME=/Users/chegar/binaries/jdk-21.0.1.jdk/Contents/Home/
$ ./gradlew -p benchmarks run --args "org.elasticsearch.benchmark.vector.VectorScorerBenchmark.dot.* -p dims=1024"
...
Benchmark                               (dims)   Mode  Cnt   Score   Error   Units
VectorScorerBenchmark.dotProductLucene    1024  thrpt    5   6.437 ± 0.055  ops/us
VectorScorerBenchmark.dotProductNative    1024  thrpt    5  27.302 ± 0.691  ops/us
VectorScorerBenchmark.dotProductScalar    1024  thrpt    5   3.010 ± 0.044  ops/us

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rjernst rjernst rjernst left review comments

@benwtrent benwtrent benwtrent approved these changes

@tveasey tveasey tveasey left review comments

@ldematte ldematte ldematte approved these changes

+1 more reviewer

@jpountz jpountz jpountz left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

>enhancement release highlight :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team test-arm Pull Requests that should be tested against arm agents test-windows Trigger CI checks on Windows v8.14.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@ChrisHegarty @elasticsearchmachine @mark-vieira @rjernst @jpountz @ldematte @benwtrent @tveasey @giladgal @elasticmachine