Add optimistic collection to DiversifyingNearestChildren* vector queries #15063
Conversation
|
This PR does not have an entry in lucene/CHANGES.txt. Consider adding one. If the PR doesn't need a changelog entry, then add the skip-changelog label to it and you will stop receiving this reminder on future updates to the PR. |
|
The refactor seems more general and cleaner, thanks, but I think in the test you ran we are seeing a latency increase. Personally I think it's OK given the recall increase and better consistency, but the data is a bit at odds with the statement in the PR description. |
Its an increase in latency with an increase in recall. Let me adjust the test so that we get similar recall on both. Regardless, even with a slight latency increase, fixing the inconsistency takes precedence (IMO). |
| * GITHUB#14226: Use optimistic-with-checking KNN Query execution strategy in place of cross-thread | ||
| global queue min-score checking. Improves performance and consistency. (Mike Sokolov, Dzung Bui, | ||
| Ben Trent) | ||
|
|
There was a problem hiding this comment.
I noticed that its actually in 10x for 10.3 release, but the change note was only in v11.
…lector-all-the-things
…ies (#15063) Optimistic knn search addresses a major issue where we return inconsistent results due to race conditions in the shared queue previously used over multi-segment search. It not only returns consistent results, but is generally better overall when it comes to recall & latency. This PR refactors the optimistic querying in AbstractKnnVectorQuery so that more sub-queries can take advantage of this logic. Additionally, this refactors PatienceKnnVectorQuery to better fit this API and utilize the underlying multi-segment search logic. Here are the results for nested vs. baseline. Note how baseline recall fluctuates, where it is consistent in candidate. closes: #15059
These classes are no longer used after PR apache#14226 and apache#15063 switched KNN queries to optimistic collection strategy. The previous approach used MultiLeafKnnCollector with a shared BlockingFloatHeap to exchange top scores across segments during concurrent search. This caused non-deterministic results due to race conditions in score synchronization between threads.
These classes are no longer used after PR #14226 and #15063 switched KNN queries to optimistic collection strategy. The previous approach used MultiLeafKnnCollector with a shared BlockingFloatHeap to exchange top scores across segments during concurrent search. This caused non-deterministic results due to race conditions in score synchronization between threads.
Optimistic knn search addresses a major issue where we return inconsistent results due to race conditions in the shared queue previously used over multi-segment search.
It not only returns consistent results, but is generally better overall when it comes to recall & latency.
This PR refactors the optimistic querying in AbstractKnnVectorQuery so that more sub-queries can take advantage of this logic.
Additionally, this refactors PatienceKnnVectorQuery to better fit this API and utilize the underlying multi-segment search logic.
Here are the results for nested vs. baseline. Note how baseline recall fluctuates, where it is consistent in candidate.
closes: #15059
BASELINE
Here is baseline with a larger fanout to get similar recall as candidate
Much higher latency and netCPU time.
CANDIDATE