Please describe the end goal of this project
Lucene introduced Intra Segment Search in version 10 with apache/lucene#13542 giving users the ability to split a segment based on DocIds for concurrent processing in different threads.
This Meta will track all the tasks required for enabling the same in OpenSearch. End goal is to enable it for all queries and aggregations that support ConcurrentSegmentSearch without any reduction in performance.
Supporting References
#18338
Issues
#18849 End User configurations for IntraSegmentConcurrentSearch
#18851 Slicing mechanism changes for equal distribution
#18854 Making the custom collector and collector managers work
[ WIP] There are more tracks of work will update as I create sub-issues
Related component
Search
Please describe the end goal of this project
Lucene introduced Intra Segment Search in version 10 with apache/lucene#13542 giving users the ability to split a segment based on DocIds for concurrent processing in different threads.
This Meta will track all the tasks required for enabling the same in OpenSearch. End goal is to enable it for all queries and aggregations that support ConcurrentSegmentSearch without any reduction in performance.
Supporting References
#18338
Issues
#18849 End User configurations for IntraSegmentConcurrentSearch
#18851 Slicing mechanism changes for equal distribution
#18854 Making the custom collector and collector managers work
[ WIP] There are more tracks of work will update as I create sub-issues
Related component
Search