Feature/Search: Reorganize section, and add content about hybrid search#106
Feature/Search: Reorganize section, and add content about hybrid search#106
Conversation
| ## Learn | ||
|
|
||
| To learn more about vector search, please visit the corresponding page about | ||
| [](#vector-store). | ||
|
|
||
|
|
||
| :::{todo} | ||
| Bring page into the same shape like the others in this section. | ||
| Maybe just move the [](#vector-store) page here without further ado. | ||
| ::: |
There was a problem hiding this comment.
Better repurpose and dissolve the main-level Vector Store page completely, and just move it here, in order to provide a better narrative and enhanced guidance. wdyt?
There was a problem hiding this comment.
What do you think about this, @surister? In my impression, looking at it from a fresh perspective, listing "Vector Store" as a main feature does not make too much sense, because you will mostly use it for searching anyway.
In this spirit, I am planning to move that page to the revamped /search/ section, in order to make the trio of (fts, vector, hybrid) complete. Do you agree?
There was a problem hiding this comment.
Hm. Thinking about it once more, the Vector Store page might still make sense to have "Vector" enumerated on the top menu level in that section. Otherwise, it might make people feel it is missing here.
Maybe still keep that page, but switch the content around?
There was a problem hiding this comment.
Hi again. I decided to refactor mercilessly, and dissolved both the "Vector Store" and the "Geospatial Data" page, refactoring them into "Vector Search" and "Geospatial Search", now making up a quartett together with the newly added "Hybrid Search" page.
There was a problem hiding this comment.
That other patch adds relevant redirects to compensate for refactored pages now available on a different location.
surister
left a comment
There was a problem hiding this comment.
Brilliant, I love the idea and look forward putting in more content, great stuff
|
At the beginning you talk about how vectors is not enough hence we need to mix with bm25, this is very well written in the description of https://haystackconf.com/us2023/talk-16/, maybe it can serve as an inspiration? |
Thanks a stack, appreciate it.
I've adjusted my OP by adding a few inspirations I had slightly used for conceiving the first sketch of this patch. I was sure you would provide much better resources than coming from my quick research. Thanks! I will be so happy to accept well rewritten paragraphs on this PR, based on the material you are suggesting, if you can afford a few cycles. Every chunk counts! Otherwise, we will probably need to add it to the backlog summary at GH-101, because I need to take care about other obligations. |
I've diverted this to the backlog at #101 (comment). Thanks! |
- Spend a dedicated slot for each of FTS vs. Vector vs. Hybrid, in order to give the structure more strength about adding more relevant content next to the introductory tidbits. - Refactor feature/vector page into feature/search/vector, renaming it from "Vector Store" to "Vector Search". It has already been curated well, and provides the same valuable shape to this documentation slot like all the other "feature card" pages in the whole "All Features" subsection.
- FTS: Re-shuffle content cards in "Learn" subsection. Add blog articles about "Indexing and Storage in CrateDB" and "Indexing Text for Both Effective Search and Accurate Analysis". - Vector+Hybrid: This and that.
- Hybrid: Bring page into the same shape like the others. Add a little "Usage" section, that can be improved later. - Hybrid: Add two output examples from blog article. - Hybrid: Cross-linking. - FTS: Add another SQL examples. - Vector: Minor improvements. - Advanced Querying: Add references to FTS-, Vector-, and Hybrid-Search pages.
About
Inspired by Doing Hybrid Search in CrateDB (thanks @surister!), this patch intends to reorganize the "Search" section, in order to make its structure strong enough to cover the discrimination between Full-Text Search, Vector Search, and Hybrid Search well.
Preview
https://cratedb-guide--106.org.readthedocs.build/feature/search/
Details
Thoughts
Inspirations
Only just a bit. Relevant sections need to be improved further.
/cc @matriv, @mkleen, @seut, @BaurzhanSakhariev, @karynzv, @hlcianfagna, @hammerhead, @proddata, @WalBeh, @selina-meyer, @donmadeus, @widmogrod, @kneth