feat: subindex for all document stores#456
Conversation
Codecov Report
@@ Coverage Diff @@
## main #456 +/- ##
==========================================
+ Coverage 83.37% 86.02% +2.64%
==========================================
Files 134 134
Lines 6533 6640 +107
==========================================
+ Hits 5447 5712 +265
+ Misses 1086 928 -158
Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
|
📝 Docs are deployed on https://ft-feat-secondary-index--jina-docs.netlify.app 🎉 |
| if getattr(self, '_subindices', None): | ||
| for selector, da in self._subindices.items(): | ||
| ids_subindex = DocumentArray(self[ids])[selector, 'id'] | ||
| da._del_docs_by_ids(ids_subindex) |
There was a problem hiding this comment.
I think here should be da._del_docs(ids_subindex), to also delete ids in self._offset2ids
This PR is a draft for storing secondary indices as a dict with different docarrays
ToDo:
Implement InMemory as syntax sugarExample usage:
This feature lets you store a separate index for a given nesting level, and then search through it.
This feature works with access paths like
'@c'and'@cc', but also with multi-modal access paths like'@.[image]. Example:Design doc:
Design doc: https://docs.google.com/presentation/d/1rntTa1Ur2WmdAvOUEI01l2WeNSWgtqk2xjth_gdIr50/edit#slide=id.g13443d1ddb2_1_23
Continuation of this: #403