For scan queries, LanternDB currently interfaces with Usearch in one of two ways:
- LanternDB copy-s
node requested by Usearch into a temporary memory buffer and returns its pointer to Usearch
- LanternDB keeps a pin on the Buffer holding the
node and returns its pointer to Usearch
In both cases LanternDB has to keep track of all the currently possibly alive objects. Now, in both cases we use a fixed sized array for this tracking. This requires a larger than necessary allocation in some cases and fails returning the requested results in others, when the fixed buffer is filled.
A better approach is to use a linked list for both of these use-cases. Grow it as necessary and traverse-cleanup upon exit.
For scan queries, LanternDB currently interfaces with Usearch in one of two ways:
noderequested by Usearch into a temporary memory buffer and returns its pointer to Usearchnodeand returns its pointer to UsearchIn both cases LanternDB has to keep track of all the currently possibly alive objects. Now, in both cases we use a fixed sized array for this tracking. This requires a larger than necessary allocation in some cases and fails returning the requested results in others, when the fixed buffer is filled.
A better approach is to use a linked list for both of these use-cases. Grow it as necessary and traverse-cleanup upon exit.