Skip to content

No sled vector storage#2

Merged
generall merged 2 commits intomasterfrom
no_sled_vector_storage
Dec 12, 2020
Merged

No sled vector storage#2
generall merged 2 commits intomasterfrom
no_sled_vector_storage

Conversation

@generall
Copy link
Member

Turns out that sled consumes too much memory and could not be used to store vectors. This PR replaces sled with rocksdb which is almost a drop in replacement.

@generall generall merged commit ac6221f into master Dec 12, 2020
@generall generall deleted the no_sled_vector_storage branch January 2, 2021 21:08
xzfc added a commit that referenced this pull request May 12, 2024
# This is the 1st commit message:

Introduce InvertedIndexImmutableRam

# The commit message #2 will be skipped:

# fixup! Introduce InvertedIndexImmutableRam
@coderabbitai coderabbitai bot mentioned this pull request Jun 11, 2025
IvanPleshkov added a commit that referenced this pull request Jul 1, 2025
# This is the 1st commit message:

bq encodings

fix tests

Welford's Algorithm

review remarks

remove debug println in test

try fix ci

revert cargo-nextest

async query 4 and 8 bits

use quantized storage for query in hnsw build

fix after rebase

fix tests

Welford's Algorithm

review remarks

remove debug println in test

try fix ci

revert cargo-nextest

async query 4 and 8 bits

use quantized storage for query in hnsw build

fix after rebase

are you happy fmt

# This is the commit message #2:

quantization config without option

# This is the commit message #3:

x64 simd for u128

# This is the commit message #4:

fmt

# This is the commit message #5:

neon simd

# This is the commit message #6:

refactor

# This is the commit message #7:

fmt
generall added a commit that referenced this pull request Oct 9, 2025
generall added a commit that referenced this pull request Oct 9, 2025
* Cleanup README.md

* Rename `PyResult` into `Result`

* Rename `interface` module into `types` and simplify imports

`wildcard::goes::brrr::*`

* Add `PointId` conversion

* Add `Payload` conversion

* fixup! Add `Payload` conversion

Avoid cloning `Payload` for output conversion

* fixup! Add `PointId` conversion

Fix example

* review fixes #1: fix uuid parsing, avoid .into, use custom types

* review fixes #2: rewrite point id to avoid chain if

* review fixes #3: allow ot use uuid + remove comment

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
timvisee pushed a commit that referenced this pull request Nov 14, 2025
* Cleanup README.md

* Rename `PyResult` into `Result`

* Rename `interface` module into `types` and simplify imports

`wildcard::goes::brrr::*`

* Add `PointId` conversion

* Add `Payload` conversion

* fixup! Add `Payload` conversion

Avoid cloning `Payload` for output conversion

* fixup! Add `PointId` conversion

Fix example

* review fixes #1: fix uuid parsing, avoid .into, use custom types

* review fixes #2: rewrite point id to avoid chain if

* review fixes #3: allow ot use uuid + remove comment

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
HighBestCoder pushed a commit to HighBestCoder/qdrant that referenced this pull request Dec 14, 2025
…Lists (qdrant#3327)

Summary:
Pull Request resolved: facebookresearch/faiss#3327

**Context**
1. [Issue 2621](facebookresearch/faiss#2621) discuss inconsistency between OnDiskInvertedList and InvertedList. OnDiskInvertedList is supposed to handle disk based multiple Index Shards. Thus, we should name it differently when merging invls from index shard.
2. [Issue 2876](facebookresearch/faiss#2876) provides usecase of shifting ids when merging invls from different shards.

**In this diff**,
1. To address qdrant#1 above, I renamed the merge_from function to merge_from_multiple without touching merge_from base class.
why so? To continue to allow merge invl from one index to ondiskinvl from other index.

2. To address qdrant#2 above, I have added support of shift_ids in merge_from_multiple to shift ids from different shards. This can be used when each shard has same set of ids but different data. This is not recommended if id is already unique across shards.

Reviewed By: mdouze

Differential Revision: D55482518

fbshipit-source-id: 95470c7449160488d2b45b024d134cbc037a2083
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant