Skip to content

Drop GraphLayersBackwardCompatibility#5486

Merged
xzfc merged 1 commit intodevfrom
links-comp-drop-GraphLayersBackwardCompatibility
Nov 26, 2024
Merged

Drop GraphLayersBackwardCompatibility#5486
xzfc merged 1 commit intodevfrom
links-comp-drop-GraphLayersBackwardCompatibility

Conversation

@xzfc
Copy link
Member

@xzfc xzfc commented Nov 20, 2024

This PR drops the support for reading/converting of the old graph.bin file format used before Qdrant 0.8.4.
This old format stores everything in a single graph.bin file while newer versions of qdrant store the graph in two files, graph.bin+links.bin.

@xzfc xzfc force-pushed the links-comp-drop-GraphLayersBackwardCompatibility branch from 9fd5193 to 528377e Compare November 20, 2024 09:02
Copy link
Member

@timvisee timvisee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity. What happens if a user still has this file? Will we just ignore it, or do we crash?

I'd prefer if we could ignore it if that isn't too much work so that we can build a fresh HNSW graph later on.

@IvanPleshkov
Copy link
Contributor

Out of curiosity. What happens if a user still has this file? Will we just ignore it, or do we crash?

I'd prefer if we could ignore it if that isn't too much work so that we can build a fresh HNSW graph later on.

I agree with this question. We don't have a conversion from the old format. Such situations are possible even if a user updates qdrant correctly (minor-by-minor version). We probably need to implement a converter and remove everything in the next minor release. Let's discuss it. To avoid spontaneously merging before the decision, I mark this PR as changes required.

Copy link
Contributor

@IvanPleshkov IvanPleshkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mark as requested changes while old version support discussion

@timvisee
Copy link
Member

Such situations are possible even if a user updates qdrant correctly (minor-by-minor version).

Good point.

@IvanPleshkov
Copy link
Contributor

Such situations are possible even if a user updates qdrant correctly (minor-by-minor version).

Good point.

I'm reading current dev and it seems we have conversion and we can merge this PR. @xzfc do you have a chance to check manually if the old format will work correctly?

@xzfc
Copy link
Member Author

xzfc commented Nov 26, 2024

The conversion happens at 0.11.11.0.0 with the following message:

[segment::segment_constructor::segment_constructor_base] Migrating segment 0.4.0 -> 0.5.0

So, this was a dead code already.

@xzfc xzfc merged commit 0d90fad into dev Nov 26, 2024
@xzfc xzfc deleted the links-comp-drop-GraphLayersBackwardCompatibility branch November 26, 2024 19:17
coszio pushed a commit that referenced this pull request Dec 2, 2024
timvisee pushed a commit that referenced this pull request Dec 9, 2024
@timvisee timvisee mentioned this pull request Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants