Merged
Conversation
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
|
|
||
| // Create BoltDB-based stable store and log store | ||
| boltDBPath := filepath.Join(dataDir, "raft.db") | ||
| store, err := boltdb.NewBoltStore(boltDBPath) |
Member
There was a problem hiding this comment.
I understand the boltDB will be more suitable for the Raft case.
But I am still considering if it will make the background knowledge requirement more complex for Oxia?
Collaborator
Author
There was a problem hiding this comment.
Good point. Let me check how difficult it would be to have it based on the KV interface we have already.
mattisonchao
approved these changes
Oct 16, 2025
Collaborator
Author
|
There are problems with the deps that are making Pebble fail in some weird ways (that's also the reason why I was trying with #760) |
merlimat
added a commit
that referenced
this pull request
Oct 20, 2025
Until now, we've used a custom comparer in Pebble to achieve the hiearchical sorting. This PR is changing it to use a key-encoding scheme and byte-wise sorting instead. There are a few reasons: 1. Custom comparer is not working anymore in Pebble 2.1. There are several additional restrictions that makes it possible to continue using the current compare function. We are locked out of upgrading Pebble if we don't change this. 2. We need to upgrade to Pebble 2 because there is a conflict version with ZStd depedency. Newer Zstd is making Pebble 1.x crashing. There are other dependencies that start requiring new Zstd, such as the hashicorp/raft dep in #757. 3. All in all, this will make all the internal DB operations faster, since after the keys are encoded in the wrapper layer, than all the comparisons will just by byte-wise comparison (eg: db iterators, compactions, ...) There is a new marker added by Oxia to the db so we know the key encoding format. Later, we will add more encoding option (eg: natural sorting). The first time a db is opened, if the marker is not set, it will be converted to new format. --------- Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Example of usage: