Skip to content

Raft based metadata provider#757

Merged
merlimat merged 19 commits intooxia-db:mainfrom
merlimat:raft
Oct 21, 2025
Merged

Raft based metadata provider#757
merlimat merged 19 commits intooxia-db:mainfrom
merlimat:raft

Conversation

@merlimat
Copy link
Copy Markdown
Collaborator

@merlimat merlimat commented Oct 15, 2025

Example of usage:

bin/oxia coordinator --conf test.yaml \
                --metadata raft \
                --raft-address node-1:9000 \
                --raft-bootstrap-nodes node-1:9000,node-2:9000,node-3:9000

Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>

// Create BoltDB-based stable store and log store
boltDBPath := filepath.Join(dataDir, "raft.db")
store, err := boltdb.NewBoltStore(boltDBPath)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the boltDB will be more suitable for the Raft case.

But I am still considering if it will make the background knowledge requirement more complex for Oxia?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Let me check how difficult it would be to have it based on the KV interface we have already.

@merlimat
Copy link
Copy Markdown
Collaborator Author

There are problems with the deps that are making Pebble fail in some weird ways (that's also the reason why I was trying with #760)

merlimat added a commit that referenced this pull request Oct 20, 2025
Until now, we've used a custom comparer in Pebble to achieve the
hiearchical sorting.

This PR is changing it to use a key-encoding scheme and byte-wise
sorting instead. There are a few reasons:

1. Custom comparer is not working anymore in Pebble 2.1. There are
several additional restrictions that makes it possible to continue using
the current compare function. We are locked out of upgrading Pebble if
we don't change this.
2. We need to upgrade to Pebble 2 because there is a conflict version
with ZStd depedency. Newer Zstd is making Pebble 1.x crashing. There are
other dependencies that start requiring new Zstd, such as the
hashicorp/raft dep in #757.
3. All in all, this will make all the internal DB operations faster,
since after the keys are encoded in the wrapper layer, than all the
comparisons will just by byte-wise comparison (eg: db iterators,
compactions, ...)

There is a new marker added by Oxia to the db so we know the key
encoding format. Later, we will add more encoding option (eg: natural
sorting).

The first time a db is opened, if the marker is not set, it will be
converted to new format.

---------

Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
@merlimat merlimat merged commit 5cf30fe into oxia-db:main Oct 21, 2025
5 checks passed
@merlimat merlimat deleted the raft branch October 21, 2025 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants