Skip to content

storage: allocate NodeID via one-off RPC to initialized node #32574

@tbg

Description

@tbg

Our handling of the NodeID is currently quite involved (and leaks into many places) because we pretty much require a working node to be set up before it can use its own KV store to request a NodeID.

There are two levels of improvement here. The first is that once Gossip is connected, and the Node is uninitialized, we use a one-off RPC AllocateNodeID which simply asks a remote node of our choosing (taken from Gossip) to allocate a NodeID for us. This shifts the complexity of using the KV store to a node that has it in perfect working condition. (Similar improvements apply to StoreID allocation, which we may be able to hoist out of the depths of (*Node).Start where it is buried right now).

This removes uninitialized NodeIDs from everywhere except Gossip. Removing it from Gossip is more involved, depending on what guarantees we want to offer. The basic idea is to read the Gossip bootstrap list directly to send AllocateNodeID, and not starting Gossip before that has succeeded. However, consider the case in which a node isn't able to reach out to other nodes (perhaps its join flags are stale) but other nodes are able to reach out to this node (this is a poor setup, but it could happen): we'd get stuck, unless there's at least a minimal subset of Gossip running that would add the peer's address to the list of addresses we pick from for AllocateNodeID. This doesn't seem terrible to implement, but it's a little more involved.

Related to #30553.

Metadata

Metadata

Assignees

Labels

A-kv-serverRelating to the KV-level RPC serverC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions