Skip to content

feat: peers discovery lookups#411

Merged
MarcosNicolau merged 18 commits into
mainfrom
feat/peers-lookup
Sep 19, 2024
Merged

feat: peers discovery lookups#411
MarcosNicolau merged 18 commits into
mainfrom
feat/peers-lookup

Conversation

@MarcosNicolau

@MarcosNicolau MarcosNicolau commented Sep 17, 2024

Copy link
Copy Markdown
Contributor

Motivation

Implements lookups in the discovery server.

Description
In the discovery protocol, for every x unit of time (currently hardcoded to 30 min) we need to run lookups to find new peers and fill our buckets. In theory, we should preferably run lookups on the least filled buckets, that is: when sending find_nodes, we should set a public key (target field) such that its distance from the peer to be queried returns the corresponding bucket number. This step is not trivial at all unless we do some brute generation. Instead, we are just sending lookups to three randomly generated pub keys. See more here.

Changes in detail:

  • Fix find_node request target field.
  • New validation when inserting a node that checks if it is already inserted.
  • discovery peer now spawns a new task for the lookups.
  • peers_lookup function that handles the random lookups.

And here is the lookup flow in detail:

  1. Every 30min we spawn three concurrent lookups: one closest to our pubkey and three others closest to a random generated pubkeys.
  2. Every lookup starts with the closest nodes from our table. Each lookup keeps track of:
    • Peers that have already been asked for nodes
    • Peers that have been already seen
    • Potential peers to query for nodes: a vector of up to 16 entries holding the closest peers to the pubkey. This vector is initially filled with nodes from our table.
  3. We send a find_node to the closest 3 nodes (that we have not yet asked) from the pubkey.
  4. We wait for the neighbors' response and push or replace those who are closer to the potential peers.
  5. We select three other nodes from the potential peers vector and do the same until one lookup has no node to ask.

Finally, this implementation is supported by some custom unit and e2e tests.

Closes #154

@MarcosNicolau MarcosNicolau marked this pull request as ready for review September 18, 2024 13:29
@MarcosNicolau MarcosNicolau requested a review from a team as a code owner September 18, 2024 13:29
Comment thread crates/net/net.rs Outdated

@ElFantasma ElFantasma left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job!

@MarcosNicolau

Copy link
Copy Markdown
Contributor Author

I've added recursive lookups, @ElFantasma would you mind doing a review, thanks.

@ElFantasma ElFantasma left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@MarcosNicolau MarcosNicolau added this pull request to the merge queue Sep 19, 2024
Merged via the queue into main with commit 8b444f0 Sep 19, 2024
@MarcosNicolau MarcosNicolau deleted the feat/peers-lookup branch September 19, 2024 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement discovery's main loop

2 participants