Context

Fixes #5117

How things currently work:

• Topology of the peers (public keys and addresses) are stored on-chain.
• There are (Un)RegisterPeer instructions for changing topology.
• Initial topology is provided in the genesis block (via RegisterPeer instructions)
• In case genesis block is not provided to the peer, there is sumeragi.trusted_peers config parameter with addresses of initial peers

Solution

Planned changes:

• Peer addresses will not be stored on-chain (removed from topology)
• Each peer will store current peer addresses off-chain and do periodic gossiping
• Peer A will accept connection from peer B if peer B public key is in topology
• Peer public address will be added to the config.
  • Public address will be sent when connecting to other peers - so that other peers will know our peer's external address and can broadcast this address to other peers (more details in "Use case: New peer added")
  • Note that peer internal address is already in config - network.address
  • Public port can be different from internal port e.g. in case when peer is in docker or behind nginx

Use case: Peer changes address

• Initially there are three peers: A, B, C
• C changes its address
• C knows addresses of other peers and tries to establish connection with them
• Other peers accept connection because C public key is in topology

Use case: New peer added

• Initially there are peers A and B with addresses iroha1:8000 and iroha2:8000
• We want to add peer C with address iroha3:8000
• First, we invoke RegisterPeer instruction to add C public key to topology
• Then peer C is started with the following config:
  • sumeragi.trusted_peers - only peer A
  • network.public_address - iroha3:8000
• C tries to connect to A
  • A receives connection from 9.9.9.9:52143 (52143 is some random port chosen by C for outcoming connection, and 9.9.9.9 is some IP address, e.g. local IP address in kubernetes cluster)
  • A accepts such connection because C public key is in topology
  • C sends its public address (iroha3:8000) to A
  • A gossip address of C to B
  • B receives address of C and connects to C

Migration Guide

• New parameter was added (network.public_address in the config, or P2P_PUBLIC_ADDRESS env var). This parameter should contain external address of the peer used for p2p (as seen by other peers)
• genesis.json — changed topology:

Before:

"topology": [
    {
        "address": "irohad0:1001",
        "id": "ed0120A88F67F9D5D94F63DC57661D0B15054298BBB66B39E601FED71E97CD593A7FDB"
    },
    {
        "address": "irohad0:1002",
        "id": "ed0120647E9781F012EF44281A13D673178767465E1622E7BB1094C4C375B98133DBD1"
    },
    ...
]

After:

"topology": [
    "ed0120A88F67F9D5D94F63DC57661D0B15054298BBB66B39E601FED71E97CD593A7FDB",
    "ed0120647E9781F012EF44281A13D673178767465E1622E7BB1094C4C375B98133DBD1",
    ...
]

Review notes

Most meaningful changes are:

• crates/iroha_core/src/peers_gossiper.rs - logic related to peers gossiping
• crates/iroha_p2p/src/network.rs
  • Now peer connects to addresses received from PeersGossiper service. (Previously peer connects to addresses of peers from topology)
  • In each pair of peers, both try to establish initial connection (one of two connections will be dropped based on disambiguator rule). This is needed since when adding new peer, other peers might not know its address
• crates/iroha_p2p/src/peer.rs - after establishing connection, peer now sends its public_address
• crates/iroha_data_model/src/peer.rs - moved address from PeerId to Peer

Other changes are mostly related to Peer/PeerId

TODO:

• Fix tests (in particular multiple_networks)
• Add new tests
• Fix docker compose

Checklist

• I've read CONTRIBUTING.md.
• (optional) I've written unit tests for the code changes.
• All review comments have been resolved.
• All CI checks pass.