Dialing failed when connecting 4 nodes locally to abci #3178
Description
Tendermint version : 0.28.1-07263298
ABCI app : https://github.com/gauthamzz/vimana/blob/master/tendermint/app.py
Environment:
- OS (e.g. from /etc/os-release): Ubuntu 18.04
- Install tools:
- Others:
What happened:
Created a docker-compose to connect to 4 nodes using abci written in python.
version: '3'
services:
node0:
container_name: tmnode0
image: "tendermint/localnode"
ports:
- "26656-26657:26656-26657"
environment:
- ID=0
- LOG=$${LOG:-tendermint.log}
volumes:
- ./build:/tendermint:Z
command: node --proxy_app=tcp://abci0:26658 --consensus.create_empty_blocks=false
# command: node --proxy_app=kvstore
networks:
localnet:
ipv4_address: 192.167.10.2
extra_hosts:
- "dockerhost:192.167.0.1"
node1:
container_name: tmnode1
image: "tendermint/localnode"
ports:
- "26659-26660:26656-26657"
environment:
- ID=1
- LOG=$${LOG:-tendermint.log}
volumes:
- ./build:/tendermint:Z
command: node --proxy_app=tcp://abci1:26658 --consensus.create_empty_blocks=false
# command: node --proxy_app=kvstore
depends_on:
- abci1
networks:
localnet:
ipv4_address: 192.167.10.3
node2:
container_name: tmnode2
image: "tendermint/localnode"
environment:
- ID=2
- LOG=$${LOG:-tendermint.log}
ports:
- "26661-26662:26656-26657"
volumes:
- ./build:/tendermint:Z
command: node --proxy_app=tcp://abci2:26658 --consensus.create_empty_blocks=false
# command: node --proxy_app=kvstore
depends_on:
- abci2
networks:
localnet:
ipv4_address: 192.167.10.4
node3:
container_name: tmnode3
image: "tendermint/localnode"
environment:
- ID=3
- LOG=$${LOG:-tendermint.log}
ports:
- "26663-26664:26656-26657"
volumes:
- ./build:/tendermint:Z
command: node --proxy_app=tcp://abci3:26658 --consensus.create_empty_blocks=false
# command: node --proxy_app=kvstore
depends_on:
- abci3
networks:
localnet:
ipv4_address: 192.167.10.5
abci0:
container_name: abci0
image: "abcinode"
build:
context: .
dockerfile: abci.Dockerfile
command: python3 vimana/tendermint/app.py
networks:
localnet:
ipv4_address: 192.167.10.6
abci1:
container_name: abci1
image: "abcinode"
build:
context: .
dockerfile: abci.Dockerfile
command: python3 vimana/tendermint/app.py
networks:
localnet:
ipv4_address: 192.167.10.7
abci2:
container_name: abci2
image: "abcinode"
build:
context: .
dockerfile: abci.Dockerfile
command: python3 vimana/tendermint/app.py
networks:
localnet:
ipv4_address: 192.167.10.8
abci3:
container_name: abci3
image: "abcinode"
build:
context: .
dockerfile: abci.Dockerfile
command: python3 vimana/tendermint/app.py
networks:
localnet:
ipv4_address: 192.167.10.9
networks:
localnet:
driver: bridge
ipam:
driver: default
config:
-
subnet: 192.167.10.0/16
It works fine with kvstore app.
What you expected to happen:
The nodes were supposed to send transactions with each other, but only initial state is committed .
Have you tried the latest version: yes/no
yes
How to reproduce it (as minimally and precisely as possible):
This is the https://github.com/gauthamzz/vimana/blob/master/docker/abci.Dockerfile abci dockerfile. I used the same tendermint/localnet.
make localnet-start
Logs (paste a small part showing an error (< 10 lines) or link a pastebin, gist, etc. containing more of the log file):
tmnode1 | E[21016-01-21|09:26:55.506] Dialing failed module=pex addr=cfe71f7e4c5c27d0fc4e2f16d10294c47a9c8cbb@192.167.10.3:26656 err="self ID<cfe71f7e4c5c27d0fc4e2f16d10294c47a9c8cbb>" attempts=0 tmnode0 | E[21016-01-21|09:26:55.997] Dialing failed module=pex addr=a619771b0df58e531bd847803376de1b880491bc@192.167.10.2:26656 err="self ID<a619771b0df58e531bd847803376de1b880491bc>" attempts=0 tmnode3 | E[21016-01-21|09:26:56.674] Dialing failed module=pex addr=4fff6add7b50fe166b437053eae5d036a47b4b5d@192.167.10.5:26656 err="self ID<4fff6add7b50fe166b437053eae5d036a47b4b5d>" attempts=0 tmnode2 | E[21016-01-21|09:27:24.501] Dialing failed module=pex addr=9b17a1d854ef727fdc8294f35301b2b56ce33cbe@192.167.10.4:26656 err="self ID<9b17a1d854ef727fdc8294f35301b2b56ce33cbe>" attempts=0
Config (you can paste only the changes you've made):
node command runtime flags:
node --proxy_app=tcp://abci3:26658 --consensus.create_empty_blocks=false
/dump_consensus_state output for consensus bugs
"jsonrpc": "2.0",
"id": "",
"result": {
"round_state": {
"height": "7",
"round": "0",
"step": 2,
"start_time": "2019-01-21T09:37:22.087776813Z",
"commit_time": "2019-01-21T09:37:21.087776813Z",
"validators": {
"validators": [
{
"address": "4E457BA974FC7F1A5F4943CD2B8EB811DD5E9921",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "tWLf20xV8gkLNGFkFPClLQDKW8tUO3Mu0AzK+yHSn2M="
},
"voting_power": "1",
"proposer_priority": "-2"
},
{
"address": "A31DA4CF8FA1EF32501491217C32A66C7FA6ACB7",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "M1PQhySOKUxf3kuM2MDVBRVlUjy6igpZBz0QakKnupc="
},
"voting_power": "1",
"proposer_priority": "-2"
},
{
"address": "D7CAA1DA24366A928E45382521B90C53C61AF4DF",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "HXhhyH5ztt27UZdhKsTF0sC9l+kZltPSJa3/A1dtwDM="
},
"voting_power": "1",
"proposer_priority": "-2"
},
{
"address": "F9F4E95A91F3237CA1B12D7DC50BA3E44A99BC22",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "vjWgJFijm0DjATSZw++lQG2MzGNzY+RdD6nb9slQQfc="
},
"voting_power": "1",
"proposer_priority": "2"
}
],
"proposer": {
"address": "D7CAA1DA24366A928E45382521B90C53C61AF4DF",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "HXhhyH5ztt27UZdhKsTF0sC9l+kZltPSJa3/A1dtwDM="
},
"voting_power": "1",
"proposer_priority": "-2"
}
},
"proposal": null,
"proposal_block": null,
"proposal_block_parts": null,
"locked_round": "-1",
"locked_block": null,
"locked_block_parts": null,
"valid_round": "-1",
"valid_block": null,
"valid_block_parts": null,
"votes": [
{
"round": "0",
"prevotes": [
"nil-Vote",
"nil-Vote",
"nil-Vote",
"nil-Vote"
],
"prevotes_bit_array": "BA{4:____} 0/4 = 0.00",
"precommits": [
"nil-Vote",
"nil-Vote",
"nil-Vote",
"nil-Vote"
],
"precommits_bit_array": "BA{4:____} 0/4 = 0.00"
},
{
"round": "1",
"prevotes": [
"nil-Vote",
"nil-Vote",
"nil-Vote",
"nil-Vote"
],
"prevotes_bit_array": "BA{4:____} 0/4 = 0.00",
"precommits": [
"nil-Vote",
"nil-Vote",
"nil-Vote",
"nil-Vote"
],
"precommits_bit_array": "BA{4:____} 0/4 = 0.00"
}
],
"commit_round": "-1",
"last_commit": {
"votes": [
"Vote{0:4E457BA974FC 6/00/2(Precommit) 45D30C13ACFD 43271D877838 @ 2019-01-21T09:37:20.98657301Z}",
"Vote{1:A31DA4CF8FA1 6/00/2(Precommit) 45D30C13ACFD 4F5C2EA36B47 @ 2019-01-21T09:37:20.986586448Z}",
"Vote{2:D7CAA1DA2436 6/00/2(Precommit) 45D30C13ACFD 2AFCCD954EAB @ 2019-01-21T09:37:20.985439927Z}",
"Vote{3:F9F4E95A91F3 6/00/2(Precommit) 45D30C13ACFD 920262B8183B @ 2019-01-21T09:37:20.987928295Z}"
],
"votes_bit_array": "BA{4:xxxx} 4/4 = 1.00",
"peer_maj_23s": {}
},
"last_validators": {
"validators": [
{
"address": "4E457BA974FC7F1A5F4943CD2B8EB811DD5E9921",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "tWLf20xV8gkLNGFkFPClLQDKW8tUO3Mu0AzK+yHSn2M="
},
"voting_power": "1",
"proposer_priority": "-3"
},
{
"address": "A31DA4CF8FA1EF32501491217C32A66C7FA6ACB7",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "M1PQhySOKUxf3kuM2MDVBRVlUjy6igpZBz0QakKnupc="
},
"voting_power": "1",
"proposer_priority": "-3"
},
{
"address": "D7CAA1DA24366A928E45382521B90C53C61AF4DF",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "HXhhyH5ztt27UZdhKsTF0sC9l+kZltPSJa3/A1dtwDM="
},
"voting_power": "1",
"proposer_priority": "1"
},
{
"address": "F9F4E95A91F3237CA1B12D7DC50BA3E44A99BC22",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "vjWgJFijm0DjATSZw++lQG2MzGNzY+RdD6nb9slQQfc="
},
"voting_power": "1",
"proposer_priority": "1"
}
],
"proposer": {
"address": "A31DA4CF8FA1EF32501491217C32A66C7FA6ACB7",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "M1PQhySOKUxf3kuM2MDVBRVlUjy6igpZBz0QakKnupc="
},
"voting_power": "1",
"proposer_priority": "-3"
}
}
},
"peers": [
{
"node_address": "cfe71f7e4c5c27d0fc4e2f16d10294c47a9c8cbb@0.0.0.0:26656",
"peer_state": {
"round_state": {
"height": "7",
"round": "0",
"step": 1,
"start_time": "2019-01-21T09:37:21.316511984Z",
"proposal": false,
"proposal_block_parts_header": {
"total": "0",
"hash": ""
},
"proposal_block_parts": null,
"proposal_pol_round": "-1",
"proposal_pol": "____",
"prevotes": "____",
"precommits": "____",
"last_commit_round": "0",
"last_commit": "xxxx",
"catchup_commit_round": "-1",
"catchup_commit": "____"
},
"stats": {
"votes": "13",
"block_parts": "2"
}
}
},
{
"node_address": "9b17a1d854ef727fdc8294f35301b2b56ce33cbe@0.0.0.0:26656",
"peer_state": {
"round_state": {
"height": "7",
"round": "0",
"step": 1,
"start_time": "2019-01-21T09:37:21.314743254Z",
"proposal": false,
"proposal_block_parts_header": {
"total": "0",
"hash": ""
},
"proposal_block_parts": null,
"proposal_pol_round": "-1",
"proposal_pol": "____",
"prevotes": "____",
"precommits": "____",
"last_commit_round": "0",
"last_commit": "xxxx",
"catchup_commit_round": "-1",
"catchup_commit": "____"
},
"stats": {
"votes": "11",
"block_parts": "1"
}
}
},
{
"node_address": "4fff6add7b50fe166b437053eae5d036a47b4b5d@0.0.0.0:26656",
"peer_state": {
"round_state": {
"height": "7",
"round": "0",
"step": 1,
"start_time": "2019-01-21T09:37:21.315087061Z",
"proposal": false,
"proposal_block_parts_header": {
"total": "0",
"hash": ""
},
"proposal_block_parts": null,
"proposal_pol_round": "-1",
"proposal_pol": "____",
"prevotes": "____",
"precommits": "____",
"last_commit_round": "0",
"last_commit": "xxxx",
"catchup_commit_round": "-1",
"catchup_commit": "____"
},
"stats": {
"votes": "11",
"block_parts": "2"
}
}
}
]
}
}
Anything else we need to know:
One a single node it was working fine, with the tendermint and abci.
I am just trying to run an abci and tendermint node on localcluster. If i am doing wrong way, any links will help. Thank 😸
The blocks are getting send sometimes and most of the times the height is not increasing/tx is not going.