-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Description
Hello guys, I hope this message finds you well.
I could not find another way to ask this question, but could you inform me about the configurations to achieve the throughput of 10K operations/second?
"speed: Tendermint blocks can commit to finality in the order of 1 second. Tendermint can handle transaction volume at the rate of 10,000 transactions per second for 250byte transactions. The bottleneck is in the application."
I am using four (4) persistent_peers but I can't reach the 10K throughput.
I barely achieve 2K, I am getting this throughput through tm-monitor.
I tried many variants of tm-bench command (with r=1000, r=10K, c=100, etc. ) through 6 machines:
./tm-bench -v -c 50 -T 60 -r 100 -s 300 192.168.2.29:26657 -broadcast-tx-method commit
Even send the tx to all endpoints, I can't reach more than 2k.
I believe that the batch is getting full because of the logs says during several outputs:
" I[1126-12-01|16:34:31.285] Executed block module=state height=992 validTxs=36514 invalidTxs=0
I[1126-12-01|16:34:31.299] Committed state module=state height=992 txs=36514 appHash=F8CF380000000000"
Thanks in advance!!!
##########################################################
################ Detailed machine configuration ################
I have four nodes with the configuration below running the persistent_peers:
Linux s9 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 26
Stepping: 5
CPU MHz: 2261.099
BogoMIPS: 4521.86
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 8192K
NUMA node0 CPU(s): 0,2,4,6,8,10,12,14
NUMA node1 CPU(s): 1,3,5,7,9,11,13,15
Each one has this config.toml:
#### This is a TOML config file.
#### For more information, see https://github.com/toml-lang/toml
### main base config options #####
#### TCP or UNIX socket address of the ABCI application,
#### or the name of an ABCI application compiled in with the Tendermint binary
proxy_app = "tcp://192.168.2.29:26658"
#### A custom human readable name for this node
moniker = "s9"
#### If this node is many blocks behind the tip of the chain, FastSync
#### allows them to catchup quickly by downloading blocks in parallel
#### and verifying their commits
fast_sync = true
#### Database backend: leveldb | memdb | cleveldb
db_backend = "leveldb"
#### Database directory
db_dir = "data"
#### Output level for logging, including package level options
log_level = "main:info,state:info,*:error"
#### Output format: 'plain' (colored text) or 'json'
log_format = "plain"
### additional base config options #####
#### Path to the JSON file containing the initial validator set and other meta data
genesis_file = "config/genesis.json"
#### Path to the JSON file containing the private key to use as a validator in the consensus protocol
priv_validator_file = "config/priv_validator.json"
#### TCP or UNIX socket address for Tendermint to listen on for
#### connections from an external PrivValidator process
priv_validator_laddr = ""
#### Path to the JSON file containing the private key to use for node authentication in the p2p protocol
node_key_file = "config/node_key.json"
#### Mechanism to connect to the ABCI application: socket | grpc
abci = "socket"
#### TCP or UNIX socket address for the profiling server to listen on
prof_laddr = ""
#### If true, query the ABCI app on connecting to a new peer
#### so the app can decide if we should keep the connection or not
filter_peers = false
### advanced configuration options #####
#### rpc server configuration options #####
[rpc]
#### TCP or UNIX socket address for the RPC server to listen on
laddr = "tcp://192.168.2.29:26657"
#### A list of origins a cross-domain request can be executed from
#### Default value '[]' disables cors support
#### Use '["*"]' to allow any origin
cors_allowed_origins = "[*]"
#### A list of methods the client is allowed to use with cross-domain requests
cors_allowed_methods = "[HEAD GET POST]"
#### A list of non simple headers the client is allowed to use with cross-domain requests
cors_allowed_headers = "[Origin Accept Content-Type X-Requested-With X-Server-Time]"
#### TCP or UNIX socket address for the gRPC server to listen on
#### NOTE: This server only supports /broadcast_tx_commit
grpc_laddr = ""
#### Maximum number of simultaneous connections.
#### Does not include RPC (HTTP&WebSocket) connections. See max_open_connections
#### If you want to accept more significant number than the default, make sure
#### you increase your OS limits.
#### 0 - unlimited.
#### Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files}
#### 1024 - 40 - 10 - 50 = 924 = ~900
grpc_max_open_connections = 1024
#### Activate unsafe RPC commands like /dial_seeds and /unsafe_flush_mempool
unsafe = false
#### Maximum number of simultaneous connections (including WebSocket).
#### Does not include gRPC connections. See grpc_max_open_connections
#### If you want to accept more significant number than the default, make sure
#### you increase your OS limits.
#### 0 - unlimited.
#### Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files}
#### 1024 - 40 - 10 - 50 = 924 = ~900
max_open_connections = 8192
### peer to peer configuration options #####
[p2p]
#### Address to listen for incoming connections
laddr = "tcp://192.168.2.29:26656"
#### Address to advertise to peers for them to dial
#### If empty, will use the same port as the laddr,
#### and will introspect on the listener or use UPnP
#### to figure out the address.
external_address = ""
#### Comma separated list of seed nodes to connect to
seeds = ""
#### Comma separated list of nodes to keep persistent connections to
persistent_peers = "33d312fe0853b9882227dc4238f46ae697ae47cf@192.168.2.30:26656,19cc59c4cfd17cf300cdef05cd86a9b6b3fbda0d@192.168.2.31:26656,87c0a035944c168af9b55d0f13eb22e724224fd2@192.168.2.32:26656"
#### UPNP port forwarding
upnp = false
#### Path to address book
addr_book_file = "config/addrbook.json"
#### Set true for strict address routability rules
#### Set false for private or local networks
addr_book_strict = false
#### Maximum number of inbound peers
max_num_inbound_peers = 500
#### Maximum number of outbound peers to connect to, excluding persistent peers
max_num_outbound_peers = 500
#### Time to wait before flushing messages out on the connection
flush_throttle_timeout = "100ms"
#### Maximum size of a message packet payload, in bytes
max_packet_msg_payload_size = 1024
#### Rate at which packets can be sent, in bytes/second
send_rate = 5120000
#### Rate at which packets can be received, in bytes/second
recv_rate = 5120000
#### Set true to enable the peer-exchange reactor
pex = true
#### Seed mode, in which node constantly crawls the network and looks for
#### peers. If another node asks it for addresses, it responds and disconnects.
#### Does not work if the peer-exchange reactor is disabled.
seed_mode = false
#### Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
private_peer_ids = ""
#### Toggle to disable guard against peers connecting from the same ip.
allow_duplicate_ip = true
#### Peer connection configuration.
handshake_timeout = "20s"
dial_timeout = "10s"
### mempool configuration options #####
[mempool]
recheck = true
broadcast = true
wal_dir = ""
#### size of the mempool
size = 500000
#### size of the cache (used to filter transactions we saw earlier)
cache_size = 200000
### consensus configuration options #####
[consensus]
wal_file = "data/cs.wal/wal"
timeout_propose = "15s"
timeout_propose_delta = "5000ms"
timeout_prevote = "15s"
timeout_prevote_delta = "5000ms"
timeout_precommit = "15s"
timeout_precommit_delta = "5000ms"
timeout_commit = "30s"
#### Make progress as soon as we have all the precommits (as if TimeoutCommit = 0)
skip_timeout_commit = false
#### EmptyBlocks mode and possible interval between empty blocks
create_empty_blocks = true
create_empty_blocks_interval = "1s"
#### Reactor sleep duration parameters
peer_gossip_sleep_duration = "200ms"
peer_query_maj23_sleep_duration = "4s"
#### Block time parameters. Corresponds to the minimum time increment between consecutive blocks.
blocktime_iota = "1s"
### transactions indexer configuration options #####
[tx_index]
#### What indexer to use for transactions
#### Options:
#### 1) "null" (default)
#### 2) "kv" - the simplest possible indexer, backed by key-value storage (defaults to levelDB; see DBBackend).
indexer = "kv"
#### Comma-separated list of tags to index (by default the only tag is "tx.hash")
#### You can also index transactions by height by adding "tx.height" tag here.
#### It's recommended to index only a subset of tags due to possible memory
#### bloat. This is, of course, depends on the indexer's DB and the volume of
#### transactions.
index_tags = ""
#### When set to true, tells indexer to index all tags (predefined tags:
#### "tx.hash", "tx.height" and all tags from DeliverTx responses).
#### Note this may be not desirable (see the comment above). IndexTags has a
#### precedence over IndexAllTags (i.e. when given both, IndexTags will be
#### indexed).
index_all_tags = false
### instrumentation configuration options #####
[instrumentation]
#### When true, Prometheus metrics are served under /metrics on
#### PrometheusListenAddr.
#### Check out the documentation for the list of available metrics.
prometheus = false
#### Address to listen for Prometheus collector(s) connections
prometheus_listen_addr = ":26660"
#### Maximum number of simultaneous connections.
#### If you want to accept more significant number than the default, make sure
#### you increase your OS limits.
#### 0 - unlimited.
max_open_connections = 3
#### Instrumentation namespace
namespace = "tendermint"