-
Notifications
You must be signed in to change notification settings - Fork 780
Description
With regards to concurrent calls to ABCI methods, the current specification states that
In principle, each of the four ABCI++ connections operates concurrently with one another. This means applications need to ensure access to state is thread safe. Both the default in-process ABCI client and the default Go ABCI server use a global lock to guard the handling of events across all connections, so they are not concurrent at all. This means whether your app is compiled in-process with Tendermint using the NewLocalClient, or run out-of-process using the SocketServer, ABCI messages from all connections are received in sequence, one at a time.
The existence of this global mutex means Go application developers can get thread safety for application state by routing all reads and writes through the ABCI system. Thus it may be unsafe to expose application state directly to an RPC interface, and unless explicit measures are taken, all queries should be routed through the ABCI Query method.
In the recalled 0.36 branch, changes had been made to allow for more concurrency
In principle, each of the four ABCI++ connections operates concurrently with one another. This means applications need to ensure access to state is thread safe. Up to v0.35.x, both the default in-process ABCI client and the default Go ABCI server used a global lock to guard the handling of events across all connections, so they were not concurrent at all. This meant whether your app was compiled in-process with Tendermint using the NewLocalClient, or run out-of-process using the SocketServer, ABCI messages from all connections were received in sequence, one at a time. This is no longer the case starting from v0.36.0: the global locks have been removed and it is up to the Application to synchronize access to its state when handling ABCI++ methods on all connections. Nevertheless, as all ABCI calls are now synchronous, ABCI messages using the same connection are still received in sequence.
Also
Before invoking Commit, Tendermint locks the mempool and flushes the mempool connection. This ensures that no new messages will be received on the mempool connection during this processing step, providing an opportunity to safely update all four connection states to the latest committed state at the same time.
and
After the Commit call returns, while still holding the mempool lock, CheckTx is run again on all transactions that remain in the node's local mempool after filtering those included in the block. Parameter Type in RequestCheckTx indicates whether an incoming transaction is new (CheckTxType_New), or a recheck (CheckTxType_Recheck).
Finally, after re-checking transactions in the mempool, Tendermint will unlock the mempool connection. New transactions are once again able to be processed through CheckTx.
Meaning that connections are not completely independent.
The goal of this issue is to clearly document what is the concurrency model of ABCI and, possibly, simplify it so applications can improve concurrency in handling ABCI requests.
DoD:
A formal model of the concurrency model of ABCI >1, that is, what ABCI methods may and cannot be called concurrently. The model allows for multi-threaded handling of calls over multiple connections.