Skip to content

mempool: allow ReapX and CheckTx functions to run in parallel#4759

Merged
mergify[bot] merged 12 commits intomasterfrom
anton/2972-mempool-lock
May 8, 2020
Merged

mempool: allow ReapX and CheckTx functions to run in parallel#4759
mergify[bot] merged 12 commits intomasterfrom
anton/2972-mempool-lock

Conversation

@melekes
Copy link
Contributor

@melekes melekes commented Apr 28, 2020

allow ReapX and CheckTx functions to run in parallel, making it not possible to block certain proposers from creating a new block.

Closes: #2972

melekes added 3 commits April 28, 2020 16:44
if you look at the code, you'll discover that it's never used. consensus
  locks the whole mempool proxyMtx for the duration of Update (including
  rechecking). Note both ReapMaxBytesMaxGas and ReapMaxTxs try to lock
  the same mutex.
ReapMaxBytesMaxGas,ReapMaxTxs,CheckTx can now run in parallel.
Update and ^ cannot run in parallel, thus preserving safety.

Closes #2972
@auto-comment
Copy link

auto-comment bot commented Apr 28, 2020

👋 Thanks for creating a PR!

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

  • Wrote tests
  • Updated CHANGELOG_PENDING.md
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Updated relevant documentation (docs/) and code comments
  • Re-reviewed Files changed in the Github PR explorer
  • Applied Appropriate Labels

Thank you for your contribution to Tendermint! 🚀

@melekes melekes added C:mempool Component: Mempool R:minor Release: Minor labels Apr 28, 2020
@melekes melekes changed the title mempool: change concurrency model between Reap and CheckTx fn mempool: allow ReapX and CheckTx functions to run in parallel Apr 28, 2020
@melekes melekes marked this pull request as ready for review April 28, 2020 14:37
@melekes melekes requested a review from tessr as a code owner April 28, 2020 14:37
@codecov-io
Copy link

codecov-io commented Apr 28, 2020

Codecov Report

Merging #4759 into master will increase coverage by 0.00%.
The diff coverage is 75.00%.

@@           Coverage Diff           @@
##           master    #4759   +/-   ##
=======================================
  Coverage   64.78%   64.78%           
=======================================
  Files         241      241           
  Lines       23053    23062    +9     
=======================================
+ Hits        14934    14941    +7     
- Misses       6901     6905    +4     
+ Partials     1218     1216    -2     
Impacted Files Coverage Δ
mempool/mempool.go 84.21% <ø> (ø)
node/node.go 58.23% <0.00%> (-0.18%) ⬇️
mempool/clist_mempool.go 82.14% <79.59%> (+0.41%) ⬆️
privval/signer_server.go 95.65% <0.00%> (-4.35%) ⬇️
blockchain/v2/reactor.go 34.70% <0.00%> (-2.99%) ⬇️
libs/clist/clist.go 66.66% <0.00%> (-1.52%) ⬇️
blockchain/v0/pool.go 78.02% <0.00%> (-0.64%) ⬇️
consensus/state.go 74.65% <0.00%> (+0.39%) ⬆️
consensus/reactor.go 73.22% <0.00%> (+0.69%) ⬆️
libs/events/events.go 98.05% <0.00%> (+4.85%) ⬆️

@melekes melekes added S:automerge Automatically merge PR when requirements pass and removed S:automerge Automatically merge PR when requirements pass labels May 4, 2020
@melekes melekes self-assigned this May 5, 2020
if (r.CheckTx.Code == abci.CodeTypeOK) && postCheckErr == nil {
// Check mempool isn't full again to reduce the chance of exceeding the
// limits.
if err := mem.isFull(len(tx)); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should wrap this in a mutex to serialize the checks? It will be serialized by the mutex in addTx() anyway, so serializing here as well shouldn't incur any additional cost.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

serializing here as well shouldn't incur any additional cost.

not sure we will get much by doing that

Copy link

@jinsankim jinsankim Nov 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could cause missing transaction that BroadcastTxSync() returns a success (code = 0) but the tx is not added to mempool.

@melekes melekes added the S:automerge Automatically merge PR when requirements pass label May 8, 2020
@mergify mergify bot merged commit 52784f6 into master May 8, 2020
@mergify mergify bot deleted the anton/2972-mempool-lock branch May 8, 2020 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C:mempool Component: Mempool S:automerge Automatically merge PR when requirements pass

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mempool lock can cause induce adversarial large delay in block reap functions

6 participants