Description
Many requests devs are using unfortunately doesn't care about block ranges querying from "0" to "latest".
Responses for such eth_getLogs requests with whole network block range are processing very slow and most likely will be canceled due do timeout (if you have default 60 sec timeout).
Also this has impact on requests queue which grows overtime which eventually starts erroring with
"data": "Nethermind.JsonRpc.Exceptions.ModuleRentalTimeoutException: Unable to rent an instance of IEthRpcModule. Too many concurrent requests.\n at Nethermind.JsonRpc.Modules.BoundedModulePool`1.SlowPath() in /src/Nethermind/Nethermind.JsonRpc/Modules/BoundedModulePool.cs:line 75\n at Nethermind.JsonRpc.Modules.RpcModuleProvider.<>c__DisplayClass17_0`1.<<GetPools>b__0>d.MoveNext()\n---
messages
Steps to Reproduce
- Sync Full node. Below is example for Gnosis Mainnet Full node stats
"args": [
"--config", "gnosis",
"--datadir", "/data/ne-gno-mainnet",
"--JsonRpc.Enabled", "true",
"--JsonRpc.WebSocketsPort", "8545",
"--JsonRpc.RequestQueueLimit","1000",
"--JsonRpc.JwtSecretFile", "/secrets/jwtsecret",
"--JsonRpc.EnabledModules", "net,eth,consensus,subscribe,web3,txpool,trace",
"--JsonRpc.AdditionalRpcUrls", "http://0.0.0.0:8551|http;ws|net;eth;subscribe;engine;web3;client",
"--JsonRpc.Port", 8545,
"--JsonRpc.Host", "0.0.0.0",
"--JsonRpc.EnginePort", 8551,
"--JsonRpc.EngineHost", "0.0.0.0",
"--Receipt.TxLookupLimit", 0,
"--Merge.Enabled", "true",
"--Init.WebSocketsEnabled", "true",
"--Metrics.Enabled", "true",
"--EthStats.Enabled", "true",
"--EthStats.Secret", "${secret}",
"--EthStats.Server", "wss://ethstats.gnosischain.com/api",
"--JsonRpc.Timeout", 60000
],
- Send eth_getLogs from "0x0" to "latest".
curl --location 'https://localhost:8545' \
--header 'Content-Type: application/json' \
--data '{
"jsonrpc": "2.0",
"id": "12",
"method": "eth_getLogs",
"params": [{"address":"0xcB444e90D8198415266c6a2724b7900fb12FC56E","fromBlock":"0x0","toBlock":"latest","topics":["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef","0x0000000000000000000000000f55a8efa8db68c896241b252c032d028b605d53",null]}
]
}'
Actual behavior
Slow response >60 sec. (timeout for our set up)
"error": {
"code": -32016,
"message": "eth_getLogs request was canceled due to enabled timeout."
}
Expected behavior
Less than 5 sec processing time for 0-to-latest block ranges with successful response.
Of course this value is hardly depends of network, settings, node and hardware. Just for comparison - for above case Erigon(v3) responding in <2 sec
Screenshots
If applicable, please include screenshots to help illustrate the problem.
Desktop (please complete the following information):
Please provide the following information regarding your setup:
OS: Ubuntu 22.04.4 LTS
Kernel: 5.15.0-107-generic
Model name: AMD EPYC 9354P 32-Core Processor
Mem: 62Gi 8.6Gi 2.5Gi 84Mi 51Gi 52Gi
nvme0n1 3.5T disk
nvme1n1 3.5T disk
c2:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. Integrated Matrox G200eW3 Graphics Controller (rev 04)
- Version:
nethermind/nethermind:1.31.10
- Installation Method: Docker
- Consensus Client: Lighthouse v7.0.1
Additional context
Below is CPU and disk for when I send for same node 3 requests at the same time:
Description
Many requests devs are using unfortunately doesn't care about block ranges querying from "0" to "latest".
Responses for such eth_getLogs requests with whole network block range are processing very slow and most likely will be canceled due do timeout (if you have default 60 sec timeout).
Also this has impact on requests queue which grows overtime which eventually starts erroring with
messages
Steps to Reproduce
Actual behavior
Slow response >60 sec. (timeout for our set up)
Expected behavior
Less than 5 sec processing time for 0-to-latest block ranges with successful response.
Of course this value is hardly depends of network, settings, node and hardware. Just for comparison - for above case Erigon(v3) responding in <2 sec
Screenshots
If applicable, please include screenshots to help illustrate the problem.
Desktop (please complete the following information):
Please provide the following information regarding your setup:
nethermind/nethermind:1.31.10Additional context
Below is CPU and disk for when I send for same node 3 requests at the same time: