Support RDMA as tranport layer protocol by rsocket #9270
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the production environment, RDMA gets popular and common for a
networking card. So support RDMA as transport layer protocol to
improve performance and cost reduction.
Note that this feature is ONLY implemented/tested on Linux.
Actually, this is the v2 implementation. The v1 uses low level IB
verbs API directly, the code and discuss ses PR:
#9161
Instead of low level API, the v2 use rsocket which is implemented
by rdma-core to simplify the work in Redis.
The test result is quite exciting:
CPU: Intel(R) Xeon(R) Platinum 8260.
NIC: Mellanox ConnectX-5.
Config of redis: appendonly no, port 6379, rdma-port 6379, appendonly no,
server_cpulist 12, bgsave_cpulist 16.
For RDMA: ./redis-benchmark -h HOST -c 30 -n 10000000 -r 1000000000
--threads 8 -d 512 -t ping,set,get,lrange_100 --rdma
For TCP: ./redis-benchmark -h HOST -c 30 -n 10000000 -r 1000000000
--threads 8 -d 512 -t ping,set,get,lrange_100
====== PING_INLINE ======
TCP: QPS: 159017 AVG LAT: 0.183
v1 RDMA: QPS: 523944 AVG LAT: 0.054
v2 RDMA: QPS: 492683 AVG LAT: 0.052
====== PING_MBULK ======
TCP: QPS: 162256 AVG LAT: 0.179
v1 RDMA: QPS: 509839 AVG LAT: 0.056
v2 RDMA: QPS: 532226 AVG LAT: 0.048
====== SET ======
TCP: QPS: 154700 AVG LAT: 0.187
v1 RDMA: QPS: 492368 AVG LAT: 0.058
v2 RDMA: QPS: 295534 AVG LAT: 0.095
====== GET ======
TCP: QPS: 159022 AVG LAT: 0.182
v1 RDMA: QPS: 525099 AVG LAT: 0.054
v1 RDMA: QPS: 411488 AVG LAT: 0.065
====== LPUSH (needed to benchmark LRANGE) ======
TCP: QPS: 142537 AVG LAT: 0.207
v1 RDMA: QPS: 395038 AVG LAT: 0.073
v2 RDMA: QPS: 353232 AVG LAT: 0.079
====== LRANGE_100 (first 100 elements) ======
TCP: QPS: 36171 AVG LAT: 0.657
v1 RDMA: QPS: 55266 AVG LAT: 0.412
v2 RDMA: QPS: 52228 AVG LAT: 0.468
Signed-off-by: zhenwei pi pizhenwei@bytedance.com