Skip to content

Conversation

@pizhenwei
Copy link
Contributor

In the production environment, RDMA gets popular and common for a
networking card. So support RDMA as transport layer protocol to
improve performance and cost reduction.

Note that this feature is ONLY implemented/tested on Linux.

Actually, this is the v2 implementation. The v1 uses low level IB
verbs API directly, the code and discuss ses PR:
#9161

Instead of low level API, the v2 use rsocket which is implemented
by rdma-core to simplify the work in Redis.

The test result is quite exciting:
CPU: Intel(R) Xeon(R) Platinum 8260.
NIC: Mellanox ConnectX-5.
Config of redis: appendonly no, port 6379, rdma-port 6379, appendonly no,
server_cpulist 12, bgsave_cpulist 16.
For RDMA: ./redis-benchmark -h HOST -c 30 -n 10000000 -r 1000000000
--threads 8 -d 512 -t ping,set,get,lrange_100 --rdma
For TCP: ./redis-benchmark -h HOST -c 30 -n 10000000 -r 1000000000
--threads 8 -d 512 -t ping,set,get,lrange_100

====== PING_INLINE ======
TCP: QPS: 159017 AVG LAT: 0.183
v1 RDMA: QPS: 523944 AVG LAT: 0.054
v2 RDMA: QPS: 492683 AVG LAT: 0.052

====== PING_MBULK ======
TCP: QPS: 162256 AVG LAT: 0.179
v1 RDMA: QPS: 509839 AVG LAT: 0.056
v2 RDMA: QPS: 532226 AVG LAT: 0.048

====== SET ======
TCP: QPS: 154700 AVG LAT: 0.187
v1 RDMA: QPS: 492368 AVG LAT: 0.058
v2 RDMA: QPS: 295534 AVG LAT: 0.095

====== GET ======
TCP: QPS: 159022 AVG LAT: 0.182
v1 RDMA: QPS: 525099 AVG LAT: 0.054
v1 RDMA: QPS: 411488 AVG LAT: 0.065

====== LPUSH (needed to benchmark LRANGE) ======
TCP: QPS: 142537 AVG LAT: 0.207
v1 RDMA: QPS: 395038 AVG LAT: 0.073
v2 RDMA: QPS: 353232 AVG LAT: 0.079

====== LRANGE_100 (first 100 elements) ======
TCP: QPS: 36171 AVG LAT: 0.657
v1 RDMA: QPS: 55266 AVG LAT: 0.412
v2 RDMA: QPS: 52228 AVG LAT: 0.468

Signed-off-by: zhenwei pi pizhenwei@bytedance.com

In the production environment, RDMA gets popular and common for a
networking card. So support RDMA as transport layer protocol to
improve performance and cost reduction.

Note that this feature is ONLY implemented/tested on Linux.

Actually, this is the v2 implementation. The v1 uses low level IB
verbs API directly, the code and discuss ses PR:
    redis#9161

Instead of low level API, the v2 use rsocket which is implemented
by rdma-core to simplify the work in Redis.

The test result is quite exciting:
CPU: Intel(R) Xeon(R) Platinum 8260.
NIC: Mellanox ConnectX-5.
Config of redis: appendonly no, port 6379, rdma-port 6379, appendonly no,
                 server_cpulist 12, bgsave_cpulist 16.
For RDMA: ./redis-benchmark -h HOST -c 30 -n 10000000 -r 1000000000 \
          --threads 8 -d 512 -t ping,set,get,lrange_100 --rdma
For TCP: ./redis-benchmark -h HOST -c 30 -n 10000000 -r 1000000000 \
          --threads 8 -d 512 -t ping,set,get,lrange_100

====== PING_INLINE ======
    TCP: QPS: 159017   AVG LAT: 0.183
v1 RDMA: QPS: 523944   AVG LAT: 0.054
v2 RDMA: QPS: 492683   AVG LAT: 0.052

====== PING_MBULK ======
    TCP: QPS: 162256   AVG LAT: 0.179
v1 RDMA: QPS: 509839   AVG LAT: 0.056
v2 RDMA: QPS: 532226   AVG LAT: 0.048

====== SET ======
    TCP: QPS: 154700   AVG LAT: 0.187
v1 RDMA: QPS: 492368   AVG LAT: 0.058
v2 RDMA: QPS: 295534   AVG LAT: 0.095

====== GET ======
    TCP: QPS: 159022   AVG LAT: 0.182
v1 RDMA: QPS: 525099   AVG LAT: 0.054
v1 RDMA: QPS: 411488   AVG LAT: 0.065

====== LPUSH (needed to benchmark LRANGE) ======
    TCP: QPS: 142537   AVG LAT: 0.207
v1 RDMA: QPS: 395038   AVG LAT: 0.073
v2 RDMA: QPS: 353232   AVG LAT: 0.079

====== LRANGE_100 (first 100 elements) ======
    TCP: QPS:  36171   AVG LAT: 0.657
v1 RDMA: QPS:  55266   AVG LAT: 0.412
v2 RDMA: QPS:  52228   AVG LAT: 0.468

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
@pizhenwei
Copy link
Contributor Author

sentinel is NOT fully implemented, but I don't think it is hard work for rsocket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant