Let IO threads free argv and rewrite objects#13968
Conversation
CE Performance Automation : step 1 of 2 (build) DONE.This comment was automatically generated given a benchmark was triggered.
You can check a comparison in detail via the grafana link |
CE Performance Automation : step 2 of 2 (benchmark) FINISHED.This comment was automatically generated given a benchmark was triggered. Started benchmark suite at 2025-06-19 09:12:18.389962 and took 11288.104381 seconds to finish. In total will run 176 benchmarks. |
Automated performance analysis summaryThis comment was automatically generated given there is performance data available. Using platform named: intel64-ubuntu22.04-redis-icx1 to do the comparison. In summary:
You can check a comparison in detail via the grafana link Comparison between unstable and ShooterIT:io-thread-free.Time Period from 5 months ago. (environment used: oss-standalone) By GROUP change csv:command_group,min_change,max_change By COMMAND change csv:command,min_change,max_change
Regressions test regexp names: memtier_benchmark-1Mkeys-load-zset-listpack-with-100-elements-double-score|memtier_benchmark-1Mkeys-load-zset-with-10-elements-double-score|memtier_benchmark-1key-load-zset-with-5-elements-parsing-float-score|memtier_benchmark-1key-zset-100-elements-zrange-all-elements|memtier_benchmark-1key-zset-100-elements-zrangebyscore-all-elements Full Results table:WARNING: There were 150 benchmarks with NO datapoints for both baseline and comparison. NO datapoints for both baseline and comparison:NO DATAPOINTS test regexp names: latency-rate-limited-10000_qps-memtier_benchmark-100Kkeys-hash-hgetall-50-fields-100B-values|latency-rate-limited-10000_qps-memtier_benchmark-100Kkeys-load-hash-50-fields-with-1000B-values|latency-rate-limited-10000_qps-memtier_benchmark-100Kkeys-load-hash-50-fields-with-100B-values|latency-rate-limited-10000_qps-memtier_benchmark-100Kkeys-load-hash-50-fields-with-10B-values|latency-rate-limited-10000_qps-memtier_benchmark-10Mkeys-load-hash-5-fields-with-100B-values|latency-rate-limited-10000_qps-memtier_benchmark-10Mkeys-load-hash-5-fields-with-100B-values-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-10Mkeys-load-hash-5-fields-with-10B-values|latency-rate-limited-10000_qps-memtier_benchmark-10Mkeys-load-hash-5-fields-with-10B-values-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-100B-expire-use-case|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-10B-expire-use-case|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-1KiB-expire-use-case|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-4KiB-expire-use-case|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-bitmap-getbit-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-generic-exists-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-generic-expire-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-generic-expireat-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-generic-pexpire-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-generic-scan-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-generic-touch-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-generic-ttl-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-hash-hexists|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-hash-hget-hgetall-hkeys-hvals-with-100B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-hash-hgetall-50-fields-10B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-hash-hincrby|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-hash-hmget-5-fields-with-100B-values-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-hash-transactions-multi-exec-pipeline-20|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-list-lpop-rpop-with-100B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-list-lpop-rpop-with-10B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-list-lpop-rpop-with-1KiB-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-hash-5-fields-with-1000B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-hash-5-fields-with-1000B-values-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-hash-hmset-5-fields-with-1000B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-list-with-100B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-list-with-10B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-list-with-1KiB-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-set-intset-with-100-elements|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-set-intset-with-100-elements-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-stream-1-fields-with-100B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-stream-1-fields-with-100B-values-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-stream-5-fields-with-100B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-stream-5-fields-with-100B-values-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-string-with-100B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-string-with-100B-values-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-string-with-10B-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-string-with-10B-values-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-string-with-1KiB-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-string-with-20KiB-values|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-zset-with-10-elements-double-score|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-load-zset-with-10-elements-int-score|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-append-1-100B|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-append-1-100B-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-decr|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-get-100B|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-get-100B-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-get-10B|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-get-10B-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-get-1KiB|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-get-1KiB-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-get-20KiB|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-incrby|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-incrby-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-incrbyfloat|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-incrbyfloat-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-mget-1KiB|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-setex-100B-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-setrange-100B|latency-rate-limited-10000_qps-memtier_benchmark-1Mkeys-string-setrange-100B-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-2-elements-geopos|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-2-elements-geosearch-fromlonlat-withcoord|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-60M-elements-geodist|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-60M-elements-geodist-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-60M-elements-geohash|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-60M-elements-geohash-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-60M-elements-geopos|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-60M-elements-geopos-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat-bybox|latency-rate-limited-10000_qps-memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-hash-hscan-50-fields-10B-values|latency-rate-limited-10000_qps-memtier_benchmark-1key-list-10-elements-lrange-all-elements|latency-rate-limited-10000_qps-memtier_benchmark-1key-list-10-elements-lrange-all-elements-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-list-100-elements-lrange-all-elements|latency-rate-limited-10000_qps-memtier_benchmark-1key-list-100-elements-lrange-all-elements-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-list-10K-elements-lindex-integer|latency-rate-limited-10000_qps-memtier_benchmark-1key-list-10K-elements-lindex-string|latency-rate-limited-10000_qps-memtier_benchmark-1key-list-1K-elements-lrange-all-elements|latency-rate-limited-10000_qps-memtier_benchmark-1key-list-1K-elements-lrange-all-elements-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-pfadd-4KB-values-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-10-elements-smembers|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-10-elements-smembers-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-10-elements-smismember|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-100-elements-sismember-is-a-member|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-100-elements-sismember-not-a-member|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-100-elements-smembers|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-100-elements-smismember|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-100-elements-sscan|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-10M-elements-sismember-50pct-chance|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-1K-elements-smembers|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-1M-elements-sismember-50pct-chance|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-200K-elements-sadd-constant|latency-rate-limited-10000_qps-memtier_benchmark-1key-set-2M-elements-sadd-increasing|latency-rate-limited-10000_qps-memtier_benchmark-1key-zincrby-1M-elements-pipeline-1|latency-rate-limited-10000_qps-memtier_benchmark-1key-zrank-1M-elements-pipeline-1|latency-rate-limited-10000_qps-memtier_benchmark-1key-zrem-5M-elements-pipeline-1|latency-rate-limited-10000_qps-memtier_benchmark-1key-zrevrangebyscore-256K-elements-pipeline-1|latency-rate-limited-10000_qps-memtier_benchmark-1key-zrevrank-1M-elements-pipeline-1|latency-rate-limited-10000_qps-memtier_benchmark-1key-zset-10-elements-zrange-all-elements|latency-rate-limited-10000_qps-memtier_benchmark-1key-zset-10-elements-zrange-all-elements-long-scores|latency-rate-limited-10000_qps-memtier_benchmark-1key-zset-100-elements-zrange-all-elements|latency-rate-limited-10000_qps-memtier_benchmark-1key-zset-100-elements-zrangebyscore-all-elements|latency-rate-limited-10000_qps-memtier_benchmark-1key-zset-100-elements-zrangebyscore-all-elements-long-scores|latency-rate-limited-10000_qps-memtier_benchmark-1key-zset-100-elements-zscan|latency-rate-limited-10000_qps-memtier_benchmark-1key-zset-1M-elements-zcard-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-1key-zset-1M-elements-zrevrange-5-elements|latency-rate-limited-10000_qps-memtier_benchmark-1key-zset-1M-elements-zscore-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-2keys-lua-eval-hset-expire|latency-rate-limited-10000_qps-memtier_benchmark-2keys-lua-evalsha-hset-expire|latency-rate-limited-10000_qps-memtier_benchmark-2keys-set-10-100-elements-sdiff|latency-rate-limited-10000_qps-memtier_benchmark-2keys-set-10-100-elements-sinter|latency-rate-limited-10000_qps-memtier_benchmark-2keys-set-10-100-elements-sunion|latency-rate-limited-10000_qps-memtier_benchmark-2keys-stream-5-entries-xread-all-entries|latency-rate-limited-10000_qps-memtier_benchmark-2keys-stream-5-entries-xread-all-entries-pipeline-10|latency-rate-limited-10000_qps-memtier_benchmark-3Mkeys-load-string-with-512B-values|latency-rate-limited-10000_qps-memtier_benchmark-connection-hello|latency-rate-limited-1000_qps-memtier_benchmark-10Kkeys-load-hash-50-fields-with-10000B-values|latency-rate-limited-1000_qps-memtier_benchmark-1Mkeys-load-zset-listpack-with-100-elements-double-score|latency-rate-limited-1000_qps-memtier_benchmark-1key-100M-bits-bitmap-bitcount|latency-rate-limited-1000_qps-memtier_benchmark-1key-list-10K-elements-linsert-lrem-integer|latency-rate-limited-1000_qps-memtier_benchmark-1key-list-10K-elements-linsert-lrem-string|latency-rate-limited-1000_qps-memtier_benchmark-1key-list-10K-elements-lpos-integer|latency-rate-limited-1000_qps-memtier_benchmark-1key-list-10K-elements-lpos-string|latency-rate-limited-1000_qps-memtier_benchmark-1key-list-2K-elements-quicklist-lrange-all-elements-longs|latency-rate-limited-1000_qps-memtier_benchmark-1key-zset-1K-elements-zrange-all-elements|latency-rate-limited-1000_qps-memtier_benchmark-2keys-zset-300-elements-skiplist-encoded-zunion|latency-rate-limited-1000_qps-memtier_benchmark-2keys-zset-300-elements-skiplist-encoded-zunionstore|latency-rate-limited-100_qps-memtier_benchmark-1key-1Billion-bits-bitmap-bitcount|memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values|memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values-pipeline-10|memtier_benchmark-1Mkeys-lhash-hexists|memtier_benchmark-1Mkeys-lhash-hincbry|memtier_benchmark-1Mkeys-load-string-with-200KiB-values|memtier_benchmark-1Mkeys-load-string-with-2MB-values|memtier_benchmark-1Mkeys-string-get-200KiB|memtier_benchmark-1Mkeys-string-get-20KiB|memtier_benchmark-1Mkeys-string-get-2MB|redis-benchmark-full-suite-1Mkeys-100B|redis-benchmark-full-suite-1Mkeys-100B-pipeline-10|redis-benchmark-full-suite-1Mkeys-1KiB|redis-benchmark-full-suite-1Mkeys-1KiB-pipeline-10|vector_db_benchmark_test |
|
To make the optimizations effective, we must merge both #13968 and #14017 at the same time. Here is a record of test results. The machine is
|
🎉 Snyk checks have passed. No issues have been found so far.✅ security/snyk check is complete. No issues have been found. (View Details) ✅ license/snyk check is complete. No issues have been found. (View Details) |
sundb
left a comment
There was a problem hiding this comment.
LGTM, just a minor naming
Some objects that are allocated in the IO thread, we should let IO thread free them, so we can avoid memory arena contention and also reduce the load of the main thread.
These objects include:
OBJ_ENCODING_RAWencoding strings, since only the type object is usually allocated by IO threads.For the implementation, if the client is assigned to IO threads, we will create a
deferred_objectsarray of size 32. We will put objects into thedeferred_objectswhen main thread wants to free above objects, and finally they are be freed by IO threads.