Skip to content

Conversation

@moticless
Copy link
Collaborator

No description provided.

@fcostaoliveira
Copy link
Collaborator

fcostaoliveira commented Jul 28, 2024

CE Performance Automation : step 1 of 2 (build) DONE.

This comment was automatically generated given a benchmark was triggered.
Started building at 2024-07-29 14:19:53.901060 and took 86 seconds.
You can check each build/benchmark progress in grafana:

  • git hash: ed10f73
  • git branch: unstable
  • commit date and time: 2024-06-11 21:42:34+08:00
  • commit summary: Add new hexpired notification for HFE (Add new hexpired notification for HFE #13329)
  • test filters:
    • command priority lower limit: 0
    • command priority upper limit: 100000
    • test name regex: .*
    • command group regex: .*

You can check a comparison in detail via the grafana link

@fcostaoliveira
Copy link
Collaborator

fcostaoliveira commented Jul 28, 2024

CE Performance Automation : step 2 of 2 (benchmark) RUNNING...

This comment was automatically generated given a benchmark was triggered.

Started benchmark suite at 2024-10-10 13:41:59.155427 and took 3396.898218 seconds up until now.
Status: [#########################-------------------------------------------------------] 31.73% completed.

In total will run 271 benchmarks.
- 185 pending.
- 86 completed:
- 0 successful.
- 86 failed.
You can check a the status in detail via the grafana link

@fcostaoliveira
Copy link
Collaborator

fcostaoliveira commented Jul 28, 2024

Automated performance analysis summary

This comment was automatically generated given there is performance data available.

Using platform named: intel64-ubuntu22.04-redis-icx1 to do the comparison.

In summary:

  • Detected a total of 90 stable tests between versions.
  • Detected a total of 2 highly unstable benchmarks.

You can check a comparison in detail via the grafana link

Comparison between unstable and unstable.

Time Period from 5 months ago. (environment used: oss-standalone)

Test Case Baseline unstable (median obs. +- std.dev) Comparison unstable (median obs. +- std.dev) % change (higher-better) Note
memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values 88011 88613 +- 1.0% (2 datapoints) 0.7% No Change
memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values-pipeline-10 123756 124786 +- 1.2% (2 datapoints) 0.8% No Change
memtier_benchmark-10Mkeys-load-hash-5-fields-with-100B-values 121280 122460 +- 1.4% (2 datapoints) 1.0% No Change
memtier_benchmark-10Mkeys-load-hash-5-fields-with-100B-values-pipeline-10 321328 333384 +- 5.1% (2 datapoints) 3.8% potential IMPROVEMENT
memtier_benchmark-10Mkeys-load-hash-5-fields-with-10B-values 137032 137354 +- 0.3% (2 datapoints) 0.2% No Change
memtier_benchmark-10Mkeys-load-hash-5-fields-with-10B-values-pipeline-10 389018 401261 +- 4.3% (2 datapoints) 3.1% potential IMPROVEMENT
memtier_benchmark-1Mkeys-100B-expire-use-case 179549 179058 +- 0.4% (2 datapoints) -0.3% No Change
memtier_benchmark-1Mkeys-10B-expire-use-case 180628 180390 +- 0.2% (2 datapoints) -0.1% No Change
memtier_benchmark-1Mkeys-1KiB-expire-use-case 175156 175993 +- 0.7% (2 datapoints) 0.5% No Change
memtier_benchmark-1Mkeys-4KiB-expire-use-case 165806 165771 +- 0.0% (2 datapoints) -0.0% No Change
memtier_benchmark-1Mkeys-bitmap-getbit-pipeline-10 944843 931339 +- 2.1% (2 datapoints) -1.4% No Change
memtier_benchmark-1Mkeys-generic-exists-pipeline-10 1002533 999462 +- 0.4% (2 datapoints) -0.3% No Change
memtier_benchmark-1Mkeys-generic-expire-pipeline-10 914338 905333 +- 1.4% (2 datapoints) -1.0% No Change
memtier_benchmark-1Mkeys-generic-expireat-pipeline-10 878927 885015 +- 1.0% (2 datapoints) 0.7% No Change
memtier_benchmark-1Mkeys-generic-pexpire-pipeline-10 904473 897115 +- 1.2% (2 datapoints) -0.8% No Change
memtier_benchmark-1Mkeys-generic-scan-pipeline-10 440800 451103 +- 3.2% (2 datapoints) 2.3% No Change
memtier_benchmark-1Mkeys-generic-touch-pipeline-10 1001299 996809 +- 0.6% (2 datapoints) -0.4% No Change
memtier_benchmark-1Mkeys-generic-ttl-pipeline-10 991118 979570 +- 1.7% (2 datapoints) -1.2% No Change
memtier_benchmark-1Mkeys-hash-hget-hgetall-hkeys-hvals-with-100B-values 181582 180888 +- 0.5% (2 datapoints) -0.4% No Change
memtier_benchmark-1Mkeys-hash-hmget-5-fields-with-100B-values-pipeline-10 712800 717881 +- 1.0% (2 datapoints) 0.7% No Change
memtier_benchmark-1Mkeys-hash-transactions-multi-exec-pipeline-20 1052282 1055447 +- 0.4% (2 datapoints) 0.3% No Change
memtier_benchmark-1Mkeys-lhash-hexists 164188 164332 +- 0.1% (2 datapoints) 0.1% No Change
memtier_benchmark-1Mkeys-lhash-hincbry 171981 171674 +- 0.3% (2 datapoints) -0.2% No Change
memtier_benchmark-1Mkeys-list-lpop-rpop-with-100B-values 179458 180904 +- 1.1% (2 datapoints) 0.8% No Change
memtier_benchmark-1Mkeys-list-lpop-rpop-with-10B-values 182726 181746 +- 0.8% (2 datapoints) -0.5% No Change
memtier_benchmark-1Mkeys-list-lpop-rpop-with-1KiB-values 181067 182004 +- 0.7% (2 datapoints) 0.5% No Change
memtier_benchmark-1Mkeys-load-hash-5-fields-with-1000B-values 105849 105191 +- 0.9% (2 datapoints) -0.6% No Change
memtier_benchmark-1Mkeys-load-hash-5-fields-with-1000B-values-pipeline-10 151637 150683 +- 0.9% (2 datapoints) -0.6% No Change
memtier_benchmark-1Mkeys-load-hash-hmset-5-fields-with-1000B-values 103604 104122 +- 0.7% (2 datapoints) 0.5% No Change
memtier_benchmark-1Mkeys-load-list-with-100B-values 150731 149770 +- 0.9% (2 datapoints) -0.6% No Change
memtier_benchmark-1Mkeys-load-list-with-10B-values 163335 165537 +- 1.9% (2 datapoints) 1.3% No Change
memtier_benchmark-1Mkeys-load-list-with-1KiB-values 112930 113315 +- 0.5% (2 datapoints) 0.3% No Change
memtier_benchmark-1Mkeys-load-stream-1-fields-with-100B-values 130597 131201 +- 0.7% (2 datapoints) 0.5% No Change
memtier_benchmark-1Mkeys-load-stream-1-fields-with-100B-values-pipeline-10 345904 352891 +- 2.8% (2 datapoints) 2.0% No Change
memtier_benchmark-1Mkeys-load-stream-5-fields-with-100B-values 104901 104386 +- 0.7% (2 datapoints) -0.5% No Change
memtier_benchmark-1Mkeys-load-stream-5-fields-with-100B-values-pipeline-10 218750 227593 +- 5.5% (2 datapoints) 4.0% potential IMPROVEMENT
memtier_benchmark-1Mkeys-load-string-with-100B-values 167769 167552 +- 0.2% (2 datapoints) -0.1% No Change
memtier_benchmark-1Mkeys-load-string-with-100B-values-pipeline-10 644718 646888 +- 0.5% (2 datapoints) 0.3% No Change
memtier_benchmark-1Mkeys-load-string-with-10B-values 174084 174891 +- 0.7% (2 datapoints) 0.5% No Change
memtier_benchmark-1Mkeys-load-string-with-10B-values-pipeline-10 722797 749988 +- 5.1% (2 datapoints) 3.8% potential IMPROVEMENT
memtier_benchmark-1Mkeys-load-string-with-1KiB-values 155812 155929 +- 0.1% (2 datapoints) 0.1% No Change
memtier_benchmark-1Mkeys-load-zset-with-10-elements-double-score 92373 92241 +- 0.2% (2 datapoints) -0.1% No Change
memtier_benchmark-1Mkeys-load-zset-with-10-elements-int-score 113318 114328 +- 1.3% (2 datapoints) 0.9% No Change
memtier_benchmark-1Mkeys-string-get-100B 180456 180685 +- 0.2% (2 datapoints) 0.1% No Change
memtier_benchmark-1Mkeys-string-get-100B-pipeline-10 1013284 991291 +- 3.1% (2 datapoints) -2.2% No Change
memtier_benchmark-1Mkeys-string-get-10B 181504 181183 +- 0.3% (2 datapoints) -0.2% No Change
memtier_benchmark-1Mkeys-string-get-10B-pipeline-10 1038395 1032554 +- 0.8% (2 datapoints) -0.6% No Change
memtier_benchmark-1Mkeys-string-get-1KiB 181583 180479 +- 0.9% (2 datapoints) -0.6% No Change
memtier_benchmark-1Mkeys-string-get-1KiB-pipeline-10 985500 979286 +- 0.9% (2 datapoints) -0.6% No Change
memtier_benchmark-1Mkeys-string-mget-1KiB 122284 121950 +- 0.4% (2 datapoints) -0.3% No Change
memtier_benchmark-1key-geo-60M-elements-geodist 180662 179404 +- 1.0% (2 datapoints) -0.7% No Change
memtier_benchmark-1key-geo-60M-elements-geodist-pipeline-10 1027423 1023797 +- 0.5% (2 datapoints) -0.4% No Change
memtier_benchmark-1key-geo-60M-elements-geohash 182360 180808 +- 1.2% (2 datapoints) -0.9% No Change
memtier_benchmark-1key-geo-60M-elements-geohash-pipeline-10 1044698 1056458 +- 1.6% (2 datapoints) 1.1% No Change
memtier_benchmark-1key-geo-60M-elements-geopos 182080 181808 +- 0.2% (2 datapoints) -0.1% No Change
memtier_benchmark-1key-geo-60M-elements-geopos-pipeline-10 1063273 1038180 +- 3.4% (2 datapoints) -2.4% No Change
memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat 142904 142614 +- 0.3% (2 datapoints) -0.2% No Change
memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat-bybox 141365 141008 +- 0.4% (2 datapoints) -0.3% No Change
memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat-pipeline-10 563680 560698 +- 0.8% (2 datapoints) -0.5% No Change
memtier_benchmark-1key-list-10-elements-lrange-all-elements 167210 166132 +- 0.9% (2 datapoints) -0.6% No Change
memtier_benchmark-1key-list-100-elements-lrange-all-elements 93967 92657 +- 2.0% (2 datapoints) -1.4% No Change
memtier_benchmark-1key-list-1K-elements-lrange-all-elements 15106 14965 +- 1.3% (2 datapoints) -0.9% No Change
memtier_benchmark-1key-pfadd-4KB-values-pipeline-10 263706 264156 +- 0.2% (2 datapoints) 0.2% No Change
memtier_benchmark-1key-set-10-elements-smembers 140580 138916 +- 1.7% (2 datapoints) -1.2% No Change
memtier_benchmark-1key-set-10-elements-smembers-pipeline-10 446849 431019 +- 5.2% (2 datapoints) -3.5% potential REGRESSION
memtier_benchmark-1key-set-10-elements-smismember 172227 173545 +- 1.1% (2 datapoints) 0.8% No Change
memtier_benchmark-1key-set-100-elements-smembers 74360 74367 +- 0.0% (2 datapoints) 0.0% No Change
memtier_benchmark-1key-set-100-elements-smismember 153933 155682 +- 1.6% (2 datapoints) 1.1% No Change
memtier_benchmark-1key-set-1K-elements-smembers 11698 11730 +- 0.4% (2 datapoints) 0.3% No Change
memtier_benchmark-1key-set-200K-elements-sadd-constant 182458 181070 +- 1.1% (2 datapoints) -0.8% No Change
memtier_benchmark-1key-set-2M-elements-sadd-increasing 166204 166364 +- 0.1% (2 datapoints) 0.1% No Change
memtier_benchmark-1key-zincrby-1M-elements-pipeline-1 45871 46071 +- 0.6% (2 datapoints) 0.4% No Change
memtier_benchmark-1key-zrank-1M-elements-pipeline-1 48291 48429 +- 0.4% (2 datapoints) 0.3% No Change
memtier_benchmark-1key-zrem-5M-elements-pipeline-1 48659 48528 +- 0.4% (2 datapoints) -0.3% No Change
memtier_benchmark-1key-zrevrangebyscore-256K-elements-pipeline-1 101623 100580 +- 1.5% (2 datapoints) -1.0% No Change
memtier_benchmark-1key-zrevrank-1M-elements-pipeline-1 47945 48621 +- 2.0% (2 datapoints) 1.4% No Change
memtier_benchmark-1key-zset-10-elements-zrange-all-elements 83829 83485 +- 0.6% (2 datapoints) -0.4% No Change
memtier_benchmark-1key-zset-10-elements-zrange-all-elements-long-scores 117981 118673 +- 0.8% (2 datapoints) 0.6% No Change
memtier_benchmark-1key-zset-100-elements-zrange-all-elements 18855 19009 +- 1.1% (2 datapoints) 0.8% No Change
memtier_benchmark-1key-zset-100-elements-zrangebyscore-all-elements 18892 18957 +- 0.5% (2 datapoints) 0.3% No Change
memtier_benchmark-1key-zset-100-elements-zrangebyscore-all-elements-long-scores 171026 171693 +- 0.5% (2 datapoints) 0.4% No Change
memtier_benchmark-1key-zset-1K-elements-zrange-all-elements 3581 3566 +- 0.6% (2 datapoints) -0.4% No Change
memtier_benchmark-1key-zset-1M-elements-zcard-pipeline-10 1070597 1067713 +- 0.4% (2 datapoints) -0.3% No Change
memtier_benchmark-1key-zset-1M-elements-zrevrange-5-elements 169991 169195 +- 0.7% (2 datapoints) -0.5% No Change
memtier_benchmark-1key-zset-1M-elements-zscore-pipeline-10 915358 914338 +- 0.2% (2 datapoints) -0.1% No Change
memtier_benchmark-2keys-lua-eval-hset-expire 89384 88346 +- 1.7% (2 datapoints) -1.2% No Change
memtier_benchmark-2keys-lua-evalsha-hset-expire 104161 103273 +- 1.2% (2 datapoints) -0.9% No Change
memtier_benchmark-2keys-set-10-100-elements-sdiff 14235 15418 +- 10.9% UNSTABLE (2 datapoints) 8.3% UNSTABLE (very high variance)
memtier_benchmark-2keys-set-10-100-elements-sinter 81468 83601 +- 3.6% (2 datapoints) 2.6% No Change
memtier_benchmark-2keys-set-10-100-elements-sunion 13759 14942 +- 11.2% UNSTABLE (2 datapoints) 8.6% UNSTABLE (very high variance)
memtier_benchmark-2keys-stream-5-entries-xread-all-entries 75205 75764 +- 0.5% (4 datapoints) 0.7% No Change
memtier_benchmark-2keys-stream-5-entries-xread-all-entries-pipeline-10 117386 119222 +- 1.5% (4 datapoints) 1.6% No Change

@fcostaoliveira fcostaoliveira added action:run-benchmark Triggers the benchmark suite for this Pull Request and removed action:run-benchmark Triggers the benchmark suite for this Pull Request labels Jul 29, 2024
@moticless moticless merged commit 569584d into redis:unstable Sep 5, 2024
@moticless moticless deleted the hfe-simplify-hgetall branch September 5, 2024 09:48
@sundb sundb added this to Redis 8.0 Aug 15, 2025
@sundb sundb removed this from Redis 8.2 Aug 15, 2025
@sundb sundb moved this from Todo to Done in Redis 8.0 Aug 15, 2025
funny-dog pushed a commit to funny-dog/redis that referenced this pull request Sep 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

action:run-benchmark Triggers the benchmark suite for this Pull Request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants