Skip to content

[CRASH] valkey primary crashed #3569

Description

@ehooi

Crash report

2581:M 24 Apr 2026 11:53:12.665 * Replica 172.20.0.243:6379 asks for synchronization
2581:M 24 Apr 2026 11:53:12.666 * Full resync requested by replica 172.20.0.243:6379
2581:M 24 Apr 2026 11:53:12.666 * Delay next BGSAVE for diskless SYNC
2581:M 24 Apr 2026 11:53:13.396 * Starting BGSAVE for SYNC with target: replicas sockets using: normal sync
2581:M 24 Apr 2026 11:53:13.421 * Background RDB transfer started by pid 3341 to pipe through parent process
3341:C 24 Apr 2026 11:53:20.867 * Fork CoW for RDB: current 116 MB, peak 116 MB, average 82 MB
2581:M 24 Apr 2026 11:53:20.868 * Diskless rdb transfer, done reading from pipe, 1 replicas still up.
2581:M 24 Apr 2026 11:53:20.937 * Background RDB transfer terminated with success
2581:M 24 Apr 2026 11:53:20.937 * Streamed RDB transfer with replica 172.20.0.243:6379 succeeded (socket). Waiting for REPLCONF ACK from replica to enable streaming
2581:M 24 Apr 2026 11:53:20.937 * Synchronization with replica 172.20.0.243:6379 succeeded

=== VALKEY BUG REPORT START: Cut & paste starting from here ===
2581:M 26 Apr 2026 08:00:36.605 # === ASSERTION FAILED ===
2581:M 26 Apr 2026 08:00:36.605 # ==> listpack.c:1256 'lpValidateNext(lp, &p, lpbytes)' is not true

------ INFO OUTPUT ------
# Server
redis_version:7.2.4
server_name:valkey
valkey_version:9.0.3
valkey_release_stage:ga
redis_git_sha1:00000000
redis_git_dirty:1
redis_build_id:ff9b530358b4b831
server_mode:standalone
os:Linux 6.18.7-0-virt aarch64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
gcc_version:15.2.0
process_id:2581
process_supervised:no
run_id:563fe610ccacd98fd5efad39cc75f21fb9abb65d
tcp_port:6379
server_time_usec:1777190436605784
uptime_in_seconds:281111
uptime_in_days:3
hz:10
configured_hz:10
clients_hz:10
lru_clock:15582756
executable:/usr/bin/valkey-server
config_file:/etc/valkey/server.conf
io_threads_active:0
availability_zone:
listener0:name=tcp,bind=0.0.0.0,port=6379

# Clients
connected_clients:17
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:16390
client_recent_max_output_buffer:16577
blocked_clients:0
tracking_clients:0
pubsub_clients:3
watching_clients:0
clients_in_timeout_table:0
total_watched_keys:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0
paused_reason:none
paused_actions:none
paused_timeout_milliseconds:0

# Memory
used_memory:961465176
used_memory_human:916.92M
used_memory_rss:1202921472
used_memory_rss_human:1.12G
used_memory_peak:1234078805
used_memory_peak_human:1.15G
used_memory_peak_perc:77.91%
used_memory_overhead:140130631
used_memory_startup:899293
used_memory_dataset:821334545
used_memory_dataset_perc:85.51%
allocator_allocated:961415769
allocator_active:1202846720
allocator_resident:1202846720
allocator_muzzy:0
total_system_memory:4028239872
total_system_memory_human:3.75G
used_memory_lua:38912
used_memory_vm_eval:38912
used_memory_lua_human:38.00K
used_memory_scripts_eval:1136
number_of_cached_scripts:4
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:35840
used_memory_vm_total:74752
used_memory_vm_total_human:73.00K
used_memory_functions:237
used_memory_scripts:1373
used_memory_scripts_human:1.34K
maxmemory:2684354560
maxmemory_human:2.50G
maxmemory_policy:noeviction
allocator_frag_ratio:1.00
allocator_frag_bytes:0
allocator_rss_ratio:1.00
allocator_rss_bytes:0
rss_overhead_ratio:1.00
rss_overhead_bytes:74752
mem_fragmentation_ratio:1.25
mem_fragmentation_bytes:241505703
mem_not_counted_for_evict:16384
mem_replication_backlog:10485928
mem_total_replication_buffers:10543168
mem_clients_slaves:57408
mem_clients_normal:231493
mem_cluster_links:0
mem_cluster_slot_import:0
mem_cluster_slot_export:0
mem_aof_buffer:0
mem_allocator:libc
mem_overhead_db_hashtable_rehashing:0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:320853877
rdb_bgsave_in_progress:0
rdb_last_save_time:1776909325
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:7
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:122023936
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:2997865
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
slot_migration_fork_in_progress:0

# Stats
total_connections_received:164
total_commands_processed:373440946
instantaneous_ops_per_sec:2373
total_net_input_bytes:74082373375
total_net_output_bytes:129856353553
total_net_repl_input_bytes:31605877312
total_net_repl_output_bytes:38559914645
total_net_cluster_slot_import_bytes:0
total_net_cluster_slot_export_bytes:0
instantaneous_input_kbps:389.13
instantaneous_output_kbps:1162.18
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:344.25
rejected_connections:0
sync_full:1
sync_partial_ok:1
sync_partial_err:0
expired_keys:3058506
expired_fields:0
expired_stale_perc:3.29
expired_keys_with_volatile_items_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:20697
evicted_keys:0
evicted_clients:0
evicted_scripts:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:180554343
keyspace_misses:6008307
pubsub_channels:1
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:25401
total_forks:1
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:15
dump_payload_sanitizations:0
total_reads_processed:259444131
total_writes_processed:280130429
io_threaded_reads_processed:0
io_threaded_writes_processed:0
io_threaded_freed_objects:0
io_threaded_accept_processed:0
io_threaded_poll_processed:0
io_threaded_total_prefetch_batches:0
io_threaded_total_prefetch_entries:0
client_query_buffer_limit_disconnections:0
client_output_buffer_limit_disconnections:0
reply_buffer_shrinks:220334
reply_buffer_expands:359879
eventloop_cycles:257537423
eventloop_duration_sum:9005314099
eventloop_duration_cmd_sum:2370990308
instantaneous_eventloop_cycles_per_sec:1697
instantaneous_eventloop_duration_usec:33
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
acl_access_denied_tls_cert:0

# Replication
role:master
connected_slaves:1
slave0:ip=172.20.0.243,port=6379,state=online,offset=11595923561689,lag=0,type=replica
replicas_waiting_psync:0
master_failover_state:no-failover
master_replid:24c8375a4b600444a533c8e58dbf76813945d0c3
master_replid2:0a0f5a81950f264a96a44524f98b58d5bcfd390b
master_repl_offset:11595923635973
second_repl_offset:11557941852840
repl_backlog_active:1
repl_backlog_size:10485760
repl_backlog_first_byte_offset:11595913145894
repl_backlog_histlen:10490080

# CPU
used_cpu_sys:4263.586352
used_cpu_user:6476.110293
used_cpu_sys_children:0.628432
used_cpu_user_children:5.603736
used_cpu_sys_main_thread:4263.586338
used_cpu_user_main_thread:6476.110271

# Modules

# Commandstats
cmdstat_info:calls=84098,usec=8378892,usec_per_call=99.63,rejected_calls=0,failed_calls=0
cmdstat_zadd:calls=98492534,usec=1059418318,usec_per_call=10.76,rejected_calls=0,failed_calls=0
cmdstat_getex:calls=7580219,usec=32441006,usec_per_call=4.28,rejected_calls=0,failed_calls=0
cmdstat_mget:calls=3315111,usec=36513277,usec_per_call=11.01,rejected_calls=0,failed_calls=0
cmdstat_lpop:calls=1768,usec=3487,usec_per_call=1.97,rejected_calls=0,failed_calls=0
cmdstat_xtrim:calls=78,usec=5668646,usec_per_call=72674.95,rejected_calls=0,failed_calls=0
cmdstat_llen:calls=1768,usec=4103,usec_per_call=2.32,rejected_calls=0,failed_calls=0
cmdstat_select:calls=1,usec=1,usec_per_call=1.00,rejected_calls=0,failed_calls=0
cmdstat_set:calls=21261503,usec=96360444,usec_per_call=4.53,rejected_calls=0,failed_calls=0
cmdstat_replconf:calls=158242,usec=242629,usec_per_call=1.53,rejected_calls=0,failed_calls=0
cmdstat_setnx:calls=787258,usec=3439259,usec_per_call=4.37,rejected_calls=0,failed_calls=0
cmdstat_ping:calls=824122,usec=1241364,usec_per_call=1.51,rejected_calls=13,failed_calls=0
cmdstat_psync:calls=2,usec=290,usec_per_call=145.00,rejected_calls=0,failed_calls=0
cmdstat_multi:calls=4494,usec=2293,usec_per_call=0.51,rejected_calls=0,failed_calls=0
cmdstat_ttl:calls=21,usec=34,usec_per_call=1.62,rejected_calls=0,failed_calls=0
cmdstat_expire:calls=46859230,usec=116351021,usec_per_call=2.48,rejected_calls=0,failed_calls=0
cmdstat_zremrangebyscore:calls=4757,usec=956124,usec_per_call=200.99,rejected_calls=0,failed_calls=0
cmdstat_xadd:calls=42014899,usec=401711295,usec_per_call=9.56,rejected_calls=0,failed_calls=0
cmdstat_publish:calls=593037,usec=5125430,usec_per_call=8.64,rejected_calls=0,failed_calls=0
cmdstat_subscribe:calls=13,usec=129,usec_per_call=9.92,rejected_calls=0,failed_calls=0
cmdstat_eval:calls=67,usec=2606,usec_per_call=38.90,rejected_calls=0,failed_calls=0
cmdstat_config|rewrite:calls=1,usec=5453,usec_per_call=5453.00,rejected_calls=0,failed_calls=0
cmdstat_pexpire:calls=105906,usec=265025,usec_per_call=2.50,rejected_calls=0,failed_calls=0
cmdstat_get:calls=59210017,usec=149945129,usec_per_call=2.53,rejected_calls=0,failed_calls=0
cmdstat_xread:calls=21171186,usec=436369528,usec_per_call=20.61,rejected_calls=0,failed_calls=0
cmdstat_client|kill:calls=2,usec=118,usec_per_call=59.00,rejected_calls=0,failed_calls=0
cmdstat_client|setinfo:calls=246,usec=141,usec_per_call=0.57,rejected_calls=0,failed_calls=0
cmdstat_client|unpause:calls=1,usec=1,usec_per_call=1.00,rejected_calls=0,failed_calls=0
cmdstat_client|setname:calls=26,usec=13,usec_per_call=0.50,rejected_calls=0,failed_calls=0
cmdstat_role:calls=46,usec=410,usec_per_call=8.91,rejected_calls=0,failed_calls=0
cmdstat_xinfo|stream:calls=15901,usec=262472,usec_per_call=16.51,rejected_calls=0,failed_calls=0
cmdstat_hincrby:calls=1646926,usec=9850025,usec_per_call=5.98,rejected_calls=0,failed_calls=0
cmdstat_evalsha:calls=113414,usec=3857581,usec_per_call=34.01,rejected_calls=0,failed_calls=0
cmdstat_exec:calls=4494,usec=69928,usec_per_call=15.56,rejected_calls=0,failed_calls=0
cmdstat_unlink:calls=2432964,usec=11293099,usec_per_call=4.64,rejected_calls=0,failed_calls=0
cmdstat_hget:calls=2048,usec=5540,usec_per_call=2.71,rejected_calls=0,failed_calls=0
cmdstat_auth:calls=151,usec=789,usec_per_call=5.23,rejected_calls=0,failed_calls=0
cmdstat_del:calls=5412,usec=27456,usec_per_call=5.07,rejected_calls=0,failed_calls=0
cmdstat_zcard:calls=2675,usec=4953,usec_per_call=1.85,rejected_calls=0,failed_calls=0
cmdstat_pexpireat:calls=41278497,usec=71935496,usec_per_call=1.74,rejected_calls=0,failed_calls=0
cmdstat_xrange:calls=4631,usec=50329932,usec_per_call=10868.05,rejected_calls=0,failed_calls=0
cmdstat_setex:calls=25463168,usec=149478878,usec_per_call=5.87,rejected_calls=0,failed_calls=0
cmdstat_acl|log:calls=3,usec=9,usec_per_call=3.00,rejected_calls=0,failed_calls=0
cmdstat_acl|list:calls=1,usec=98,usec_per_call=98.00,rejected_calls=0,failed_calls=0
cmdstat_command|docs:calls=8,usec=13975,usec_per_call=1746.88,rejected_calls=0,failed_calls=0

# Errorstats
errorstat_ERR:count=2
errorstat_LOADING:count=13

# Latencystats
latency_percentiles_usec_info:p50=98.303,p99=143.359,p99.9=173.055
latency_percentiles_usec_zadd:p50=9.023,p99=30.079,p99.9=43.007
latency_percentiles_usec_getex:p50=4.015,p99=12.031,p99.9=23.039
latency_percentiles_usec_mget:p50=3.007,p99=39.167,p99.9=214.015
latency_percentiles_usec_lpop:p50=2.007,p99=6.015,p99.9=15.039
latency_percentiles_usec_xtrim:p50=72351.743,p99=139460.607,p99.9=186646.527
latency_percentiles_usec_llen:p50=2.007,p99=10.047,p99.9=30.079
latency_percentiles_usec_select:p50=1.003,p99=1.003,p99.9=1.003
latency_percentiles_usec_set:p50=4.015,p99=18.047,p99.9=27.007
latency_percentiles_usec_replconf:p50=1.003,p99=3.007,p99.9=16.063
latency_percentiles_usec_setnx:p50=4.015,p99=11.007,p99.9=23.039
latency_percentiles_usec_ping:p50=1.003,p99=4.015,p99.9=16.063
latency_percentiles_usec_psync:p50=92.159,p99=198.655,p99.9=198.655
latency_percentiles_usec_multi:p50=0.001,p99=1.003,p99.9=11.007
latency_percentiles_usec_ttl:p50=2.007,p99=3.007,p99.9=3.007
latency_percentiles_usec_expire:p50=2.007,p99=7.007,p99.9=19.071
latency_percentiles_usec_zremrangebyscore:p50=195.583,p99=444.415,p99.9=1409.023
latency_percentiles_usec_xadd:p50=9.023,p99=29.055,p99.9=46.079
latency_percentiles_usec_publish:p50=9.023,p99=27.007,p99.9=41.215
latency_percentiles_usec_subscribe:p50=10.047,p99=17.023,p99.9=17.023
latency_percentiles_usec_eval:p50=36.095,p99=90.111,p99.9=106.495
latency_percentiles_usec_config|rewrite:p50=5472.255,p99=5472.255,p99.9=5472.255
latency_percentiles_usec_pexpire:p50=2.007,p99=5.023,p99.9=20.095
latency_percentiles_usec_get:p50=2.007,p99=6.015,p99.9=19.071
latency_percentiles_usec_xread:p50=20.095,p99=41.215,p99.9=55.039
latency_percentiles_usec_client|kill:p50=47.103,p99=71.167,p99.9=71.167
latency_percentiles_usec_client|setinfo:p50=1.003,p99=2.007,p99.9=3.007
latency_percentiles_usec_client|unpause:p50=1.003,p99=1.003,p99.9=1.003
latency_percentiles_usec_client|setname:p50=0.001,p99=1.003,p99.9=1.003
latency_percentiles_usec_role:p50=7.007,p99=52.223,p99.9=52.223
latency_percentiles_usec_xinfo|stream:p50=16.063,p99=33.023,p99.9=48.127
latency_percentiles_usec_hincrby:p50=5.023,p99=19.071,p99.9=29.055
latency_percentiles_usec_evalsha:p50=30.079,p99=157.695,p99.9=185.343
latency_percentiles_usec_exec:p50=13.055,p99=40.191,p99.9=85.503
latency_percentiles_usec_unlink:p50=4.015,p99=16.063,p99.9=25.087
latency_percentiles_usec_hget:p50=2.007,p99=8.031,p99.9=19.071
latency_percentiles_usec_auth:p50=5.023,p99=10.047,p99.9=23.039
latency_percentiles_usec_del:p50=5.023,p99=18.047,p99.9=31.103
latency_percentiles_usec_zcard:p50=2.007,p99=5.023,p99.9=31.103
latency_percentiles_usec_pexpireat:p50=1.003,p99=12.031,p99.9=18.047
latency_percentiles_usec_xrange:p50=10878.975,p99=12189.695,p99.9=14155.775
latency_percentiles_usec_setex:p50=5.023,p99=16.063,p99.9=28.031
latency_percentiles_usec_acl|log:p50=3.007,p99=3.007,p99.9=3.007
latency_percentiles_usec_acl|list:p50=98.303,p99=98.303,p99.9=98.303
latency_percentiles_usec_command|docs:p50=1720.319,p99=1884.159,p99.9=1884.159

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=2943109,expires=2943103,avg_ttl=283216917,keys_with_volatile_items=0

------ CLIENT LIST OUTPUT ------
id=44 addr=172.20.2.8:59660 laddr=172.20.0.86:6379 fd=24 name=*redacted* age=159196 idle=1786 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1600 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=4594 tot-net-out=8562 tot-cmds=70
id=113 addr=172.20.2.213:35376 laddr=172.20.0.86:6379 fd=12 name=*redacted* age=75753 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=5540 rbp=4200 obl=0 oll=0 omem=0 tot-mem=22506 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=3083606685 tot-net-out=5544776665 tot-cmds=17247234
id=142 addr=172.20.1.207:59328 laddr=172.20.0.86:6379 fd=11 name=*redacted* age=34228 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=43950 argv-mem=0 multi-mem=0 rbs=11872 rbp=5936 obl=0 oll=0 omem=0 tot-mem=56404 events=r cmd=setex user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=1456936012 tot-net-out=2616014074 tot-cmds=8081515
id=48 addr=172.20.0.243:38866 laddr=172.20.0.86:6379 fd=14 name=*redacted* age=158844 idle=0 flags=S capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=2 omem=32896 tot-mem=34496 events=r cmd=replconf user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=7581357 tot-net-out=37842763236 tot-cmds=157945
id=147 addr=172.20.1.49:59788 laddr=172.20.0.86:6379 fd=17 name=*redacted* age=30339 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=5792 rbp=4061 obl=0 oll=0 omem=0 tot-mem=22758 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=1234804927 tot-net-out=2233259403 tot-cmds=6878161
id=49 addr=172.20.0.243:38868 laddr=172.20.0.86:6379 fd=19 name=*redacted* age=158844 idle=1 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=2048 rbp=1024 obl=0 oll=0 omem=0 tot-mem=2624 events=r cmd=ping user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=13195572 tot-net-out=105693674 tot-cmds=246756
id=148 addr=172.20.6.60:38300 laddr=172.20.0.86:6379 fd=15 name=*redacted* age=28583 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=5816 rbp=3987 obl=0 oll=0 omem=0 tot-mem=22782 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=1139205888 tot-net-out=2062298073 tot-cmds=6354207
id=158 addr=172.20.5.232:38232 laddr=172.20.0.86:6379 fd=18 name=*redacted* age=14123 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=5888 rbp=4041 obl=0 oll=0 omem=0 tot-mem=22854 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=523488029 tot-net-out=958223146 tot-cmds=2914435
id=50 addr=172.20.0.243:38882 laddr=172.20.0.86:6379 fd=20 name=*redacted* age=158844 idle=0 flags=P capa= db=0 sub=1 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=140 obl=0 oll=0 omem=0 tot-mem=1880 events=r cmd=subscribe user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=188 tot-net-out=32492093 tot-cmds=3
id=139 addr=172.20.2.23:41878 laddr=172.20.0.86:6379 fd=8 name=*redacted* age=37462 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=6033 qbuf-free=10351 argv-mem=5932 multi-mem=0 rbs=6016 rbp=4031 obl=22 oll=0 omem=0 tot-mem=29042 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=1550389371 tot-net-out=2773160132 tot-cmds=8594740
id=43 addr=172.20.4.51:40162 laddr=172.20.0.86:6379 fd=23 name=*redacted* age=159202 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=54 multi-mem=0 rbs=2048 rbp=1024 obl=0 oll=0 omem=0 tot-mem=2742 events=r cmd=xread user=*redacted* redir=-1 resp=2 lib-name=redis-rs lib-ver=0.27.6 tot-net-in=2287195652 tot-net-out=9196137560 tot-cmds=21187090
id=149 addr=172.20.6.56:52714 laddr=172.20.0.86:6379 fd=16 name=*redacted* age=28473 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=7870 rbp=4261 obl=0 oll=0 omem=0 tot-mem=24836 events=r cmd=setex user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=1173441136 tot-net-out=2120929491 tot-cmds=6546702
id=94 addr=172.20.3.218:47322 laddr=172.20.0.86:6379 fd=22 name=*redacted* age=106630 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=6032 rbp=3016 obl=0 oll=0 omem=0 tot-mem=22998 events=r cmd=getex user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=4289430317 tot-net-out=7715505107 tot-cmds=24018283
id=52 addr=172.20.4.51:36688 laddr=172.20.0.86:6379 fd=25 name=*redacted* age=158611 idle=3569 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1600 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=redis-rs lib-ver=0.27.6 tot-net-in=31540 tot-net-out=2567 tot-cmds=119
id=66 addr=172.20.0.86:53250 laddr=172.20.0.86:6379 fd=10 name=*redacted* age=141450 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=1600 events=r cmd=ping user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=11683125 tot-net-out=94132857 tot-cmds=219762
id=67 addr=172.20.0.86:53254 laddr=172.20.0.86:6379 fd=13 name=*redacted* age=141450 idle=0 flags=P capa= db=0 sub=1 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=140 obl=0 oll=0 omem=0 tot-mem=1880 events=r cmd=subscribe user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=188 tot-net-out=28934811 tot-cmds=3
id=64 addr=172.20.17.192:60756 laddr=172.20.0.86:6379 fd=7 name=*redacted* age=141470 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=1600 events=r cmd=ping user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=11815567 tot-net-out=94138380 tot-cmds=219604
id=65 addr=172.20.17.192:60770 laddr=172.20.0.86:6379 fd=9 name=*redacted* age=141470 idle=0 flags=P capa= db=0 sub=1 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=140 obl=0 oll=0 omem=0 tot-mem=1880 events=r cmd=subscribe user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=188 tot-net-out=28938704 tot-cmds=3

------ CURRENT CLIENT INFO ------
id=43 addr=172.20.4.51:40162 laddr=172.20.0.86:6379 fd=23 name=*redacted* age=159202 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=54 multi-mem=0 rbs=2048 rbp=1024 obl=0 oll=0 omem=0 tot-mem=2742 events=r cmd=xread user=*redacted* redir=-1 resp=2 lib-name=redis-rs lib-ver=0.27.6 tot-net-in=2287195652 tot-net-out=9196137560 tot-cmds=21187090
argc: 8
argv[0]: "XREAD"
argv[1]: 5 bytes
argv[2]: 4 bytes
argv[3]: 5 bytes
argv[4]: 3 bytes
argv[5]: 7 bytes
argv[6]: 10 bytes
argv[7]: 15 bytes

------ EXECUTING CLIENT INFO ------
id=43 addr=172.20.4.51:40162 laddr=172.20.0.86:6379 fd=23 name=*redacted* age=159202 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=54 multi-mem=0 rbs=2048 rbp=1024 obl=0 oll=0 omem=0 tot-mem=2742 events=r cmd=xread user=*redacted* redir=-1 resp=2 lib-name=redis-rs lib-ver=0.27.6 tot-net-in=2287195652 tot-net-out=9196137560 tot-cmds=21187090
argc: 8
argv[0]: "XREAD"
argv[1]: 5 bytes
argv[2]: 4 bytes
argv[3]: 5 bytes
argv[4]: 3 bytes
argv[5]: 7 bytes
argv[6]: 10 bytes
argv[7]: 15 bytes

------ MODULES INFO OUTPUT ------

------ CONFIG DEBUG OUTPUT ------
slave-read-only yes
lazyfree-lazy-expire yes
lazyfree-lazy-user-del yes
list-compress-depth 0
proto-max-bulk-len 512mb
import-mode no
debug-context ""
lazyfree-lazy-server-del yes
repl-diskless-sync yes
sanitize-dump-payload no
lazyfree-lazy-user-flush yes
io-threads 1
replica-read-only yes
activedefrag no
repl-diskless-load on-empty-db
client-query-buffer-limit 1gb
dual-channel-replication-enabled no
lazyfree-lazy-eviction yes

------ FAST MEMORY TEST ------
2581:M 26 Apr 2026 08:00:36.606 # Bio worker thread #0 terminated
2581:M 26 Apr 2026 08:00:36.606 # Bio worker thread #1 terminated
2581:M 26 Apr 2026 08:00:36.607 # Bio worker thread #2 terminated
2581:M 26 Apr 2026 08:00:36.607 # Bio worker thread #3 terminated
*** Preparing to test memory region aaaabeafb000 (217088 bytes)
*** Preparing to test memory region aaaaede97000 (8671232 bytes)
*** Preparing to test memory region ffff57e29000 (16384 bytes)
*** Preparing to test memory region ffff57e35000 (32768 bytes)
*** Preparing to test memory region ffff57eed000 (16384 bytes)
*** Preparing to test memory region ffff57fd1000 (16384 bytes)
......
*** Preparing to test memory region ffffa1d4f000 (274432 bytes)
*** Preparing to test memory region ffffa1d96000 (12005376 bytes)
*** Preparing to test memory region ffffa2911000 (4812800 bytes)
*** Preparing to test memory region ffffa2db8000 (12361728 bytes)
*** Preparing to test memory region ffffa3985000 (8015872 bytes)
*** Preparing to test memory region ffffa413a000 (3571712 bytes)
*** Preparing to test memory region ffffa44a3000 (1957888 bytes)
*** Preparing to test memory region ffffa4684000 (2334720 bytes)
*** Preparing to test memory region ffffa48c1000 (1839104 bytes)
*** Preparing to test memory region ffffa4a85000 (5296128 bytes)
*** Preparing to test memory region ffffa4f96000 (352256 bytes)
*** Preparing to test memory region ffffa4fef000 (1769472 bytes)
*** Preparing to test memory region ffffa51a2000 (999424 bytes)
*** Preparing to test memory region ffffa5299000 (6553600 bytes)
*** Preparing to test memory region ffffa58dd000 (2863104 bytes)
*** Preparing to test memory region ffffa5ba8000 (1593344 bytes)
*** Preparing to test memory region ffffa5d34000 (1114112 bytes)
*** Preparing to test memory region ffffa5e48000 (4931584 bytes)
*** Preparing to test memory region ffffa6300000 (1298432 bytes)
*** Preparing to test memory region ffffa6445000 (278528 bytes)
*** Preparing to test memory region ffffa6490000 (2007040 bytes)
*** Preparing to test memory region ffffa667e000 (679936 bytes)
*** Preparing to test memory region ffffa672c000 (3444736 bytes)
*** Preparing to test memory region ffffa6a79000 (630784 bytes)
*** Preparing to test memory region ffffa6b1b000 (3297280 bytes)
*** Preparing to test memory region ffffa6e48000 (1609728 bytes)
*** Preparing to test memory region ffffa6fd5000 (2957312 bytes)
*** Preparing to test memory region ffffa72af000 (1200128 bytes)
*** Preparing to test memory region ffffa73d9000 (1265664 bytes)
*** Preparing to test memory region ffffa7512000 (1212416 bytes)
*** Preparing to test memory region ffffa763b000 (4075520 bytes)
*** Preparing to test memory region ffffa7a26000 (4112384 bytes)
*** Preparing to test memory region ffffa7e16000 (430080 bytes)
*** Preparing to test memory region ffffa7e86000 (393216 bytes)
*** Preparing to test memory region ffffa7eea000 (1093632 bytes)
*** Preparing to test memory region ffffa7ffd000 (180224 bytes)
*** Preparing to test memory region ffffa8031000 (2871296 bytes)
*** Preparing to test memory region ffffa82f2000 (1409024 bytes)
*** Preparing to test memory region ffffa8452000 (1085440 bytes)
*** Preparing to test memory region ffffa855d000 (958464 bytes)
*** Preparing to test memory region ffffa864f000 (282624 bytes)
*** Preparing to test memory region ffffa8697000 (602112 bytes)
*** Preparing to test memory region ffffa872c000 (868352 bytes)
*** Preparing to test memory region ffffa8801000 (86016 bytes)
*** Preparing to test memory region ffffa881a000 (2281472 bytes)
*** Preparing to test memory region ffffa8a4e000 (724992 bytes)
*** Preparing to test memory region ffffa8b07000 (487424 bytes)
*** Preparing to test memory region ffffa8b80000 (774144 bytes)
*** Preparing to test memory region ffffa8c43000 (1081344 bytes)
*** Preparing to test memory region ffffa8d53000 (1236992 bytes)
*** Preparing to test memory region ffffa8e84000 (454656 bytes)
*** Preparing to test memory region ffffa8ef9000 (118784 bytes)
*** Preparing to test memory region ffffa8f1e000 (970752 bytes)
*** Preparing to test memory region ffffa900d000 (241664 bytes)
*** Preparing to test memory region ffffa904d000 (49152 bytes)
*** Preparing to test memory region ffffa905a000 (360448 bytes)
*** Preparing to test memory region ffffa90b4000 (221184 bytes)
*** Preparing to test memory region ffffa90eb000 (245760 bytes)
*** Preparing to test memory region ffffa9128000 (98304 bytes)
*** Preparing to test memory region ffffa9142000 (1118208 bytes)
*** Preparing to test memory region ffffa9254000 (708608 bytes)
*** Preparing to test memory region ffffa9303000 (67723264 bytes)
*** Preparing to test memory region ffffad399000 (16384 bytes)
*** Preparing to test memory region ffffae3aa000 (1171456 bytes)
*** Preparing to test memory region ffffae56a000 (94208 bytes)
*** Preparing to test memory region ffffae589000 (8192 bytes)
.2581:M 26 Apr 2026 08:00:36.608 # valkey 9.0.3 crashed by signal: 11, si_code: 1
2581:M 26 Apr 2026 08:00:36.608 # Accessing address: 0x4000

------ INFO OUTPUT ------
# Server
redis_version:7.2.4
server_name:valkey
valkey_version:9.0.3
valkey_release_stage:ga
redis_git_sha1:00000000
redis_git_dirty:1
redis_build_id:ff9b530358b4b831
server_mode:standalone
os:Linux 6.18.7-0-virt aarch64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
gcc_version:15.2.0
process_id:2581
process_supervised:no
run_id:563fe610ccacd98fd5efad39cc75f21fb9abb65d
tcp_port:6379
server_time_usec:1777190436605784
uptime_in_seconds:281111
uptime_in_days:3
hz:10
configured_hz:10
clients_hz:10
lru_clock:15582756
executable:/usr/bin/valkey-server
config_file:/etc/valkey/server.conf
io_threads_active:0
availability_zone:
listener0:name=tcp,bind=0.0.0.0,port=6379

# Clients
connected_clients:17
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:16390
client_recent_max_output_buffer:16577
blocked_clients:0
tracking_clients:0
pubsub_clients:3
watching_clients:0
clients_in_timeout_table:0
total_watched_keys:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0
paused_reason:none
paused_actions:none
paused_timeout_milliseconds:0

# Memory
used_memory:961465813
used_memory_human:916.93M
used_memory_rss:1202921472
used_memory_rss_human:1.12G
used_memory_peak:1234078805
used_memory_peak_human:1.15G
used_memory_peak_perc:77.91%
used_memory_overhead:140130631
used_memory_startup:899293
used_memory_dataset:821335182
used_memory_dataset_perc:85.51%
allocator_allocated:961415769
allocator_active:1202846720
allocator_resident:1202846720
allocator_muzzy:0
total_system_memory:4028239872
total_system_memory_human:3.75G
used_memory_lua:38912
used_memory_vm_eval:38912
used_memory_lua_human:38.00K
used_memory_scripts_eval:1136
number_of_cached_scripts:4
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:35840
used_memory_vm_total:74752
used_memory_vm_total_human:73.00K
used_memory_functions:237
used_memory_scripts:1373
used_memory_scripts_human:1.34K
maxmemory:2684354560
maxmemory_human:2.50G
maxmemory_policy:noeviction
allocator_frag_ratio:1.00
allocator_frag_bytes:0
allocator_rss_ratio:1.00
allocator_rss_bytes:0
rss_overhead_ratio:1.00
rss_overhead_bytes:74752
mem_fragmentation_ratio:1.25
mem_fragmentation_bytes:241505703
mem_not_counted_for_evict:16384
mem_replication_backlog:10485928
mem_total_replication_buffers:10543168
mem_clients_slaves:57408
mem_clients_normal:231493
mem_cluster_links:0
mem_cluster_slot_import:0
mem_cluster_slot_export:0
mem_aof_buffer:0
mem_allocator:libc
mem_overhead_db_hashtable_rehashing:0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:320853877
rdb_bgsave_in_progress:0
rdb_last_save_time:1776909325
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:7
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:122023936
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:2997865
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
slot_migration_fork_in_progress:0

# Stats
total_connections_received:164
total_commands_processed:373440946
instantaneous_ops_per_sec:2373
total_net_input_bytes:74082373375
total_net_output_bytes:129856353553
total_net_repl_input_bytes:31605877312
total_net_repl_output_bytes:38559914645
total_net_cluster_slot_import_bytes:0
total_net_cluster_slot_export_bytes:0
instantaneous_input_kbps:389.13
instantaneous_output_kbps:1162.18
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:344.25
rejected_connections:0
sync_full:1
sync_partial_ok:1
sync_partial_err:0
expired_keys:3058506
expired_fields:0
expired_stale_perc:3.29
expired_keys_with_volatile_items_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:20697
evicted_keys:0
evicted_clients:0
evicted_scripts:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:180554343
keyspace_misses:6008307
pubsub_channels:1
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:25401
total_forks:1
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:15
dump_payload_sanitizations:0
total_reads_processed:259444131
total_writes_processed:280130429
io_threaded_reads_processed:0
io_threaded_writes_processed:0
io_threaded_freed_objects:0
io_threaded_accept_processed:0
io_threaded_poll_processed:0
io_threaded_total_prefetch_batches:0
io_threaded_total_prefetch_entries:0
client_query_buffer_limit_disconnections:0
client_output_buffer_limit_disconnections:0
reply_buffer_shrinks:220334
reply_buffer_expands:359879
eventloop_cycles:257537423
eventloop_duration_sum:9005314099
eventloop_duration_cmd_sum:2370990308
instantaneous_eventloop_cycles_per_sec:1697
instantaneous_eventloop_duration_usec:33
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
acl_access_denied_tls_cert:0

# Replication
role:master
connected_slaves:1
slave0:ip=172.20.0.243,port=6379,state=online,offset=11595923561689,lag=0,type=replica
replicas_waiting_psync:0
master_failover_state:no-failover
master_replid:24c8375a4b600444a533c8e58dbf76813945d0c3
master_replid2:0a0f5a81950f264a96a44524f98b58d5bcfd390b
master_repl_offset:11595923635973
second_repl_offset:11557941852840
repl_backlog_active:1
repl_backlog_size:10485760
repl_backlog_first_byte_offset:11595913145894
repl_backlog_histlen:10490080

# CPU
used_cpu_sys:4263.589218
used_cpu_user:6476.110293
used_cpu_sys_children:0.628432
used_cpu_user_children:5.603736
used_cpu_sys_main_thread:4263.589106
used_cpu_user_main_thread:6476.110271

# Modules

# Commandstats
cmdstat_info:calls=84098,usec=8378892,usec_per_call=99.63,rejected_calls=0,failed_calls=0
cmdstat_zadd:calls=98492534,usec=1059418318,usec_per_call=10.76,rejected_calls=0,failed_calls=0
cmdstat_getex:calls=7580219,usec=32441006,usec_per_call=4.28,rejected_calls=0,failed_calls=0
cmdstat_mget:calls=3315111,usec=36513277,usec_per_call=11.01,rejected_calls=0,failed_calls=0
cmdstat_lpop:calls=1768,usec=3487,usec_per_call=1.97,rejected_calls=0,failed_calls=0
cmdstat_xtrim:calls=78,usec=5668646,usec_per_call=72674.95,rejected_calls=0,failed_calls=0
cmdstat_llen:calls=1768,usec=4103,usec_per_call=2.32,rejected_calls=0,failed_calls=0
cmdstat_select:calls=1,usec=1,usec_per_call=1.00,rejected_calls=0,failed_calls=0
cmdstat_set:calls=21261503,usec=96360444,usec_per_call=4.53,rejected_calls=0,failed_calls=0
cmdstat_replconf:calls=158242,usec=242629,usec_per_call=1.53,rejected_calls=0,failed_calls=0
cmdstat_setnx:calls=787258,usec=3439259,usec_per_call=4.37,rejected_calls=0,failed_calls=0
cmdstat_ping:calls=824122,usec=1241364,usec_per_call=1.51,rejected_calls=13,failed_calls=0
cmdstat_psync:calls=2,usec=290,usec_per_call=145.00,rejected_calls=0,failed_calls=0
cmdstat_multi:calls=4494,usec=2293,usec_per_call=0.51,rejected_calls=0,failed_calls=0
cmdstat_ttl:calls=21,usec=34,usec_per_call=1.62,rejected_calls=0,failed_calls=0
cmdstat_expire:calls=46859230,usec=116351021,usec_per_call=2.48,rejected_calls=0,failed_calls=0
cmdstat_zremrangebyscore:calls=4757,usec=956124,usec_per_call=200.99,rejected_calls=0,failed_calls=0
cmdstat_xadd:calls=42014899,usec=401711295,usec_per_call=9.56,rejected_calls=0,failed_calls=0
cmdstat_publish:calls=593037,usec=5125430,usec_per_call=8.64,rejected_calls=0,failed_calls=0
cmdstat_subscribe:calls=13,usec=129,usec_per_call=9.92,rejected_calls=0,failed_calls=0
cmdstat_eval:calls=67,usec=2606,usec_per_call=38.90,rejected_calls=0,failed_calls=0
cmdstat_config|rewrite:calls=1,usec=5453,usec_per_call=5453.00,rejected_calls=0,failed_calls=0
cmdstat_pexpire:calls=105906,usec=265025,usec_per_call=2.50,rejected_calls=0,failed_calls=0
cmdstat_get:calls=59210017,usec=149945129,usec_per_call=2.53,rejected_calls=0,failed_calls=0
cmdstat_xread:calls=21171186,usec=436369528,usec_per_call=20.61,rejected_calls=0,failed_calls=0
cmdstat_client|kill:calls=2,usec=118,usec_per_call=59.00,rejected_calls=0,failed_calls=0
cmdstat_client|setinfo:calls=246,usec=141,usec_per_call=0.57,rejected_calls=0,failed_calls=0
cmdstat_client|unpause:calls=1,usec=1,usec_per_call=1.00,rejected_calls=0,failed_calls=0
cmdstat_client|setname:calls=26,usec=13,usec_per_call=0.50,rejected_calls=0,failed_calls=0
cmdstat_role:calls=46,usec=410,usec_per_call=8.91,rejected_calls=0,failed_calls=0
cmdstat_xinfo|stream:calls=15901,usec=262472,usec_per_call=16.51,rejected_calls=0,failed_calls=0
cmdstat_hincrby:calls=1646926,usec=9850025,usec_per_call=5.98,rejected_calls=0,failed_calls=0
cmdstat_evalsha:calls=113414,usec=3857581,usec_per_call=34.01,rejected_calls=0,failed_calls=0
cmdstat_exec:calls=4494,usec=69928,usec_per_call=15.56,rejected_calls=0,failed_calls=0
cmdstat_unlink:calls=2432964,usec=11293099,usec_per_call=4.64,rejected_calls=0,failed_calls=0
cmdstat_hget:calls=2048,usec=5540,usec_per_call=2.71,rejected_calls=0,failed_calls=0
cmdstat_auth:calls=151,usec=789,usec_per_call=5.23,rejected_calls=0,failed_calls=0
cmdstat_del:calls=5412,usec=27456,usec_per_call=5.07,rejected_calls=0,failed_calls=0
cmdstat_zcard:calls=2675,usec=4953,usec_per_call=1.85,rejected_calls=0,failed_calls=0
cmdstat_pexpireat:calls=41278497,usec=71935496,usec_per_call=1.74,rejected_calls=0,failed_calls=0
cmdstat_xrange:calls=4631,usec=50329932,usec_per_call=10868.05,rejected_calls=0,failed_calls=0
cmdstat_setex:calls=25463168,usec=149478878,usec_per_call=5.87,rejected_calls=0,failed_calls=0
cmdstat_acl|log:calls=3,usec=9,usec_per_call=3.00,rejected_calls=0,failed_calls=0
cmdstat_acl|list:calls=1,usec=98,usec_per_call=98.00,rejected_calls=0,failed_calls=0
cmdstat_command|docs:calls=8,usec=13975,usec_per_call=1746.88,rejected_calls=0,failed_calls=0

# Errorstats
errorstat_ERR:count=2
errorstat_LOADING:count=13

# Latencystats
latency_percentiles_usec_info:p50=98.303,p99=143.359,p99.9=173.055
latency_percentiles_usec_zadd:p50=9.023,p99=30.079,p99.9=43.007
latency_percentiles_usec_getex:p50=4.015,p99=12.031,p99.9=23.039
latency_percentiles_usec_mget:p50=3.007,p99=39.167,p99.9=214.015
latency_percentiles_usec_lpop:p50=2.007,p99=6.015,p99.9=15.039
latency_percentiles_usec_xtrim:p50=72351.743,p99=139460.607,p99.9=186646.527
latency_percentiles_usec_llen:p50=2.007,p99=10.047,p99.9=30.079
latency_percentiles_usec_select:p50=1.003,p99=1.003,p99.9=1.003
latency_percentiles_usec_set:p50=4.015,p99=18.047,p99.9=27.007
latency_percentiles_usec_replconf:p50=1.003,p99=3.007,p99.9=16.063
latency_percentiles_usec_setnx:p50=4.015,p99=11.007,p99.9=23.039
latency_percentiles_usec_ping:p50=1.003,p99=4.015,p99.9=16.063
latency_percentiles_usec_psync:p50=92.159,p99=198.655,p99.9=198.655
latency_percentiles_usec_multi:p50=0.001,p99=1.003,p99.9=11.007
latency_percentiles_usec_ttl:p50=2.007,p99=3.007,p99.9=3.007
latency_percentiles_usec_expire:p50=2.007,p99=7.007,p99.9=19.071
latency_percentiles_usec_zremrangebyscore:p50=195.583,p99=444.415,p99.9=1409.023
latency_percentiles_usec_xadd:p50=9.023,p99=29.055,p99.9=46.079
latency_percentiles_usec_publish:p50=9.023,p99=27.007,p99.9=41.215
latency_percentiles_usec_subscribe:p50=10.047,p99=17.023,p99.9=17.023
latency_percentiles_usec_eval:p50=36.095,p99=90.111,p99.9=106.495
latency_percentiles_usec_config|rewrite:p50=5472.255,p99=5472.255,p99.9=5472.255
latency_percentiles_usec_pexpire:p50=2.007,p99=5.023,p99.9=20.095
latency_percentiles_usec_get:p50=2.007,p99=6.015,p99.9=19.071
latency_percentiles_usec_xread:p50=20.095,p99=41.215,p99.9=55.039
latency_percentiles_usec_client|kill:p50=47.103,p99=71.167,p99.9=71.167
latency_percentiles_usec_client|setinfo:p50=1.003,p99=2.007,p99.9=3.007
latency_percentiles_usec_client|unpause:p50=1.003,p99=1.003,p99.9=1.003
latency_percentiles_usec_client|setname:p50=0.001,p99=1.003,p99.9=1.003
latency_percentiles_usec_role:p50=7.007,p99=52.223,p99.9=52.223
latency_percentiles_usec_xinfo|stream:p50=16.063,p99=33.023,p99.9=48.127
latency_percentiles_usec_hincrby:p50=5.023,p99=19.071,p99.9=29.055
latency_percentiles_usec_evalsha:p50=30.079,p99=157.695,p99.9=185.343
latency_percentiles_usec_exec:p50=13.055,p99=40.191,p99.9=85.503
latency_percentiles_usec_unlink:p50=4.015,p99=16.063,p99.9=25.087
latency_percentiles_usec_hget:p50=2.007,p99=8.031,p99.9=19.071
latency_percentiles_usec_auth:p50=5.023,p99=10.047,p99.9=23.039
latency_percentiles_usec_del:p50=5.023,p99=18.047,p99.9=31.103
latency_percentiles_usec_zcard:p50=2.007,p99=5.023,p99.9=31.103
latency_percentiles_usec_pexpireat:p50=1.003,p99=12.031,p99.9=18.047
latency_percentiles_usec_xrange:p50=10878.975,p99=12189.695,p99.9=14155.775
latency_percentiles_usec_setex:p50=5.023,p99=16.063,p99.9=28.031
latency_percentiles_usec_acl|log:p50=3.007,p99=3.007,p99.9=3.007
latency_percentiles_usec_acl|list:p50=98.303,p99=98.303,p99.9=98.303
latency_percentiles_usec_command|docs:p50=1720.319,p99=1884.159,p99.9=1884.159

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=2943109,expires=2943103,avg_ttl=283216917,keys_with_volatile_items=0

------ CLIENT LIST OUTPUT ------
id=44 addr=172.20.2.8:59660 laddr=172.20.0.86:6379 fd=24 name=*redacted* age=159196 idle=1786 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1600 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=4594 tot-net-out=8562 tot-cmds=70
id=113 addr=172.20.2.213:35376 laddr=172.20.0.86:6379 fd=12 name=*redacted* age=75753 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=5540 rbp=4200 obl=0 oll=0 omem=0 tot-mem=22506 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=3083606685 tot-net-out=5544776665 tot-cmds=17247234
id=142 addr=172.20.1.207:59328 laddr=172.20.0.86:6379 fd=11 name=*redacted* age=34228 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=43950 argv-mem=0 multi-mem=0 rbs=11872 rbp=5936 obl=0 oll=0 omem=0 tot-mem=56404 events=r cmd=setex user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=1456936012 tot-net-out=2616014074 tot-cmds=8081515
id=48 addr=172.20.0.243:38866 laddr=172.20.0.86:6379 fd=14 name=*redacted* age=158844 idle=0 flags=S capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=2 omem=32896 tot-mem=34496 events=r cmd=replconf user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=7581357 tot-net-out=37842763236 tot-cmds=157945
id=147 addr=172.20.1.49:59788 laddr=172.20.0.86:6379 fd=17 name=*redacted* age=30339 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=5792 rbp=4061 obl=0 oll=0 omem=0 tot-mem=22758 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=1234804927 tot-net-out=2233259403 tot-cmds=6878161
id=49 addr=172.20.0.243:38868 laddr=172.20.0.86:6379 fd=19 name=*redacted* age=158844 idle=1 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=2048 rbp=1024 obl=0 oll=0 omem=0 tot-mem=2624 events=r cmd=ping user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=13195572 tot-net-out=105693674 tot-cmds=246756
id=148 addr=172.20.6.60:38300 laddr=172.20.0.86:6379 fd=15 name=*redacted* age=28583 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=5816 rbp=3987 obl=0 oll=0 omem=0 tot-mem=22782 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=1139205888 tot-net-out=2062298073 tot-cmds=6354207
id=158 addr=172.20.5.232:38232 laddr=172.20.0.86:6379 fd=18 name=*redacted* age=14123 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=5888 rbp=4041 obl=0 oll=0 omem=0 tot-mem=22854 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=523488029 tot-net-out=958223146 tot-cmds=2914435
id=50 addr=172.20.0.243:38882 laddr=172.20.0.86:6379 fd=20 name=*redacted* age=158844 idle=0 flags=P capa= db=0 sub=1 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=140 obl=0 oll=0 omem=0 tot-mem=1880 events=r cmd=subscribe user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=188 tot-net-out=32492093 tot-cmds=3
id=139 addr=172.20.2.23:41878 laddr=172.20.0.86:6379 fd=8 name=*redacted* age=37462 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=6033 qbuf-free=10351 argv-mem=5932 multi-mem=0 rbs=6016 rbp=4031 obl=22 oll=0 omem=0 tot-mem=29042 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=1550389371 tot-net-out=2773160132 tot-cmds=8594740
id=43 addr=172.20.4.51:40162 laddr=172.20.0.86:6379 fd=23 name=*redacted* age=159202 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=54 multi-mem=0 rbs=2048 rbp=1024 obl=0 oll=0 omem=0 tot-mem=2742 events=r cmd=xread user=*redacted* redir=-1 resp=2 lib-name=redis-rs lib-ver=0.27.6 tot-net-in=2287195652 tot-net-out=9196137560 tot-cmds=21187090
id=149 addr=172.20.6.56:52714 laddr=172.20.0.86:6379 fd=16 name=*redacted* age=28473 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=7870 rbp=4261 obl=0 oll=0 omem=0 tot-mem=24836 events=r cmd=setex user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=1173441136 tot-net-out=2120929491 tot-cmds=6546702
id=94 addr=172.20.3.218:47322 laddr=172.20.0.86:6379 fd=22 name=*redacted* age=106630 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=16384 argv-mem=0 multi-mem=0 rbs=6032 rbp=3016 obl=0 oll=0 omem=0 tot-mem=22998 events=r cmd=getex user=*redacted* redir=-1 resp=2 lib-name=ioredis lib-ver=5.10.1 tot-net-in=4289430317 tot-net-out=7715505107 tot-cmds=24018283
id=52 addr=172.20.4.51:36688 laddr=172.20.0.86:6379 fd=25 name=*redacted* age=158611 idle=3569 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1600 events=r cmd=xadd user=*redacted* redir=-1 resp=2 lib-name=redis-rs lib-ver=0.27.6 tot-net-in=31540 tot-net-out=2567 tot-cmds=119
id=66 addr=172.20.0.86:53250 laddr=172.20.0.86:6379 fd=10 name=*redacted* age=141450 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=1600 events=r cmd=ping user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=11683125 tot-net-out=94132857 tot-cmds=219762
id=67 addr=172.20.0.86:53254 laddr=172.20.0.86:6379 fd=13 name=*redacted* age=141450 idle=0 flags=P capa= db=0 sub=1 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=140 obl=0 oll=0 omem=0 tot-mem=1880 events=r cmd=subscribe user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=188 tot-net-out=28934811 tot-cmds=3
id=64 addr=172.20.17.192:60756 laddr=172.20.0.86:6379 fd=7 name=*redacted* age=141470 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=1600 events=r cmd=ping user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=11815567 tot-net-out=94138380 tot-cmds=219604
id=65 addr=172.20.17.192:60770 laddr=172.20.0.86:6379 fd=9 name=*redacted* age=141470 idle=0 flags=P capa= db=0 sub=1 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=140 obl=0 oll=0 omem=0 tot-mem=1880 events=r cmd=subscribe user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=188 tot-net-out=28938704 tot-cmds=3

------ CURRENT CLIENT INFO ------
id=43 addr=172.20.4.51:40162 laddr=172.20.0.86:6379 fd=23 name=*redacted* age=159202 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=54 multi-mem=0 rbs=2048 rbp=1024 obl=0 oll=0 omem=0 tot-mem=2742 events=r cmd=xread user=*redacted* redir=-1 resp=2 lib-name=redis-rs lib-ver=0.27.6 tot-net-in=2287195652 tot-net-out=9196137560 tot-cmds=21187090
argc: 8
argv[0]: "XREAD"
argv[1]: 5 bytes
argv[2]: 4 bytes
argv[3]: 5 bytes
argv[4]: 3 bytes
argv[5]: 7 bytes
argv[6]: 10 bytes
argv[7]: 15 bytes

------ EXECUTING CLIENT INFO ------
id=43 addr=172.20.4.51:40162 laddr=172.20.0.86:6379 fd=23 name=*redacted* age=159202 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=54 multi-mem=0 rbs=2048 rbp=1024 obl=0 oll=0 omem=0 tot-mem=2742 events=r cmd=xread user=*redacted* redir=-1 resp=2 lib-name=redis-rs lib-ver=0.27.6 tot-net-in=2287195652 tot-net-out=9196137560 tot-cmds=21187090
argc: 8
argv[0]: "XREAD"
argv[1]: 5 bytes
argv[2]: 4 bytes
argv[3]: 5 bytes
argv[4]: 3 bytes
argv[5]: 7 bytes
argv[6]: 10 bytes
argv[7]: 15 bytes

------ MODULES INFO OUTPUT ------

------ CONFIG DEBUG OUTPUT ------
slave-read-only yes
lazyfree-lazy-expire yes
lazyfree-lazy-user-del yes
list-compress-depth 0
proto-max-bulk-len 512mb
import-mode no
debug-context ""
lazyfree-lazy-server-del yes
repl-diskless-sync yes
sanitize-dump-payload no
lazyfree-lazy-user-flush yes
io-threads 1
replica-read-only yes
activedefrag no
repl-diskless-load on-empty-db
client-query-buffer-limit 1gb
dual-channel-replication-enabled no
lazyfree-lazy-eviction yes

------ FAST MEMORY TEST ------
2581:signal-handler (1777190436) Crashed running signal handler. Providing reduced version of recursive crash report.
2581:M 26 Apr 2026 08:00:36.609 # valkey 9.0.3 crashed by signal: 11, si_code: 1
2581:M 26 Apr 2026 08:00:36.609 # Accessing address: 0xffffae3a8b0c

=== VALKEY BUG REPORT END. Make sure to include from START to END. ===

Additional information

AWS EC2 t4g.medium
OS: alpine-3.23.3-aarch64-uefi-tiny-r0

1 primary + sentinel : the crashed instance
1 secondary + sentinel
1 sentinel

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions