I had to omit some of the logs, but if necessary let me know I will share:
Primary and Replica lose the connection:
...
Primary:
12:29:37.3548501 *:174] 19:M Partial resynchronization request from 192.168.171.26:6380 accepted. Sending 88776 bytes of backlog starting from offset 35945097.
12:29:37.3548721 *:174] 19:M Connection with replica 192.168.171.26:6380 lost.
...
Replica:
...
12:29:37.3471591 *:174] 18:S Reconnecting to PRIMARY 192.168.230.105:6380
12:29:37.3471671 *:174] 18:S PRIMARY <-> REPLICA sync started
...
Long process of desperate attempt to perform partial sync (for some reason it takes long time and extremely high number of attempts, I'll leave only the ones before the full sync):
Primary:
...
12:30:06.5486051 *:174] 19:M Replica 192.168.171.26:6380 asks for synchronization
12:30:06.5486751 *:174] 19:M Partial resynchronization request from 192.168.171.26:6380 accepted. Sending 134133608 bytes of backlog starting from offset 35945097.
Replica:
...
12:30:06.5486841 *:174] 18:S Successful partial resynchronization with primary.
12:30:06.5486971 *:174] 18:S PRIMARY <-> REPLICA sync: Primary accepted a Partial Resynchronization.
12:30:06.5489211 *:174] 18:S Connection with primary lost.
...
Attempt of the full sync:
Primary:
12:30:06.5577381 *:174] 19:M Dual channel replication: Replica 192.168.171.26:6380 is capable of dual channel synchronization, and partial sync isn't possible. Full sync will continue with dedicated RDB channel.
...
12:30:07.7366921 *:174] 19:M Synchronization with replica 192.168.171.26:6380 succeeded
12:30:07.8172691 *:174] 19:M Background RDB transfer terminated with success
Replica:
12:30:06.5577761 *:174] 18:S Dual channel replication: PSYNC is not possible, initialize RDB channel.
...
12:30:07.7362651 *:174] 18:S PRIMARY <-> REPLICA sync: Finished with success
~20+ minutes later crash out of a sudden:
Primary:
12:54:38.0792621 *:142] === VALKEY BUG REPORT START: Cut & paste starting from here ===
12:54:38.0793471 *:176] 19:M valkey 8.1.1 crashed by signal: 11, si_code: 1
12:54:38.0793551 *:176] 19:M Accessing address: 0x4
12:54:38.0793601 *:176] 19:M Crashed running the instruction at: 0x594d8d
12:54:38.0793641 *:142] ------ STACK TRACE ------
12:54:38.0793681 *:142] EIP:
12:54:38.0794321 *:142] redis-server *:6380 [cluster](expireScanCallback+0xd)[0x594d8d]
12:54:38.0836771 *:142] 20 bio_close_file
12:54:38.0837261 *:142] /lib64/libc.so.6(+0xa3c1e)[0x7f8cd98dfc1e]
12:54:38.0837311 *:142] /lib64/libc.so.6(pthread_cond_wait+0x12a)[0x7f8cd98e2548]
12:54:38.0838761 *:142] redis-server *:6380 [cluster](bioProcessBackgroundJobs+0x16c)[0x53ff2c]
12:54:38.0839291 *:142] /lib64/libc.so.6(+0xa6f6c)[0x7f8cd98e2f6c]
12:54:38.0839671 *:142] /lib64/libc.so.6(clone+0x42)[0x7f8cd996a180]
12:54:38.0839811 *:142] 22 bio_lazy_free
12:54:38.0840051 *:142] /lib64/libc.so.6(+0xa3c1e)[0x7f8cd98dfc1e]
12:54:38.0840631 *:142] /lib64/libc.so.6(pthread_cond_wait+0x12a)[0x7f8cd98e2548]
12:54:38.0841821 *:142] redis-server *:6380 [cluster](bioProcessBackgroundJobs+0x16c)[0x53ff2c]
12:54:38.0842151 *:142] /lib64/libc.so.6(+0xa6f6c)[0x7f8cd98e2f6c]
12:54:38.0842641 *:142] /lib64/libc.so.6(clone+0x42)[0x7f8cd996a180]
12:54:38.0842731 *:142] 21 bio_aof
12:54:38.0842921 *:142] /lib64/libc.so.6(+0xa3c1e)[0x7f8cd98dfc1e]
12:54:38.0843371 *:142] /lib64/libc.so.6(pthread_cond_wait+0x12a)[0x7f8cd98e2548]
12:54:38.0844581 *:142] redis-server *:6380 [cluster](bioProcessBackgroundJobs+0x16c)[0x53ff2c]
12:54:38.0845011 *:142] /lib64/libc.so.6(+0xa6f6c)[0x7f8cd98e2f6c]
12:54:38.0845451 *:142] /lib64/libc.so.6(clone+0x42)[0x7f8cd996a180]
12:54:38.0845551 *:142] 19 redis-server *
12:54:38.0845781 *:142] /lib64/libc.so.6(+0x57900)[0x7f8cd9893900]
12:54:38.0849621 *:142] redis-server *:6380 [cluster](expireScanCallback+0xd)[0x594d8d]
12:54:38.0849801 *:142] redis-server *:6380 [cluster](activeExpireCycle+0xb31)[0x5959d1]
12:54:38.0849841 *:142] redis-server *:6380 [cluster](databasesCron+0x4aa)[0x47828a]
12:54:38.0850351 *:142] redis-server *:6380 [cluster](serverCron+0x78a)[0x478e3a]
12:54:38.0850621 *:142] redis-server *:6380 [cluster][0x460df9]
12:54:38.0851231 *:142] redis-server *:6380 [cluster](aeMain+0x130)[0x46ad60]
12:54:38.0852371 *:142] redis-server *:6380 [cluster](main+0x4d7)[0x45f587]
12:54:38.0852641 *:142] /lib64/libc.so.6(+0x40e6c)[0x7f8cd987ce6c]
12:54:38.0853011 *:142] /lib64/libc.so.6(__libc_start_main+0x87)[0x7f8cd987cf35]
12:54:38.0853761 *:142] redis-server *:6380 [cluster](_start+0x21)[0x460cc1]
12:54:38.0854521 *:142] 4/4 expected stacktraces.
12:54:38.0854591 *:142] ------ STACK TRACE DONE ------
12:54:38.0854631 *:142] ------ REGISTERS ------
12:54:38.0854661 *:142] 19:M 16 May 2025 12:54:38.085 #
12:54:38.0854701 *:142] RAX:0000000000000020 RBX:00007f8cd657cb80
12:54:38.0854741 *:142] RCX:0000000000000005 RDX:0000000000000020
12:54:38.0854771 *:142] RDI:00007ffd5c0d3300 RSI:0000000000000000
12:54:38.0854811 *:142] RBP:00007ffd5c0d3210 RSP:00007ffd5c0d31f0
12:54:38.0854841 *:142] R8 :fffffffffffff550 R9 :00007f8cd9400bb0
12:54:38.0854881 *:142] R10:0000000000000003 R11:00007f8cd98141c0
12:54:38.0854911 *:142] R12:0000000000000005 R13:0000000000000001
12:54:38.0854951 *:142] R14:00007f8cd9447240 R15:0000000000000000
12:54:38.0854981 *:142] RIP:0000000000594d8d EFL:0000000000010206
12:54:38.0855021 *:142] CSGSFS:002b000000000033
12:54:38.0855071 *:174] 19:M hide-user-data-from-log is on, skip logging stack content to avoid spilling user data.
12:54:38.0856531 *:142] ------ INFO OUTPUT ------
12:54:38.0875091 *:142] # Server
12:54:38.0875311 *:142] redis_version:7.2.4
12:54:38.0875351 *:142] server_name:valkey
12:54:38.0875381 *:142] valkey_version:8.1.1
12:54:38.0875451 *:142] valkey_release_stage:ga
12:54:38.0875481 *:142] redis_git_sha1:00000000
12:54:38.0875511 *:142] redis_git_dirty:0
12:54:38.0875551 *:142] redis_build_id:bb1dae1294af36a1
12:54:38.0875591 *:142] server_mode:cluster
12:54:38.0875631 *:142] os:Linux 6.4.0-150600.23.38.1.29362.2.PTF.1237234-default x86_64
12:54:38.0875671 *:142] arch_bits:64
12:54:38.0875701 *:142] monotonic_clock:POSIX clock_gettime
12:54:38.0875741 *:142] multiplexing_api:epoll
12:54:38.0875771 *:142] gcc_version:13.3.1
12:54:38.0875801 *:142] process_id:19
12:54:38.0875841 *:142] process_supervised:no
12:54:38.0875881 *:142] run_id:db239e6fde18af9009c83484483eddd27cd88106
12:54:38.0875921 *:142] tcp_port:6380
12:54:38.0875961 *:142] server_time_usec:1747400078078966
12:54:38.0875991 *:142] uptime_in_seconds:2319
12:54:38.0876031 *:142] uptime_in_days:0
12:54:38.0876071 *:142] hz:10
12:54:38.0876111 *:142] configured_hz:10
12:54:38.0876151 *:142] clients_hz:10
12:54:38.0876181 *:142] lru_clock:2569614
12:54:38.0876221 *:142] executable:/redis-server
12:54:38.0876251 *:142] config_file:/redis-conf/redis.conf
12:54:38.0876291 *:142] io_threads_active:0
12:54:38.0876331 *:142] availability_zone:
12:54:38.0876361 *:142] listener2:name=tls,bind=*,bind=-::*,port=6380
12:54:38.0876401 *:142] # Clients
12:54:38.0876441 *:142] connected_clients:90
12:54:38.0876471 *:142] cluster_connections:30
12:54:38.0876511 *:142] maxclients:8192
12:54:38.0876551 *:142] client_recent_max_input_buffer:32768
12:54:38.0876581 *:142] client_recent_max_output_buffer:869192
12:54:38.0876621 *:142] blocked_clients:2
12:54:38.0876711 *:142] tracking_clients:16
12:54:38.0876751 *:142] pubsub_clients:0
12:54:38.0876781 *:142] watching_clients:0
12:54:38.0876821 *:142] clients_in_timeout_table:0
12:54:38.0876861 *:142] total_watched_keys:0
12:54:38.0876901 *:142] total_blocking_keys:0
12:54:38.0876941 *:142] total_blocking_keys_on_nokey:0
12:54:38.0876971 *:142] paused_reason:none
12:54:38.0877011 *:142] paused_actions:none
12:54:38.0877041 *:142] paused_timeout_milliseconds:0
12:54:38.0877081 *:142] # Memory
12:54:38.0877121 *:142] used_memory:841497664
12:54:38.0877151 *:142] used_memory_human:802.51M
12:54:38.0877191 *:142] used_memory_rss:943075328
12:54:38.0877221 *:142] used_memory_rss_human:899.39M
12:54:38.0877261 *:142] used_memory_peak:1073132136
12:54:38.0877301 *:142] used_memory_peak_human:1023.42M
12:54:38.0877331 *:142] used_memory_peak_perc:78.42%
12:54:38.0877371 *:142] used_memory_overhead:149417464
12:54:38.0877401 *:142] used_memory_startup:3787280
12:54:38.0877441 *:142] used_memory_dataset:692080200
12:54:38.0877481 *:142] used_memory_dataset_perc:82.62%
12:54:38.0877521 *:142] allocator_allocated:841927848
12:54:38.0877551 *:142] allocator_active:892010496
12:54:38.0877591 *:142] allocator_resident:928251904
12:54:38.0877621 *:142] allocator_muzzy:0
12:54:38.0877661 *:142] total_system_memory:103352115200
12:54:38.0877691 *:142] total_system_memory_human:96.25G
12:54:38.0877721 *:142] used_memory_lua:77824
12:54:38.0877761 *:142] used_memory_vm_eval:77824
12:54:38.0877801 *:142] used_memory_lua_human:76.00K
12:54:38.0877831 *:142] used_memory_scripts_eval:18608
12:54:38.0877871 *:142] number_of_cached_scripts:20
12:54:38.0877961 *:142] number_of_functions:0
12:54:38.0877991 *:142] number_of_libraries:0
12:54:38.0878031 *:142] used_memory_vm_functions:33792
12:54:38.0878061 *:142] used_memory_vm_total:111616
12:54:38.0878101 *:142] used_memory_vm_total_human:109.00K
12:54:38.0878141 *:142] used_memory_functions:224
12:54:38.0878171 *:142] used_memory_scripts:18832
12:54:38.0878211 *:142] used_memory_scripts_human:18.39K
12:54:38.0878241 *:142] maxmemory:4294967296
12:54:38.0878281 *:142] maxmemory_human:4.00G
12:54:38.0878311 *:142] maxmemory_policy:noeviction
12:54:38.0878351 *:142] allocator_frag_ratio:1.06
12:54:38.0878391 *:142] allocator_frag_bytes:50082648
12:54:38.0878421 *:142] allocator_rss_ratio:1.04
12:54:38.0878461 *:142] allocator_rss_bytes:36241408
12:54:38.0878501 *:142] rss_overhead_ratio:1.02
12:54:38.0878541 *:142] rss_overhead_bytes:14823424
12:54:38.0878571 *:142] mem_fragmentation_ratio:1.12
12:54:38.0878611 *:142] mem_fragmentation_bytes:101596856
12:54:38.0878641 *:142] mem_not_counted_for_evict:0
12:54:38.0878681 *:142] mem_replication_backlog:134219196
12:54:38.0878711 *:142] mem_total_replication_buffers:134607688
12:54:38.0878751 *:142] mem_clients_slaves:389960
12:54:38.0878781 *:142] mem_clients_normal:1605356
12:54:38.0878831 *:142] mem_cluster_links:32160
12:54:38.0878861 *:142] mem_aof_buffer:0
12:54:38.0878901 *:142] mem_allocator:jemalloc-5.3.0
12:54:38.0878931 *:142] mem_overhead_db_hashtable_rehashing:0
12:54:38.0878971 *:142] active_defrag_running:0
12:54:38.0879001 *:142] lazyfree_pending_objects:0
12:54:38.0879041 *:142] lazyfreed_objects:0
12:54:38.0879081 *:142] # Persistence
12:54:38.0879111 *:142] loading:0
12:54:38.0879191 *:142] async_loading:0
12:54:38.0879231 *:142] current_cow_peak:0
12:54:38.0879261 *:142] current_cow_size:0
12:54:38.0879301 *:142] current_cow_size_age:0
12:54:38.0879331 *:142] current_fork_perc:0.00
12:54:38.0879361 *:142] current_save_keys_processed:0
12:54:38.0879391 *:142] current_save_keys_total:0
12:54:38.0879431 *:142] rdb_changes_since_last_save:1629440
12:54:38.0879461 *:142] rdb_bgsave_in_progress:0
12:54:38.0879501 *:142] rdb_last_save_time:1747397759
12:54:38.0879531 *:142] rdb_last_bgsave_status:ok
12:54:38.0879581 *:142] rdb_last_bgsave_time_sec:1
12:54:38.0879621 *:142] rdb_current_bgsave_time_sec:-1
12:54:38.0879651 *:142] rdb_saves:0
12:54:38.0879691 *:142] rdb_last_cow_size:46673920
12:54:38.0879721 *:142] rdb_last_load_keys_expired:0
12:54:38.0879761 *:142] rdb_last_load_keys_loaded:0
12:54:38.0879791 *:142] aof_enabled:0
12:54:38.0879831 *:142] aof_rewrite_in_progress:0
12:54:38.0879861 *:142] aof_rewrite_scheduled:0
12:54:38.0879901 *:142] aof_last_rewrite_time_sec:-1
12:54:38.0879931 *:142] aof_current_rewrite_time_sec:-1
12:54:38.0879971 *:142] aof_last_bgrewrite_status:ok
12:54:38.0880011 *:142] aof_rewrites:0
12:54:38.0880041 *:142] aof_rewrites_consecutive_failures:0
12:54:38.0880081 *:142] aof_last_write_status:ok
12:54:38.0880111 *:142] aof_last_cow_size:0
12:54:38.0880151 *:142] module_fork_in_progress:0
12:54:38.0880181 *:142] module_fork_last_cow_size:0
12:54:38.0880221 *:142] # Stats
12:54:38.0880251 *:142] total_connections_received:5525
12:54:38.0880291 *:142] total_commands_processed:9932803
12:54:38.0880321 *:142] instantaneous_ops_per_sec:24373
12:54:38.0880571 *:142] total_net_input_bytes:6299647479
12:54:38.0880601 *:142] total_net_output_bytes:11863618588
12:54:38.0880631 *:142] total_net_repl_input_bytes:0
12:54:38.0880671 *:142] total_net_repl_output_bytes:5639838905
12:54:38.0880701 *:142] instantaneous_input_kbps:1757.29
12:54:38.0880731 *:142] instantaneous_output_kbps:25926.41
12:54:38.0880771 *:142] instantaneous_input_repl_kbps:0.00
12:54:38.0880801 *:142] instantaneous_output_repl_kbps:444.04
12:54:38.0880841 *:142] rejected_connections:0
12:54:38.0880881 *:142] sync_full:2
12:54:38.0880911 *:142] sync_partial_ok:5170
12:54:38.0880941 *:142] sync_partial_err:1
12:54:38.0880971 *:142] expired_keys:9355
12:54:38.0881001 *:142] expired_stale_perc:15.92
12:54:38.0881031 *:142] expired_time_cap_reached_count:0
12:54:38.0881061 *:142] expire_cycle_cpu_milliseconds:159
12:54:38.0881101 *:142] evicted_keys:0
12:54:38.0881131 *:142] evicted_clients:0
12:54:38.0881161 *:142] evicted_scripts:0
12:54:38.0881201 *:142] total_eviction_exceeded_time:0
12:54:38.0881231 *:142] current_eviction_exceeded_time:0
12:54:38.0881271 *:142] keyspace_hits:2913039
12:54:38.0881301 *:142] keyspace_misses:5328970
12:54:38.0881341 *:142] pubsub_channels:0
12:54:38.0881371 *:142] pubsub_patterns:0
12:54:38.0881411 *:142] pubsubshard_channels:0
12:54:38.0881451 *:142] latest_fork_usec:5843
12:54:38.0881481 *:142] total_forks:2
12:54:38.0881521 *:142] migrate_cached_sockets:0
12:54:38.0881551 *:142] slave_expires_tracked_keys:0
12:54:38.0881591 *:142] active_defrag_hits:0
12:54:38.0881631 *:142] active_defrag_misses:0
12:54:38.0881661 *:142] active_defrag_key_hits:0
12:54:38.0881791 *:142] active_defrag_key_misses:0
12:54:38.0881831 *:142] total_active_defrag_time:0
12:54:38.0881861 *:142] current_active_defrag_time:0
12:54:38.0881891 *:142] tracking_total_keys:1
12:54:38.0881931 *:142] tracking_total_items:3
12:54:38.0881981 *:142] tracking_total_prefixes:12
12:54:38.0882021 *:142] unexpected_error_replies:0
12:54:38.0882061 *:142] total_error_replies:4124
12:54:38.0882091 *:142] dump_payload_sanitizations:0
12:54:38.0882131 *:142] total_reads_processed:2041647
12:54:38.0882161 *:142] total_writes_processed:2411693
12:54:38.0882201 *:142] io_threaded_reads_processed:0
12:54:38.0882241 *:142] io_threaded_writes_processed:0
12:54:38.0882271 *:142] io_threaded_freed_objects:0
12:54:38.0882311 *:142] io_threaded_accept_processed:0
12:54:38.0882351 *:142] io_threaded_poll_processed:0
12:54:38.0882381 *:142] io_threaded_total_prefetch_batches:0
12:54:38.0882411 *:142] io_threaded_total_prefetch_entries:0
12:54:38.0882451 *:142] client_query_buffer_limit_disconnections:0
12:54:38.0882481 *:142] client_output_buffer_limit_disconnections:0
12:54:38.0882521 *:142] reply_buffer_shrinks:1908
12:54:38.0882561 *:142] reply_buffer_expands:1966
12:54:38.0882601 *:142] eventloop_cycles:2000465
12:54:38.0882631 *:142] eventloop_duration_sum:221058162
12:54:38.0882971 *:142] eventloop_duration_cmd_sum:38145162
12:54:38.0883011 *:142] instantaneous_eventloop_cycles_per_sec:2043
12:54:38.0883041 *:142] instantaneous_eventloop_duration_usec:215
12:54:38.0883071 *:142] acl_access_denied_auth:0
12:54:38.0883111 *:142] acl_access_denied_cmd:0
12:54:38.0883151 *:142] acl_access_denied_key:0
12:54:38.0883181 *:142] acl_access_denied_channel:0
12:54:38.0883221 *:142] # Replication
12:54:38.0883251 *:142] role:master
12:54:38.0883391 *:142] connected_slaves:1
12:54:38.0883431 *:142] slave0:ip=192.168.171.26,port=6380,state=online,offset=5774107852,lag=1,type=replica
12:54:38.0883471 *:142] replicas_waiting_psync:0
12:54:38.0883501 *:142] master_failover_state:no-failover
12:54:38.0883531 *:142] master_replid:60d37e92cae5c8144e67b2dbfdccbdcd4c0f7f4a
12:54:38.0883561 *:142] master_replid2:0000000000000000000000000000000000000000
12:54:38.0883601 *:142] master_repl_offset:5774184935
12:54:38.0883631 *:142] second_repl_offset:-1
12:54:38.0883671 *:142] repl_backlog_active:1
12:54:38.0883711 *:142] repl_backlog_size:134217728
12:54:38.0883751 *:142] repl_backlog_first_byte_offset:5639960233
12:54:38.0883781 *:142] repl_backlog_histlen:134224703
12:54:38.0883821 *:142] # CPU
12:54:38.0883851 *:142] used_cpu_sys:95.463777
12:54:38.0883891 *:142] used_cpu_user:145.912438
12:54:38.0883921 *:142] used_cpu_sys_children:0.374917
12:54:38.0883961 *:142] used_cpu_user_children:0.719443
12:54:38.0884001 *:142] used_cpu_sys_main_thread:95.274208
12:54:38.0884031 *:142] used_cpu_user_main_thread:145.871879
12:54:38.0884071 *:142] # Modules
12:54:38.0884141 *:142] # Commandstats
...
12:54:38.0887151 *:142] # Errorstats
12:54:38.0887191 *:142] errorstat_MOVED:count=4122
12:54:38.0887221 *:142] errorstat_NOSCRIPT:count=2
12:54:38.0887261 *:142] # Latencystats
12:54:38.0887291 *:142] # Cluster
12:54:38.0887331 *:142] cluster_enabled:1
12:54:38.0887361 *:142] # Keyspace
12:54:38.0887401 *:142] db0:keys=261817,expires=6064,avg_ttl=188427
12:54:38.0887441 *:142] # Cluster info
12:54:38.0887481 *:142] cluster_state:ok
12:54:38.0887521 *:142] cluster_slots_assigned:16384
12:54:38.0887551 *:142] cluster_slots_ok:16384
12:54:38.0887581 *:142] cluster_slots_pfail:0
12:54:38.0887621 *:142] cluster_slots_fail:0
12:54:38.0887651 *:142] cluster_known_nodes:16
12:54:38.0887691 *:142] cluster_size:8
12:54:38.0887731 *:142] cluster_current_epoch:23
12:54:38.0887771 *:142] cluster_my_epoch:20
12:54:38.0887811 *:142] cluster_stats_messages_ping_sent:33119
12:54:38.0887851 *:142] cluster_stats_messages_pong_sent:33821
12:54:38.0887881 *:142] cluster_stats_messages_meet_sent:12
12:54:38.0887911 *:142] cluster_stats_messages_sent:66952
12:54:38.0887951 *:142] cluster_stats_messages_ping_received:33077
12:54:38.0887991 *:142] cluster_stats_messages_pong_received:33433
12:54:38.0888021 *:142] cluster_stats_messages_meet_received:9
12:54:38.0888061 *:142] cluster_stats_messages_received:66519
12:54:38.0888101 *:142] total_cluster_links_buffer_limit_exceeded:0
12:54:38.0888141 *:142] ------ CLUSTER NODES OUTPUT ------
...
12:54:38.0889171 *:142] 979a1fba895257d05a99232ef1f2c500d830a1d7 192.168.230.127:6380@16380,,tls-port=6380,shard-id=8fc0a4a9fe42cdaa280044fb808af8d0974774b6 slave f7df088a9934ff845fc642f53c3b5d1c11d98e9b 0 1747400077070 23 connected
12:54:38.0889211 *:142] ------ CLIENT LIST OUTPUT ------
...
12:54:38.1036471 *:142] ------ MODULES INFO OUTPUT ------
...
12:54:38.1044281 *:142] ------ CONFIG DEBUG OUTPUT ------
12:54:38.1044331 *:142] lazyfree-lazy-expire yes
12:54:38.1044391 *:142] slave-read-only yes
12:54:38.1044441 *:142] io-threads 1
12:54:38.1044501 *:142] debug-context ""
12:54:38.1044551 *:142] list-compress-depth 0
12:54:38.1044611 *:142] client-query-buffer-limit 1gb
12:54:38.1044671 *:142] lazyfree-lazy-user-del yes
12:54:38.1044721 *:142] lazyfree-lazy-eviction yes
12:54:38.1044781 *:142] sanitize-dump-payload no
12:54:38.1044831 *:142] replica-read-only yes
12:54:38.1044891 *:142] dual-channel-replication-enabled yes
12:54:38.1044941 *:142] repl-diskless-sync yes
12:54:38.1045001 *:142] repl-diskless-load swapdb
12:54:38.1045061 *:142] lazyfree-lazy-server-del yes
12:54:38.1045121 *:142] lazyfree-lazy-user-flush yes
12:54:38.1045171 *:142] proto-max-bulk-len 512mb
12:54:38.1045221 *:142] import-mode no
12:54:38.1045281 *:142] activedefrag no
12:54:38.1045331 *:142] ------ FAST MEMORY TEST ------
...
12:54:42.8677741 *:142] O.O.O.O.O
12:54:42.8678061 *:142] Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
12:54:42.8678111 *:142] ------ DUMPING CODE AROUND EIP ------
12:54:42.8678151 *:142] Symbol: expireScanCallback (base: 0x594d80)
12:54:42.8678181 *:142] Module: redis-server *:6380 [cluster] (base 0x400000)
12:54:42.8678221 *:142] $ xxd -r -p /tmp/dump.hex /tmp/dump.bin
12:54:42.8678251 *:142] $ objdump --adjust-vma=0x594d80 -D -b binary -m i386:x86-64 /tmp/dump.bin
12:54:42.8678291 *:142] ------
12:54:42.8678331 *:176] 19:M dump of function (hexdump of 141 bytes):
12:54:42.8678381 *:142] 554889e541554154534883ec08f64604010f84e9000000488b4610488b57084889fb488b3f4989c44929d44885c00f88cc0000004839d07c274d85e47e084c0163208343280148834310014883c4085b415c415d5dc3662e0f1f840000000000e8fbfdffff85c074d08b0dc14d2100488343180185c975c1e813b3feffe85e9eeeff8b15a84d210085d275ad8b
12:54:42.8678451 *:142] === VALKEY BUG REPORT END. Make sure to include from START to END. ===
12:54:42.8678481 *:142] Please report the crash by opening an issue on github:
...
12:54:49.0705981 *:345] Error during redis-server execution, errorsignal: segmentation fault (core dumped)
12:54:49.0706621 *:348] redis-server stopped
12:54:49.0706711 main.go:38] Shutdown handler called
12:54:49.0707671 *:313] http server stop error: http: Server closed
12:54:49.4207861 *:436] Failover error: FailoverError: failover failed
12:54:49.4213971 *:158] error happened during fail over and forget: cannot failover by address 192.168.230.105:6380: FailoverError: failover failed
12:54:49.4214211 main.go:42] fail over done, shut down
12:54:49.4214461 *:145] received Stop Signal...
12:54:49.4214701 main.go:22] run() exits with an error: redisnode.startAndMonitorError() stopped ungracefully signal: segmentation fault (core dumped)
Crash of the replica after an attempt of the failover:
12:54:41.0246611 *:174] 18:S Node 8c91ae6dfc552a8acf18ad4d246d46bf9c8eaba1 () reported node d17784575cc5e197323562d0860d931f4a7f379d () as not reachable.
...
12:54:41.0554441 *:174] 18:S Node 59ee1fc6f5dae4c78fc685480370d30ef41c2811 () reported node d17784575cc5e197323562d0860d931f4a7f379d () as not reachable.
12:54:41.1124691 *:174] 18:S Start of election delayed for 734 milliseconds (rank #0, primary rank #0, offset 5774182301).
12:54:41.2131731 *:174] 18:S Currently unable to failover: Waiting the delay before I can start a new failover.
12:54:41.9218651 *:174] 18:S Starting a failover election for epoch 24, node config epoch is 20
12:54:41.9297631 *:174] 18:S Currently unable to failover: Waiting for votes, but majority still not reached.
12:54:41.9298051 *:174] 18:S Needed quorum: 5. Number of votes received so far: 1
12:54:41.9323781 *:174] 18:S Failover election won: I'm the new primary.
12:54:41.9323981 *:174] 18:S configEpoch set to 24 after successful failover
12:54:41.9324041 *:174] 18:S Setting myself to primary in shard 2ce4771bf50965476a73c95bb892805fd5e78a2a after failover; my old primary is d17784575cc5e197323562d0860d931f4a7f379d ()
12:54:41.9324411 *:174] 18:M Connection with primary lost.
12:54:41.9324491 *:174] 18:M Caching the disconnected primary state.
12:54:41.9324691 *:229] log catcher is delaying logging cluster state
12:54:41.9326421 *:174] 18:M Discarding previously cached primary state.
12:54:41.9326751 *:174] 18:M Setting secondary replication ID to 60d37e92cae5c8144e67b2dbfdccbdcd4c0f7f4a, valid up to offset: 5774182302. New replication ID is 9d2c0159a00723b73d65b9b4cb87fabf69bc3931
...
12:54:44.1549381 *:174] 18:M Node f7df088a9934ff845fc642f53c3b5d1c11d98e9b () reported node d17784575cc5e197323562d0860d931f4a7f379d () as not reachable.
12:54:46.1701171 *:174] 18:M Node f5e128b37af36270c1f437ed74fb2a198d483664 () reported node d17784575cc5e197323562d0860d931f4a7f379d () as not reachable.
....
12:55:00.7085791 *:174] 18:M PONG contains mismatching sender ID. About node d17784575cc5e197323562d0860d931f4a7f379d () in shard 2ce4771bf50965476a73c95bb892805fd5e78a2a added 2333854 ms ago, having flags 7177
12:55:01.9056441 *:174] 18:M Cluster meet 192.168.230.105:6380 (user request from 'id=360 addr=192.168.2.227:46466 laddr=192.168.171.26:6380 fd=117 name=*redacted* user=*redacted* lib-name= lib-ver=').
12:55:02.0036611 *:174] 18:M Successfully completed handshake with c77dd66d63bffa699ba6f5ccdf550d281c49816e ()
12:55:02.0036841 *:174] 18:M Address updated for node c77dd66d63bffa699ba6f5ccdf550d281c49816e (), now 192.168.230.105:6380
...
12:55:02.3344811 *:174] 18:M Replica 192.168.230.105:6380 asks for synchronization
12:55:02.3344991 *:174] 18:M Full resync requested by replica 192.168.230.105:6380
12:55:02.3345051 *:174] 18:M Dual channel replication: Replica 192.168.230.105:6380 is capable of dual channel synchronization, and partial sync isn't possible. Full sync will continue with dedicated RDB channel.
12:55:02.3387921 *:174] 18:M Replica 192.168.230.105:6380 asks for synchronization
12:55:02.3388041 *:174] 18:M Starting BGSAVE for SYNC with target: replicas sockets using: dual-channel
12:55:02.3388101 *:174] 18:M Dual channel replication: Sending to replica 192.168.230.105:6380 RDB end offset 5778038404 and client-id 363
12:55:02.3590171 *:174] 18:M Background RDB transfer started by pid 30 to direct socket to replica
12:55:02.3646331 *:174] 18:M Replica 192.168.230.105:6380 asks for synchronization
12:55:02.3646751 *:174] 18:M Partial resynchronization request from 192.168.230.105:6380 accepted. Sending 0 bytes of backlog starting from offset 5778038405.
12:55:07.4825921 *:174] 18:M Dual channel replication: Replica 192.168.230.105:6380 rdb channel disconnected.
12:55:07.4830151 *:174] 18:M Synchronization with replica 192.168.230.105:6380 succeeded
...
12:55:07.4895211 *:174] 30:C Fork CoW for RDB: current 87 MB, peak 94 MB, average 62 MB
12:55:07.5544691 *:174] 18:M Background RDB transfer terminated with success
12:55:09.1229211 *:174] 18:M Cluster forget d17784575cc5e197323562d0860d931f4a7f379d (user request from 'id=371 addr=192.168.2.227:45980 laddr=192.168.171.26:6380 fd=117 name=*redacted* user=*redacted* lib-name= lib-ver=').
12:55:09.4237581 *:174] 18:M Failover auth granted to 979a1fba895257d05a99232ef1f2c500d830a1d7 () for epoch 26
12:55:09.4438241 *:174] 18:M Node f7df088a9934ff845fc642f53c3b5d1c11d98e9b () is now a replica of node 979a1fba895257d05a99232ef1f2c500d830a1d7 () in shard 8fc0a4a9fe42cdaa280044fb808af8d0974774b6
12:55:12.0966061 *:142] === VALKEY BUG REPORT START: Cut & paste starting from here ===
12:55:12.0966401 *:176] 18:M valkey 8.1.1 crashed by signal: 11, si_code: 1
12:55:12.0966471 *:176] 18:M Accessing address: 0x4
12:55:12.0966541 *:176] 18:M Crashed running the instruction at: 0x594d8d
12:55:12.0966601 *:142] ------ STACK TRACE ------
12:55:12.0966641 *:142] EIP:
12:55:12.0968301 *:142] redis-server *:6380 [cluster](expireScanCallback+0xd)[0x594d8d]
12:55:12.1002801 *:142] 20 bio_aof
12:55:12.1003081 *:142] /lib64/libc.so.6(+0xa3c1e)[0x7f1c23913c1e]
12:55:12.1003301 *:142] /lib64/libc.so.6(pthread_cond_wait+0x12a)[0x7f1c23916548]
12:55:12.1004691 *:142] redis-server *:6380 [cluster](bioProcessBackgroundJobs+0x16c)[0x53ff2c]
12:55:12.1005001 *:142] /lib64/libc.so.6(+0xa6f6c)[0x7f1c23916f6c]
12:55:12.1005601 *:142] /lib64/libc.so.6(clone+0x42)[0x7f1c2399e180]
12:55:12.1005661 *:142] 21 bio_lazy_free
12:55:12.1006181 *:142] /lib64/libc.so.6(+0xa3c1e)[0x7f1c23913c1e]
12:55:12.1006671 *:142] /lib64/libc.so.6(pthread_cond_wait+0x12a)[0x7f1c23916548]
12:55:12.1008311 *:142] redis-server *:6380 [cluster](bioProcessBackgroundJobs+0x16c)[0x53ff2c]
12:55:12.1008601 *:142] /lib64/libc.so.6(+0xa6f6c)[0x7f1c23916f6c]
12:55:12.1009261 *:142] /lib64/libc.so.6(clone+0x42)[0x7f1c2399e180]
12:55:12.1009351 *:142] 18 redis-server *
12:55:12.1009581 *:142] /lib64/libc.so.6(+0x57900)[0x7f1c238c7900]
12:55:12.1010811 *:142] redis-server *:6380 [cluster](expireScanCallback+0xd)[0x594d8d]
12:55:12.1011931 *:142] redis-server *:6380 [cluster](activeExpireCycle+0xb31)[0x5959d1]
12:55:12.1012811 *:142] redis-server *:6380 [cluster](databasesCron+0x4aa)[0x47828a]
12:55:12.1013511 *:142] redis-server *:6380 [cluster](serverCron+0x78a)[0x478e3a]
12:55:12.1014341 *:142] redis-server *:6380 [cluster][0x460df9]
12:55:12.1015161 *:142] redis-server *:6380 [cluster](aeMain+0x130)[0x46ad60]
12:55:12.1015901 *:142] redis-server *:6380 [cluster](main+0x4d7)[0x45f587]
12:55:12.1016131 *:142] /lib64/libc.so.6(+0x40e6c)[0x7f1c238b0e6c]
12:55:12.1016541 *:142] /lib64/libc.so.6(__libc_start_main+0x87)[0x7f1c238b0f35]
12:55:12.1017431 *:142] redis-server *:6380 [cluster](_start+0x21)[0x460cc1]
12:55:12.1017501 *:142] 19 bio_close_file
12:55:12.1018011 *:142] /lib64/libc.so.6(+0xa3c1e)[0x7f1c23913c1e]
12:55:12.1018281 *:142] /lib64/libc.so.6(pthread_cond_wait+0x12a)[0x7f1c23916548]
12:55:12.1019791 *:142] redis-server *:6380 [cluster](bioProcessBackgroundJobs+0x16c)[0x53ff2c]
12:55:12.1020341 *:142] /lib64/libc.so.6(+0xa6f6c)[0x7f1c23916f6c]
12:55:12.1020871 *:142] /lib64/libc.so.6(clone+0x42)[0x7f1c2399e180]
12:55:12.1020971 *:142] 4/4 expected stacktraces.
12:55:12.1021691 *:142] ------ STACK TRACE DONE ------
12:55:12.1021731 *:142] ------ REGISTERS ------
12:55:12.1021761 *:142] 18:M 16 May 2025 12:55:12.102 #
12:55:12.1021791 *:142] RAX:0000000000000060 RBX:00007f1c1082d280
12:55:12.1021821 *:142] RCX:0000000000000006 RDX:0000000000000040
12:55:12.1021861 *:142] RDI:00007ffdc0c06980 RSI:0000000000000000
12:55:12.1021891 *:142] RBP:00007ffdc0c06890 RSP:00007ffdc0c06870
12:55:12.1021921 *:142] R8 :fffffffffffff550 R9 :00007f1c23400c28
12:55:12.1021951 *:142] R10:0000000000000003 R11:00007f1c238481c0
12:55:12.1021991 *:142] R12:0000000000000006 R13:0000000000000001
12:55:12.1022021 *:142] R14:00007f1c1f7626a0 R15:0000000000000005
12:55:12.1022051 *:142] RIP:0000000000594d8d EFL:0000000000010202
12:55:12.1022081 *:142] CSGSFS:002b000000000033
12:55:12.1022131 *:174] 18:M hide-user-data-from-log is on, skip logging stack content to avoid spilling user data.
12:55:12.1022191 *:142] ------ INFO OUTPUT ------
12:55:12.1040221 *:142] # Server
12:55:12.1040351 *:142] redis_version:7.2.4
12:55:12.1040391 *:142] server_name:valkey
12:55:12.1040421 *:142] valkey_version:8.1.1
12:55:12.1040451 *:142] valkey_release_stage:ga
12:55:12.1040491 *:142] redis_git_sha1:00000000
12:55:12.1040521 *:142] redis_git_dirty:0
12:55:12.1040551 *:142] redis_build_id:bb1dae1294af36a1
12:55:12.1040591 *:142] server_mode:cluster
12:55:12.1040621 *:142] os:Linux 6.4.0-150600.23.38.1.29362.2.PTF.1237234-default x86_64
12:55:12.1040651 *:142] arch_bits:64
12:55:12.1040681 *:142] monotonic_clock:POSIX clock_gettime
12:55:12.1040711 *:142] multiplexing_api:epoll
12:55:12.1040741 *:142] gcc_version:13.3.1
12:55:12.1040781 *:142] process_id:18
12:55:12.1040811 *:142] process_supervised:no
12:55:12.1040841 *:142] run_id:8370e7578ae823c5674a97246787e65383c18fd1
12:55:12.1040941 *:142] tcp_port:6380
12:55:12.1040971 *:142] server_time_usec:1747400112096469
12:55:12.1041001 *:142] uptime_in_seconds:2348
12:55:12.1041041 *:142] uptime_in_days:0
12:55:12.1041071 *:142] hz:10
12:55:12.1041101 *:142] configured_hz:10
12:55:12.1041131 *:142] clients_hz:10
12:55:12.1041161 *:142] lru_clock:2569648
12:55:12.1041191 *:142] executable:/redis-server
12:55:12.1041221 *:142] config_file:/redis-conf/redis.conf
12:55:12.1041251 *:142] io_threads_active:0
12:55:12.1041281 *:142] availability_zone:
12:55:12.1041321 *:142] listener2:name=tls,bind=*,bind=-::*,port=6380
12:55:12.1041351 *:142] # Clients
12:55:12.1041381 *:142] connected_clients:77
12:55:12.1041421 *:142] cluster_connections:30
12:55:12.1041451 *:142] maxclients:8192
12:55:12.1041481 *:142] client_recent_max_input_buffer:65536
12:55:12.1041511 *:142] client_recent_max_output_buffer:20504
12:55:12.1041541 *:142] blocked_clients:2
12:55:12.1041571 *:142] tracking_clients:16
12:55:12.1041601 *:142] pubsub_clients:0
12:55:12.1041631 *:142] watching_clients:0
12:55:12.1041661 *:142] clients_in_timeout_table:0
12:55:12.1041691 *:142] total_watched_keys:0
12:55:12.1041721 *:142] total_blocking_keys:0
12:55:12.1041751 *:142] total_blocking_keys_on_nokey:0
12:55:12.1041781 *:142] paused_reason:none
12:55:12.1041821 *:142] paused_actions:none
12:55:12.1041851 *:142] paused_timeout_milliseconds:0
12:55:12.1041881 *:142] # Memory
12:55:12.1041911 *:142] used_memory:816118752
12:55:12.1041931 *:142] used_memory_human:778.31M
12:55:12.1042031 *:142] used_memory_rss:907456512
12:55:12.1042061 *:142] used_memory_rss_human:865.42M
12:55:12.1042101 *:142] used_memory_peak:1071460128
12:55:12.1042131 *:142] used_memory_peak_human:1021.82M
12:55:12.1042161 *:142] used_memory_peak_perc:76.17%
12:55:12.1042191 *:142] used_memory_overhead:148242024
12:55:12.1042221 *:142] used_memory_startup:3786896
12:55:12.1042261 *:142] used_memory_dataset:667876728
12:55:12.1042291 *:142] used_memory_dataset_perc:82.22%
12:55:12.1042321 *:142] allocator_allocated:818194568
12:55:12.1042351 *:142] allocator_active:873541632
12:55:12.1042381 *:142] allocator_resident:892317696
12:55:12.1042411 *:142] allocator_muzzy:0
12:55:12.1042441 *:142] total_system_memory:103352016896
12:55:12.1042471 *:142] total_system_memory_human:96.25G
12:55:12.1042511 *:142] used_memory_lua:71680
12:55:12.1042541 *:142] used_memory_vm_eval:71680
12:55:12.1042571 *:142] used_memory_lua_human:70.00K
12:55:12.1042611 *:142] used_memory_scripts_eval:12632
12:55:12.1042641 *:142] number_of_cached_scripts:10
12:55:12.1042711 *:142] number_of_functions:0
12:55:12.1042741 *:142] number_of_libraries:0
12:55:12.1042781 *:142] used_memory_vm_functions:33792
12:55:12.1042811 *:142] used_memory_vm_total:105472
12:55:12.1042841 *:142] used_memory_vm_total_human:103.00K
12:55:12.1042871 *:142] used_memory_functions:224
12:55:12.1042911 *:142] used_memory_scripts:12856
12:55:12.1042931 *:142] used_memory_scripts_human:12.55K
12:55:12.1042971 *:142] maxmemory:4294967296
12:55:12.1043001 *:142] maxmemory_human:4.00G
12:55:12.1043031 *:142] maxmemory_policy:noeviction
12:55:12.1043061 *:142] allocator_frag_ratio:1.07
12:55:12.1043101 *:142] allocator_frag_bytes:55347064
12:55:12.1043221 *:142] allocator_rss_ratio:1.02
12:55:12.1043251 *:142] allocator_rss_bytes:18776064
12:55:12.1043281 *:142] rss_overhead_ratio:1.02
12:55:12.1043311 *:142] rss_overhead_bytes:15138816
12:55:12.1043341 *:142] mem_fragmentation_ratio:1.11
12:55:12.1043371 *:142] mem_fragmentation_bytes:91332224
12:55:12.1043401 *:142] mem_not_counted_for_evict:0
12:55:12.1043441 *:142] mem_replication_backlog:134219180
12:55:12.1043471 *:142] mem_total_replication_buffers:134619400
12:55:12.1043501 *:142] mem_clients_slaves:401672
12:55:12.1043531 *:142] mem_clients_normal:713180
12:55:12.1043561 *:142] mem_cluster_links:32160
12:55:12.1043591 *:142] mem_aof_buffer:0
12:55:12.1043621 *:142] mem_allocator:jemalloc-5.3.0
12:55:12.1043661 *:142] mem_overhead_db_hashtable_rehashing:4352
12:55:12.1043691 *:142] active_defrag_running:0
12:55:12.1043721 *:142] lazyfree_pending_objects:0
12:55:12.1043751 *:142] lazyfreed_objects:15451
12:55:12.1043781 *:142] # Persistence
12:55:12.1043811 *:142] loading:0
12:55:12.1043841 *:142] async_loading:0
12:55:12.1043901 *:142] current_cow_peak:0
12:55:12.1043931 *:142] current_cow_size:0
12:55:12.1043961 *:142] current_cow_size_age:0
12:55:12.1044001 *:142] current_fork_perc:0.00
12:55:12.1044021 *:142] current_save_keys_processed:0
12:55:12.1044061 *:142] current_save_keys_total:0
12:55:12.1044091 *:142] rdb_changes_since_last_save:1594011
12:55:12.1044121 *:142] rdb_bgsave_in_progress:0
12:55:12.1044161 *:142] rdb_last_save_time:1747397764
12:55:12.1044191 *:142] rdb_last_bgsave_status:ok
12:55:12.1044221 *:142] rdb_last_bgsave_time_sec:5
12:55:12.1044251 *:142] rdb_current_bgsave_time_sec:-1
12:55:12.1044331 *:142] rdb_saves:0
12:55:12.1044371 *:142] rdb_last_cow_size:99454976
12:55:12.1044401 *:142] rdb_last_load_keys_expired:0
12:55:12.1044431 *:142] rdb_last_load_keys_loaded:43267
12:55:12.1044461 *:142] aof_enabled:0
12:55:12.1044491 *:142] aof_rewrite_in_progress:0
12:55:12.1044521 *:142] aof_rewrite_scheduled:0
12:55:12.1044551 *:142] aof_last_rewrite_time_sec:-1
12:55:12.1044581 *:142] aof_current_rewrite_time_sec:-1
12:55:12.1044611 *:142] aof_last_bgrewrite_status:ok
12:55:12.1044651 *:142] aof_rewrites:0
12:55:12.1044681 *:142] aof_rewrites_consecutive_failures:0
12:55:12.1044711 *:142] aof_last_write_status:ok
12:55:12.1044741 *:142] aof_last_cow_size:0
12:55:12.1044771 *:142] module_fork_in_progress:0
12:55:12.1044801 *:142] module_fork_last_cow_size:0
12:55:12.1044841 *:142] # Stats
12:55:12.1044871 *:142] total_connections_received:358
12:55:12.1044901 *:142] total_commands_processed:2037195
12:55:12.1044931 *:142] instantaneous_ops_per_sec:9511
12:55:12.1044971 *:142] total_net_input_bytes:5733628668
12:55:12.1045001 *:142] total_net_output_bytes:287444927
12:55:12.1045031 *:142] total_net_repl_input_bytes:5687237720
12:55:12.1045061 *:142] total_net_repl_output_bytes:7217701
12:55:12.1045091 *:142] instantaneous_input_kbps:4823.05
12:55:12.1045121 *:142] instantaneous_output_kbps:8296.21
12:55:12.1045151 *:142] instantaneous_input_repl_kbps:0.00
12:55:12.1045181 *:142] instantaneous_output_repl_kbps:4244.82
12:55:12.1045211 *:142] rejected_connections:0
12:55:12.1045241 *:142] sync_full:1
12:55:12.1045281 *:142] sync_partial_ok:1
12:55:12.1045311 *:142] sync_partial_err:0
12:55:12.1045341 *:142] expired_keys:7144
12:55:12.1045431 *:142] expired_stale_perc:16.20
12:55:12.1045461 *:142] expired_time_cap_reached_count:0
12:55:12.1045501 *:142] expire_cycle_cpu_milliseconds:63
12:55:12.1045531 *:142] evicted_keys:0
12:55:12.1045561 *:142] evicted_clients:0
12:55:12.1045591 *:142] evicted_scripts:0
12:55:12.1045621 *:142] total_eviction_exceeded_time:0
12:55:12.1045651 *:142] current_eviction_exceeded_time:0
12:55:12.1045691 *:142] keyspace_hits:803129
12:55:12.1045721 *:142] keyspace_misses:121727
12:55:12.1045751 *:142] pubsub_channels:0
12:55:12.1045781 *:142] pubsub_patterns:0
12:55:12.1045811 *:142] pubsubshard_channels:0
12:55:12.1045851 *:142] latest_fork_usec:20097
12:55:12.1045881 *:142] total_forks:1
12:55:12.1045911 *:142] migrate_cached_sockets:0
12:55:12.1045941 *:142] slave_expires_tracked_keys:0
12:55:12.1045971 *:142] active_defrag_hits:0
12:55:12.1046001 *:142] active_defrag_misses:0
12:55:12.1046041 *:142] active_defrag_key_hits:0
12:55:12.1046061 *:142] active_defrag_key_misses:0
12:55:12.1046101 *:142] total_active_defrag_time:0
12:55:12.1046131 *:142] current_active_defrag_time:0
12:55:12.1046161 *:142] tracking_total_keys:1
12:55:12.1046191 *:142] tracking_total_items:3
12:55:12.1046221 *:142] tracking_total_prefixes:12
12:55:12.1046251 *:142] unexpected_error_replies:0
12:55:12.1046281 *:142] total_error_replies:11660
12:55:12.1046311 *:142] dump_payload_sanitizations:0
12:55:12.1046351 *:142] total_reads_processed:664825
12:55:12.1046381 *:142] total_writes_processed:57948
12:55:12.1046411 *:142] io_threaded_reads_processed:0
12:55:12.1046441 *:142] io_threaded_writes_processed:0
12:55:12.1046541 *:142] io_threaded_freed_objects:0
12:55:12.1046571 *:142] io_threaded_accept_processed:0
12:55:12.1046601 *:142] io_threaded_poll_processed:0
12:55:12.1046631 *:142] io_threaded_total_prefetch_batches:0
12:55:12.1046661 *:142] io_threaded_total_prefetch_entries:0
12:55:12.1046701 *:142] client_query_buffer_limit_disconnections:0
12:55:12.1046731 *:142] client_output_buffer_limit_disconnections:0
12:55:12.1046761 *:142] reply_buffer_shrinks:1501
12:55:12.1046791 *:142] reply_buffer_expands:1481
12:55:12.1046821 *:142] eventloop_cycles:842805
12:55:12.1046851 *:142] eventloop_duration_sum:80498704
12:55:12.1046881 *:142] eventloop_duration_cmd_sum:19350464
12:55:12.1046911 *:142] instantaneous_eventloop_cycles_per_sec:1211
12:55:12.1046951 *:142] instantaneous_eventloop_duration_usec:110
12:55:12.1046981 *:142] acl_access_denied_auth:0
12:55:12.1047011 *:142] acl_access_denied_cmd:0
12:55:12.1047041 *:142] acl_access_denied_key:0
12:55:12.1047071 *:142] acl_access_denied_channel:0
12:55:12.1047111 *:142] # Replication
12:55:12.1047141 *:142] role:master
12:55:12.1047171 *:142] connected_slaves:1
12:55:12.1047431 *:142] slave0:ip=192.168.230.105,port=6380,state=online,offset=5779384489,lag=1,type=replica
12:55:12.1047461 *:142] replicas_waiting_psync:0
12:55:12.1047491 *:142] master_failover_state:no-failover
12:55:12.1047531 *:142] master_replid:9d2c0159a00723b73d65b9b4cb87fabf69bc3931
12:55:12.1047561 *:142] master_replid2:60d37e92cae5c8144e67b2dbfdccbdcd4c0f7f4a
12:55:12.1047591 *:142] master_repl_offset:5785270940
12:55:12.1047631 *:142] second_repl_offset:5774182302
12:55:12.1047661 *:142] repl_backlog_active:1
12:55:12.1047711 *:142] repl_backlog_size:134217728
12:55:12.1047761 *:142] repl_backlog_first_byte_offset:5651020025
12:55:12.1047811 *:142] repl_backlog_histlen:134250916
12:55:12.1047851 *:142] # CPU
12:55:12.1047991 *:142] used_cpu_sys:29.023018
12:55:12.1048041 *:142] used_cpu_user:57.365797
12:55:12.1048111 *:142] used_cpu_sys_children:1.128594
12:55:12.1048161 *:142] used_cpu_user_children:4.006123
12:55:12.1048211 *:142] used_cpu_sys_main_thread:28.887097
12:55:12.1048261 *:142] used_cpu_user_main_thread:57.271144
12:55:12.1048311 *:142] # Modules
...
12:55:12.1048451 *:142] # Commandstats
...
12:55:12.1053491 *:142] # Errorstats
12:55:12.1053541 *:142] errorstat_LOADING:count=3
12:55:12.1053601 *:142] errorstat_MOVED:count=11656
12:55:12.1053661 *:142] errorstat_NOSCRIPT:count=1
12:55:12.1053711 *:142] # Latencystats
12:55:12.1053931 *:142] # Cluster
12:55:12.1053981 *:142] cluster_enabled:1
12:55:12.1054041 *:142] # Keyspace
12:55:12.1054101 *:142] db0:keys=251642,expires=1254,avg_ttl=936779
12:55:12.1054151 *:142] # Cluster info
12:55:12.1054211 *:142] cluster_state:ok
12:55:12.1054271 *:142] cluster_slots_assigned:16384
12:55:12.1054331 *:142] cluster_slots_ok:16384
12:55:12.1054381 *:142] cluster_slots_pfail:0
12:55:12.1054431 *:142] cluster_slots_fail:0
12:55:12.1054491 *:142] cluster_known_nodes:16
12:55:12.1054541 *:142] cluster_size:8
12:55:12.1054601 *:142] cluster_current_epoch:26
12:55:12.1054661 *:142] cluster_my_epoch:24
12:55:12.1054711 *:142] cluster_stats_messages_ping_sent:33416
12:55:12.1054761 *:142] cluster_stats_messages_pong_sent:33446
12:55:12.1054821 *:142] cluster_stats_messages_meet_sent:5
12:55:12.1054881 *:142] cluster_stats_messages_auth-req_sent:15
12:55:12.1054931 *:142] cluster_stats_messages_auth-ack_sent:1
12:55:12.1054981 *:142] cluster_stats_messages_sent:66883
12:55:12.1055031 *:142] cluster_stats_messages_ping_received:33401
12:55:12.1055091 *:142] cluster_stats_messages_pong_received:33773
12:55:12.1055151 *:142] cluster_stats_messages_meet_received:15
12:55:12.1055201 *:142] cluster_stats_messages_fail_received:1
12:55:12.1055261 *:142] cluster_stats_messages_auth-req_received:1
12:55:12.1055311 *:142] cluster_stats_messages_auth-ack_received:7
12:55:12.1055371 *:142] cluster_stats_messages_received:67198
12:55:12.1055431 *:142] total_cluster_links_buffer_limit_exceeded:0
12:55:12.1055481 *:142] ------ CLUSTER NODES OUTPUT ------
...
12:55:12.1056601 *:142] ------ CLIENT LIST OUTPUT ------
...
12:55:12.1122281 *:142] ------ MODULES INFO OUTPUT ------
...
12:55:12.1206321 *:142] ------ CONFIG DEBUG OUTPUT ------
12:55:12.1206381 *:142] debug-context ""
12:55:12.1206431 *:142] lazyfree-lazy-eviction yes
12:55:12.1206491 *:142] lazyfree-lazy-user-del yes
12:55:12.1206561 *:142] repl-diskless-load swapdb
12:55:12.1206621 *:142] repl-diskless-sync yes
12:55:12.1206671 *:142] client-query-buffer-limit 1gb
12:55:12.1206731 *:142] import-mode no
12:55:12.1206781 *:142] lazyfree-lazy-user-flush yes
12:55:12.1206841 *:142] activedefrag no
12:55:12.1206891 *:142] lazyfree-lazy-server-del yes
12:55:12.1206951 *:142] io-threads 1
12:55:12.1207011 *:142] replica-read-only yes
12:55:12.1207071 *:142] lazyfree-lazy-expire yes
12:55:12.1207151 *:142] sanitize-dump-payload no
12:55:12.1207551 *:142] dual-channel-replication-enabled yes
12:55:12.1207621 *:142] proto-max-bulk-len 512mb
12:55:12.1207671 *:142] slave-read-only yes
12:55:12.1207721 *:142] list-compress-depth 0
12:55:12.1207771 *:142] ------ FAST MEMORY TEST ------
...
12:55:17.0266851 *:142] Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
12:55:17.0270611 *:142] ------ DUMPING CODE AROUND EIP ------
12:55:17.0270971 *:142] Symbol: expireScanCallback (base: 0x594d80)
12:55:17.0271041 *:142] Module: redis-server *:6380 [cluster] (base 0x400000)
12:55:17.0271091 *:142] $ xxd -r -p /tmp/dump.hex /tmp/dump.bin
12:55:17.0271151 *:142] $ objdump --adjust-vma=0x594d80 -D -b binary -m i386:x86-64 /tmp/dump.bin
12:55:17.0271201 *:142] ------
12:55:17.0271251 *:176] 18:M dump of function (hexdump of 141 bytes):
12:55:17.0271341 *:142] 554889e541554154534883ec08f64604010f84e9000000488b4610488b57084889fb488b3f4989c44929d44885c00f88cc0000004839d07c274d85e47e084c0163208343280148834310014883c4085b415c415d5dc3662e0f1f840000000000e8fbfdffff85c074d08b0dc14d2100488343180185c975c1e813b3feffe85e9eeeff8b15a84d210085d275ad8b
12:55:17.0271531 *:142] === VALKEY BUG REPORT END. Make sure to include from START to END. ===
12:55:17.0271601 *:142] Please report the crash by opening an issue on github:
...
12:55:23.1525241 *:345] Error during redis-server execution, errorsignal: segmentation fault (core dumped)
12:55:23.1525831 *:348] redis-server stopped
12:55:23.1525891 main.go:38] Shutdown handler called
12:55:23.1526891 *:313] http server stop error: http: Server closed
12:55:23.4202991 *:436] Failover error: FailoverError: failover failed
12:55:23.4208961 *:158] error happened during fail over and forget: cannot failover by address 192.168.171.26:6380: FailoverError: failover failed
12:55:23.4209191 main.go:42] fail over done, shut down
12:55:23.4209391 *:145] received Stop Signal...
12:55:23.4209501 main.go:22] run() exits with an error: redisnode.startAndMonitorError() stopped ungracefully signal: segmentation fault (core dumped)
Crash report
Additional information