Crash during merges #92915
Description
Company or project name
No response
Describe what's wrong
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729062 [ 87 ] {} <Fatal> BaseDaemon: Report this error to https://github.com/ClickHouse/ClickHouse/issues
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729328 [ 87 ] {} <Fatal> BaseDaemon: Changed settings: connect_timeout = 100., connect_timeout_with_failover_ms = 3000, connect_timeout_with_failover_secure_ms = 3000, receive_timeout = 600., send_timeout = 600., tcp_keep_alive_timeout = 600., hedged_connection_timeout_ms = 300, distributed_foreground_insert = true, load_balancing = 'nearest_hostname', distributed_aggregation_memory_efficient = true, do_not_merge_across_partitions_select_final = true, log_queries = true, prefer_localhost_replica = false, parallel_view_processing = true, query_plan_max_limit_for_lazy_materialization = 10000, workload = 'default', os_threads_nice_value_query = 2, enable_full_text_index = true, output_format_parquet_string_as_string = true
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729357 [ 87 ] {} <Fatal> BaseDaemon: ########## Short fault info ############
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729367 [ 87 ] {} <Fatal> BaseDaemon: (version 25.12.1.649 (official build), build id: ED4E3839527ED91CBD658B9490E93BEC010D4119, git hash: bf4280aa19d3bade619578a749919e25ce490861, architecture: x86_64) (from thread 671) Received signal 11
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729370 [ 87 ] {} <Fatal> BaseDaemon: Signal description: Segmentation fault
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729375 [ 87 ] {} <Fatal> BaseDaemon: Address: 0x39. Access: read. Address not mapped to object.
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729379 [ 87 ] {} <Fatal> BaseDaemon: Stack trace: 0x000000001cb404a3 0x000000001c614cfa 0x000000001c61458e 0x000000001c6135ad 0x000000001ca4f2b2 0x000000001c646397 0x000000001c64e21d 0x00000000164f944b 0x0000000016500266 0x00000000164f67cf 0x00000000164fdb1a 0x00007fd08950bac3 0x00007fd08959ca74
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729383 [ 87 ] {} <Fatal> BaseDaemon: ########################################
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729390 [ 87 ] {} <Fatal> BaseDaemon: (version 25.12.1.649 (official build), build id: ED4E3839527ED91CBD658B9490E93BEC010D4119, git hash: bf4280aa19d3bade619578a749919e25ce490861) (from thread 671) (query_id: 3249988f-6814-4ce5-b445-63a74ab9cb72::20251213_0_485_106_514) (query: ) Received signal Segmentation fault (11)
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729393 [ 87 ] {} <Fatal> BaseDaemon: Address: 0x39. Access: read. Address not mapped to object.
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729396 [ 87 ] {} <Fatal> BaseDaemon: Stack trace: 0x000000001cb404a3 0x000000001c614cfa 0x000000001c61458e 0x000000001c6135ad 0x000000001ca4f2b2 0x000000001c646397 0x000000001c64e21d 0x00000000164f944b 0x0000000016500266 0x00000000164f67cf 0x00000000164fdb1a 0x00007fd08950bac3 0x00007fd08959ca74
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729440 [ 87 ] {} <Fatal> BaseDaemon: 2. DB::BuildTextIndexTransform::getSegments(unsigned long, unsigned long) const @ 0x000000001cb404a3
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729454 [ 87 ] {} <Fatal> BaseDaemon: 3. DB::MergeTask::MergeTextIndexStage::prepare() const @ 0x000000001c614cfa
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729462 [ 87 ] {} <Fatal> BaseDaemon: 4. DB::MergeTask::MergeTextIndexStage::execute() @ 0x000000001c61458e
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729470 [ 87 ] {} <Fatal> BaseDaemon: 5. DB::MergeTask::execute() @ 0x000000001c6135ad
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729479 [ 87 ] {} <Fatal> BaseDaemon: 6. DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x000000001ca4f2b2
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729487 [ 87 ] {} <Fatal> BaseDaemon: 7. DB::TaskRuntimeData::executeStep() const @ 0x000000001c646397
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729498 [ 87 ] {} <Fatal> BaseDaemon: 8. DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001c64e21d
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729512 [ 87 ] {} <Fatal> BaseDaemon: 9. ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::worker() @ 0x00000000164f944b
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729534 [ 87 ] {} <Fatal> BaseDaemon: 10. void std::__function::__policy_func<void ()>::__call_func[abi:ne210105]<ThreadFromGlobalPoolImpl<false, true>::ThreadFromGlobalPoolImpl<void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::*)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool*>(void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool*&&)::'lambda'()>(std::__function::__policy_storage const*) @ 0x0000000016500266
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729550 [ 87 ] {} <Fatal> BaseDaemon: 11. ThreadPoolImpl<std::thread>::ThreadFromThreadPool::worker() @ 0x00000000164f67cf
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729563 [ 87 ] {} <Fatal> BaseDaemon: 12. void* std::__thread_proxy[abi:ne210105]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*>>(void*) @ 0x00000000164fdb1a
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729590 [ 87 ] {} <Fatal> BaseDaemon: 13. ? @ 0x0000000000094ac3
[2025-12-23 16:07:12] 2025.12.23 15:07:12.729600 [ 87 ] {} <Fatal> BaseDaemon: 14. __clone @ 0x0000000000125a74
Does it reproduce on the most recent release?
Yes
How to reproduce
Unfortunately has no reproducible example.
All works fine on 25.11 and after update to 25.12 cluster crashes after some time and can not start.
I used text indexes and after update to 25.12 I dropped, recreate and materialized.
To reduce memory usage I used
<merge_tree>
<merge_max_block_size>512</merge_max_block_size>
</merge_tree>After some time cluster crashed. Probably during merges.
Expected behavior
No response
Error message and/or stacktrace
No response
Additional context
Maybe related
#92250