Skip to content

Assert in DatabaseReplicatedWorker.cpp:317 has failed #68572

@KochetovNicolai

Description

@KochetovNicolai

https://s3.amazonaws.com/clickhouse-test-reports/0/c2a6a78ad359ccc85e16bedc6d1051177437490c/stress_test__debug_.html

assert(zookeeper->expired() || current_task <= entry_name);

2024.08.16 04:10:30.576520 [ 98815 ] {} <Fatal> BaseDaemon: ########## Short fault info ############
2024.08.16 04:10:30.579527 [ 98815 ] {} <Fatal> BaseDaemon: (version 24.9.1.386 (official build), build id: 4C8C905DD5DF60E0EE6FBC8C91F2377550403A75, git hash: c2a6a78ad359ccc85e16bedc6d1051177437490c) (from thread 12745) Received signal 6
2024.08.16 04:10:30.623721 [ 98815 ] {} <Fatal> BaseDaemon: Signal description: Aborted
2024.08.16 04:10:30.628768 [ 98815 ] {} <Fatal> BaseDaemon: 
2024.08.16 04:10:30.636745 [ 98815 ] {} <Fatal> BaseDaemon: Stack trace: 0x000000000c62c788 0x000000000c87d13e 0x00007ff2781ce520 0x00007ff2782229fd 0x00007ff2781ce476 0x00007ff2781b47f3 0x00007ff2781b471b 0x00007ff2781c5e96 0x0000000010d65fd4 0x0000000010d62970 0x0000000010d429e6 0x00000000114a2e63 0x00000000114ace5e 0x000000001186a0d6 0x00000000118658bd 0x0000000012bd9c10 0x0000000012bef979 0x0000000016414507 0x00000000164149de 0x00000000163c11f2 0x00000000163bef03 0x00007ff278220ac3 0x00007ff2782b2850
2024.08.16 04:10:30.638114 [ 98815 ] {} <Fatal> BaseDaemon: ########################################
2024.08.16 04:10:30.645224 [ 98815 ] {} <Fatal> BaseDaemon: (version 24.9.1.386 (official build), build id: 4C8C905DD5DF60E0EE6FBC8C91F2377550403A75, git hash: c2a6a78ad359ccc85e16bedc6d1051177437490c) (from thread 12745) (query_id: 3354145e-a31b-4640-b778-12bdab7dba73) (query: CREATE TABLE rdb_test_15_4.rmt (`n` int) ENGINE = ReplicatedMergeTree ORDER BY n SETTINGS index_granularity = 12530, min_bytes_for_wide_part = 0, ratio_of_defaults_for_sparse_serialization = 0., replace_long_file_name_to_hash = false, max_file_name_length = 46, min_bytes_for_full_part_storage = 161179592, compact_parts_max_bytes_to_buffer = 450560470, compact_parts_max_granules_to_buffer = 172, compact_parts_merge_max_bytes_to_prefetch_part = 33412983, merge_max_block_size = 21835, old_parts_lifetime = 10., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1, vertical_merge_algorithm_min_columns_to_activate = 1, min_merge_bytes_to_use_direct_io = 5755625337, index_granularity_bytes = 4238708, concurrent_part_removal_threshold = 8, allow_vertical_merges_from_compact_to_wide_parts = false, cache_populated_by_fetch = true, marks_compress_block_size = 65169, primary_key_compress_block_size = 78653) Received signal Aborted (6)
2024.08.16 04:10:30.647692 [ 98815 ] {} <Fatal> BaseDaemon: 
2024.08.16 04:10:30.651743 [ 98815 ] {} <Fatal> BaseDaemon: Stack trace: 0x000000000c62c788 0x000000000c87d13e 0x00007ff2781ce520 0x00007ff2782229fd 0x00007ff2781ce476 0x00007ff2781b47f3 0x00007ff2781b471b 0x00007ff2781c5e96 0x0000000010d65fd4 0x0000000010d62970 0x0000000010d429e6 0x00000000114a2e63 0x00000000114ace5e 0x000000001186a0d6 0x00000000118658bd 0x0000000012bd9c10 0x0000000012bef979 0x0000000016414507 0x00000000164149de 0x00000000163c11f2 0x00000000163bef03 0x00007ff278220ac3 0x00007ff2782b2850
2024.08.16 04:10:30.805239 [ 98815 ] {} <Fatal> BaseDaemon: 0.0. inlined from /build/src/Common/StackTrace.cpp:349: StackTrace::tryCapture()
2024.08.16 04:10:30.817993 [ 98815 ] {} <Fatal> BaseDaemon: 0. /build/src/Common/StackTrace.cpp:318: StackTrace::StackTrace(ucontext_t const&) @ 0x000000000c62c788
2024.08.16 04:10:30.928235 [ 98815 ] {} <Fatal> BaseDaemon: 1. /build/src/Common/SignalHandlers.cpp:0: signalHandler(int, siginfo_t*, void*) @ 0x000000000c87d13e
2024.08.16 04:10:30.958635 [ 98815 ] {} <Fatal> BaseDaemon: 2. ? @ 0x00007ff2781ce520
2024.08.16 04:10:30.971208 [ 98815 ] {} <Fatal> BaseDaemon: 3. ? @ 0x00007ff2782229fd
2024.08.16 04:10:30.976735 [ 98815 ] {} <Fatal> BaseDaemon: 4. ? @ 0x00007ff2781ce476
2024.08.16 04:10:30.998751 [ 98815 ] {} <Fatal> BaseDaemon: 5. ? @ 0x00007ff2781b47f3
2024.08.16 04:10:31.000071 [ 98815 ] {} <Fatal> BaseDaemon: 6. ? @ 0x00007ff2781b471b
2024.08.16 04:10:31.004359 [ 98815 ] {} <Fatal> BaseDaemon: 7. ? @ 0x00007ff2781c5e96
2024.08.16 04:10:31.112959 [ 98815 ] {} <Fatal> BaseDaemon: 8.0. inlined from /build/contrib/llvm-project/libcxx/include/string_view:305: basic_string_view
2024.08.16 04:10:31.121528 [ 98815 ] {} <Fatal> BaseDaemon: 8.1. inlined from /build/contrib/llvm-project/libcxx/include/string:903: String::operator std::basic_string_view<char, std::char_traits<char>>[abi:v15007]() const
2024.08.16 04:10:31.127854 [ 98815 ] {} <Fatal> BaseDaemon: 8.2. inlined from /build/contrib/llvm-project/libcxx/include/string:4005: String::compare[abi:v15007](String const&) const
2024.08.16 04:10:31.137324 [ 98815 ] {} <Fatal> BaseDaemon: 8.3. inlined from /build/contrib/llvm-project/libcxx/include/string:4238: bool std::operator<[abi:v15007]<char, std::char_traits<char>, std::allocator<char>>(String const&, String const&)
2024.08.16 04:10:31.142899 [ 98815 ] {} <Fatal> BaseDaemon: 8.4. inlined from /build/contrib/llvm-project/libcxx/include/string:4296: bool std::operator<=[abi:v15007]<char, std::char_traits<char>, std::allocator<char>>(String const&, String const&)
2024.08.16 04:10:31.145291 [ 98815 ] {} <Fatal> BaseDaemon: 8. /build/src/Databases/DatabaseReplicatedWorker.cpp:317: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr<DB::Context const>)::$_0::operator()() const @ 0x0000000010d65fd4
2024.08.16 04:10:31.320465 [ 98815 ] {} <Fatal> BaseDaemon: 9.0. inlined from /build/contrib/llvm-project/libcxx/include/__mutex_base:425: bool std::condition_variable::wait_until<std::chrono::steady_clock, std::chrono::duration<long long, std::ratio<1l, 1000000000l>>, DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr<DB::Context const>)::$_0>(std::unique_lock<std::mutex>&, std::chrono::time_point<std::chrono::steady_clock, std::chrono::duration<long long, std::ratio<1l, 1000000000l>>> const&, DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr<DB::Context const>)::$_0)
2024.08.16 04:10:31.339909 [ 98815 ] {} <Fatal> BaseDaemon: 9.1. inlined from /build/contrib/llvm-project/libcxx/include/__mutex_base:471: bool std::condition_variable::wait_for[abi:v15007]<long long, std::ratio<1l, 1l>, DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr<DB::Context const>)::$_0>(std::unique_lock<std::mutex>&, std::chrono::duration<long long, std::ratio<1l, 1l>> const&, DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr<DB::Context const>)::$_0)
2024.08.16 04:10:31.348751 [ 98815 ] {} <Fatal> BaseDaemon: 9. /build/src/Databases/DatabaseReplicatedWorker.cpp:315: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr<DB::Context const>) @ 0x0000000010d62970
2024.08.16 04:10:31.939470 [ 98815 ] {} <Fatal> BaseDaemon: 10.0. inlined from /build/contrib/llvm-project/libcxx/include/vector:676: vector
2024.08.16 04:10:31.942245 [ 98815 ] {} <Fatal> BaseDaemon: 10. /build/src/Databases/DatabaseReplicated.cpp:1009: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::QueryFlags) @ 0x0000000010d429e6
2024.08.16 04:10:32.225306 [ 98815 ] {} <Fatal> BaseDaemon: 11.0. inlined from /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: ~shared_ptr
2024.08.16 04:10:32.316080 [ 98815 ] {} <Fatal> BaseDaemon: 11. /build/src/Interpreters/InterpreterCreateQuery.cpp:1446: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000114a2e63
2024.08.16 04:10:32.715285 [ 98815 ] {} <Fatal> BaseDaemon: 12. /build/src/Interpreters/InterpreterCreateQuery.cpp:1966: DB::InterpreterCreateQuery::execute() @ 0x00000000114ace5e
2024.08.16 04:10:32.889141 [ 98815 ] {} <Fatal> BaseDaemon: 13. /build/src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000001186a0d6
2024.08.16 04:10:33.266872 [ 98815 ] {} <Fatal> BaseDaemon: 14. /build/src/Interpreters/executeQuery.cpp:1395: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000118658bd
2024.08.16 04:10:33.391685 [ 98815 ] {} <Fatal> BaseDaemon: 15.0. inlined from /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: shared_ptr
2024.08.16 04:10:33.394719 [ 98815 ] {} <Fatal> BaseDaemon: 15.1. inlined from /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: std::shared_ptr<DB::IAST>::operator=[abi:v15007](std::shared_ptr<DB::IAST>&&)
2024.08.16 04:10:33.401431 [ 98815 ] {} <Fatal> BaseDaemon: 15.2. inlined from /build/contrib/llvm-project/libcxx/include/tuple:1228: _ZNSt3__15tupleIJRNS_10shared_ptrIN2DB4IASTEEERNS2_7BlockIOEEEaSB6v15007IS4_S6_TnNS_9enable_ifIXsr21_EnableAssignFromPairILb0EONS_4pairIT_T0_EEEE5valueEiE4typeELi0EEERS8_SF_
2024.08.16 04:10:33.403534 [ 98815 ] {} <Fatal> BaseDaemon: 15. /build/src/Server/TCPHandler.cpp:564: DB::TCPHandler::runImpl() @ 0x0000000012bd9c10
2024.08.16 04:10:33.782136 [ 98815 ] {} <Fatal> BaseDaemon: 16.0. inlined from /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: shared_ptr
2024.08.16 04:10:33.792860 [ 98815 ] {} <Fatal> BaseDaemon: 16.1. inlined from /build/src/Common/logger_useful.h:23: impl::getLoggerHelper(std::shared_ptr<Poco::Logger> const&)
2024.08.16 04:10:33.797303 [ 98815 ] {} <Fatal> BaseDaemon: 16. /build/src/Server/TCPHandler.cpp:2486: DB::TCPHandler::run() @ 0x0000000012bef979
2024.08.16 04:10:33.876699 [ 98815 ] {} <Fatal> BaseDaemon: 17. /build/base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000016414507
2024.08.16 04:10:33.887811 [ 98815 ] {} <Fatal> BaseDaemon: 18.0. inlined from /build/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: std::default_delete<Poco::Net::TCPServerConnection>::operator()[abi:v15007](Poco::Net::TCPServerConnection*) const
2024.08.16 04:10:33.978245 [ 98815 ] {} <Fatal> BaseDaemon: 18.1. inlined from /build/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:305: std::unique_ptr<Poco::Net::TCPServerConnection, std::default_delete<Poco::Net::TCPServerConnection>>::reset[abi:v15007](Poco::Net::TCPServerConnection*)
2024.08.16 04:10:34.006751 [ 98815 ] {} <Fatal> BaseDaemon: 18.2. inlined from /build/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:259: ~unique_ptr
2024.08.16 04:10:34.008659 [ 98815 ] {} <Fatal> BaseDaemon: 18. /build/base/poco/Net/src/TCPServerDispatcher.cpp:116: Poco::Net::TCPServerDispatcher::run() @ 0x00000000164149de
2024.08.16 04:10:34.028681 [ 98815 ] {} <Fatal> BaseDaemon: 19. /build/base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x00000000163c11f2
2024.08.16 04:10:34.043123 [ 98815 ] {} <Fatal> BaseDaemon: 20.0. inlined from /build/base/poco/Foundation/include/Poco/AutoPtr.h:77: AutoPtr
2024.08.16 04:10:34.046454 [ 98815 ] {} <Fatal> BaseDaemon: 20. /build/base/poco/Foundation/src/Thread_POSIX.cpp:332: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000163bef03
2024.08.16 04:10:34.052405 [ 98815 ] {} <Fatal> BaseDaemon: 21. ? @ 0x00007ff278220ac3
2024.08.16 04:10:34.056185 [ 98815 ] {} <Fatal> BaseDaemon: 22. ? @ 0x00007ff2782b2850
2024.08.16 04:10:34.314150 [ 98815 ] {} <Fatal> BaseDaemon: Integrity check of the executable successfully passed (checksum: BE10B2A7FFF31C014FE6BA8FB8E98802)
2024.08.16 04:10:37.254711 [ 98815 ] {} <Fatal> BaseDaemon: Report this error to https://github.com/ClickHouse/ClickHouse/issues
2024.08.16 04:10:37.260171 [ 98815 ] {} <Fatal> BaseDaemon: Changed settings: min_compress_block_size = 1420413, max_compress_block_size = 888925, max_block_size = 95519, min_external_table_block_size_bytes = 0, max_joined_block_size_rows = 21114, max_insert_threads = 3, max_threads = 1, max_parsing_threads = 1, max_read_buffer_size = 965260, connect_timeout_with_failover_ms = 2000, connect_timeout_with_failover_secure_ms = 3000, idle_connection_timeout = 36000, s3_check_objects_after_upload = true, stream_like_engine_allow_direct_select = true, enable_multiple_prewhere_read_steps = false, replication_wait_for_inactive_replica_timeout = 30, compile_sort_description = false, min_count_to_compile_sort_description = 0, group_by_two_level_threshold = 1, distributed_aggregation_memory_efficient = false, allow_nonconst_timezone_arguments = true, group_by_use_nulls = true, min_chunk_bytes_for_parallel_parsing = 12482733, output_format_parallel_formatting = false, merge_tree_coarse_index_granularity = 29, min_bytes_to_use_direct_io = 10737418240, min_bytes_to_use_mmap_io = 10737418240, log_queries = true, insert_quorum_timeout = 60000, table_function_remote_max_addresses = 200, merge_tree_read_split_ranges_into_intersecting_and_non_intersecting_injection_probability = 0.23000000417232513, http_wait_end_of_query = true, http_response_buffer_size = 6283571, fsync_metadata = false, http_send_timeout = 60., http_receive_timeout = 60., opentelemetry_start_trace_probability = 0.10000000149011612, prefer_external_sort_block_bytes = 0, max_bytes_before_external_sort = 4641009885, max_bytes_before_remerge_sort = 1711085859, max_execution_time = 60., join_algorithm = 'full_sorting_merge', cross_join_min_rows_to_compress = 0, cross_join_min_bytes_to_compress = 0, max_memory_usage = 10000000000, max_memory_usage_for_user = 22114056192, max_untracked_memory = 1048576, memory_profiler_step = 1048576, log_comment = '02447_drop_database_replica.sh', send_logs_level = 'warning', prefer_localhost_replica = false, optimize_read_in_order = false, aggregation_in_order_max_block_bytes = 35124699, read_in_order_two_level_merge_threshold = 56, allow_introspection_functions = true, database_atomic_wait_for_drop_and_detach_synchronously = true, optimize_if_transform_strings_to_enum = true, optimize_substitute_columns = true, optimize_append_index = true, distributed_ddl_output_mode = 'none', distributed_ddl_entry_format_version = 6, local_filesystem_read_method = 'mmap', remote_filesystem_read_method = 'read', local_filesystem_read_prefetch = true, remote_filesystem_read_prefetch = false, merge_tree_min_bytes_per_task_for_remote_reading = 16777216, merge_tree_compact_parts_min_granules_to_multibuffer_read = 18, async_insert_busy_timeout_max_ms = 5000, enable_filesystem_cache = true, enable_filesystem_cache_on_write_operations = true, throw_on_error_from_cache_on_write_operations = true, filesystem_cache_segments_batch_size = 50, load_marks_asynchronously = true, allow_prefetched_read_pool_for_remote_filesystem = false, allow_prefetched_read_pool_for_local_filesystem = false, filesystem_prefetch_step_bytes = 104857600, filesystem_prefetch_step_marks = 50, filesystem_prefetch_max_memory_usage = 134217728, filesystem_prefetches_limit = 10, allow_deprecated_database_ordinary = true, insert_keeper_max_retries = 100, insert_keeper_retry_initial_backoff_ms = 1, insert_keeper_retry_max_backoff_ms = 10, insert_keeper_fault_injection_probability = 0.009999999776482582, ignore_drop_queries_probability = 0.5, session_timezone = 'America/Mazatlan'
2024.08.16 04:10:40.724340 [ 3382 ] {} <Fatal> Application: Child process was terminated by signal 6.

Metadata

Metadata

Assignees

Labels

fuzzProblem found by one of the fuzzerstestingSpecial issue with list of bugs found by CI

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions