Skip to content

Logical error: 'Digest does not match'. #77233

@qoega

Description

@qoega

Describe the bug

https://s3.amazonaws.com/clickhouse-test-reports/json.html?REF=master&sha=b2da3ca6c13bc66e4c60bb2e7e46986b98df6df9&name_0=MasterCI&name_1=Stress+test+%28debug%29

How to reproduce

02156_async_insert_query_log.sh

grep -a caababe7-1f4a-4334-901e-c9b69d9237e6 clickhouse-server.stress.log  |grep 'AST differs'
2025.03.06 13:52:19.064357 [ 6688 ] {caababe7-1f4a-4334-901e-c9b69d9237e6} <Error> DatabaseReplicated (test_5): AST differs for table mv, local CREATE MATERIALIZED VIEW test_5.mv UUID 'f9056b79-6595-4824-8722-a555620c107a' TO INNER UUID 'e311a3de-2327-4797-bc32-61eee8218fc7' (`timestamp` DateTime, `c12` Nullable(String)) ENGINE = MergeTree ORDER BY timestamp SETTINGS index_granularity = 8192 AS SELECT Timestamp AS timestamp, concat(c1, c2) AS c12 FROM test_5.src, in zookeeper ATTACH MATERIALIZED VIEW test_5.mv UUID 'f9056b79-6595-4824-8722-a555620c107a' TO INNER UUID 'e311a3de-2327-4797-bc32-61eee8218fc7' (`timestamp` DateTime, `c12` Nullable(String)) ENGINE = MergeTree ORDER BY timestamp SETTINGS index_granularity = 8192 AS SELECT Timestamp AS timestamp, concat(c1, c2) AS c12 FROM test_5.src
2025.03.06 13:52:19.083039 [ 6688 ] {caababe7-1f4a-4334-901e-c9b69d9237e6} <Error> DatabaseReplicated (test_5): AST differs for table test_mv_1, local CREATE MATERIALIZED VIEW test_5.test_mv_1 UUID '8f880b48-e7d0-4628-a84e-9854cd72294b' TO INNER UUID 'ab3194ab-a330-4d88-96f1-13d84b3c85e2' (`s` String) ENGINE = MergeTree ORDER BY s SETTINGS index_granularity = 8192 DEFINER = user02884_1_test_5_13479 SQL SECURITY DEFINER AS SELECT * FROM test_5.test_table, in zookeeper ATTACH MATERIALIZED VIEW test_5.test_mv_1 UUID '8f880b48-e7d0-4628-a84e-9854cd72294b' TO INNER UUID 'ab3194ab-a330-4d88-96f1-13d84b3c85e2' (`s` String) ENGINE = MergeTree ORDER BY s SETTINGS index_granularity = 8192 DEFINER = user02884_1_test_5_13479 SQL SECURITY DEFINER AS SELECT * FROM test_5.test_table
2025.03.06 13:52:19.087034 [ 6688 ] {caababe7-1f4a-4334-901e-c9b69d9237e6} <Error> DatabaseReplicated (test_5): AST differs for table test_mv_3, local CREATE MATERIALIZED VIEW test_5.test_mv_3 UUID '2a4c7d63-5f03-47de-bbe9-7737facd7f5d' TO INNER UUID '6fde362b-e807-47fe-9e37-9f32545f3abc' (`s` String) ENGINE = MergeTree ORDER BY s SETTINGS index_granularity = 8192 SQL SECURITY NONE AS SELECT * FROM test_5.test_table, in zookeeper ATTACH MATERIALIZED VIEW test_5.test_mv_3 UUID '2a4c7d63-5f03-47de-bbe9-7737facd7f5d' TO INNER UUID '6fde362b-e807-47fe-9e37-9f32545f3abc' (`s` String) ENGINE = MergeTree ORDER BY s SETTINGS index_granularity = 8192 SQL SECURITY NONE AS SELECT * FROM test_5.test_table
2025.03.06 13:52:19.091309 [ 6688 ] {caababe7-1f4a-4334-901e-c9b69d9237e6} <Error> DatabaseReplicated (test_5): AST differs for table test_mv_5, local CREATE MATERIALIZED VIEW test_5.test_mv_5 UUID '746c48e7-ba8e-4f47-a18d-69699e74d580' TO INNER UUID 'c7ecba3d-ee35-45ad-bb75-daebb3ee8ece' (`s` String) ENGINE = MergeTree ORDER BY s SETTINGS index_granularity = 8192 SQL SECURITY NONE AS SELECT * FROM test_5.test_table, in zookeeper ATTACH MATERIALIZED VIEW test_5.test_mv_5 UUID '746c48e7-ba8e-4f47-a18d-69699e74d580' TO INNER UUID 'c7ecba3d-ee35-45ad-bb75-daebb3ee8ece' (`s` String) ENGINE = MergeTree ORDER BY s SETTINGS index_granularity = 8192 SQL SECURITY NONE AS SELECT * FROM test_5.test_table

Error message and/or stacktrace

[ 24522 ] {} BaseDaemon: ########## Short fault info ############
[ 24522 ] {} BaseDaemon: (version 25.3.1.1250 (official build), build id: 20ECC468C38522AA98EBD5EBDC25384AC0FB6C6C, git hash: b2da3ca6c13bc66e4c60bb2e7e46986b98df6df9, architecture: x86_64) (from thread 6688) Received signal 6
[ 24522 ] {} BaseDaemon: Signal description: Aborted
[ 24522 ] {} BaseDaemon:
[ 24522 ] {} BaseDaemon: Stack trace: 0x0000559e1d78b1c8 0x0000559e1da4f625 0x00007f3cca83c520 0x00007f3cca8909fd 0x00007f3cca83c476 0x00007f3cca8227f3 0x0000559e1d703adc 0x0000559e1d70424a 0x0000559e17db68ac 0x0000559e17dc012b 0x0000559e2186e926 0x0000559e218808eb 0x0000559e223bee78 0x0000559e223bcfb4 0x0000559e223bc707 0x0000559e223bc365 0x0000559e227316e9 0x0000559e2273539b 0x0000559e21f8925e 0x0000559e21f876be 0x0000559e2189366b 0x0000559e21870c7c 0x0000559e223be6d7 0x0000559e223bcfb4 0x0000559e223bc707 0x0000559e223bc365 0x0000559e227316e9 0x0000559e2272d18b 0x0000559e24389121 0x0000559e243a1299 0x0000559e28696447 0x0000559e2869691e 0x0000559e28643f92 0x0000559e28641bcf 0x00007f3cca88eac3 0x00007f3cca920850
[ 24522 ] {} BaseDaemon: ########################################
[ 24522 ] {} BaseDaemon: (version 25.3.1.1250 (official build), build id: 20ECC468C38522AA98EBD5EBDC25384AC0FB6C6C, git hash: b2da3ca6c13bc66e4c60bb2e7e46986b98df6df9) (from thread 6688) (query_id: caababe7-1f4a-4334-901e-c9b69d9237e6) (query: /* ddl_entry=query-0000000076 */ DROP TABLE test_5.async_inserts_2156 SYNC) Received signal Aborted (6)
[ 24522 ] {} BaseDaemon:
[ 24522 ] {} BaseDaemon: Stack trace: 0x0000559e1d78b1c8 0x0000559e1da4f625 0x00007f3cca83c520 0x00007f3cca8909fd 0x00007f3cca83c476 0x00007f3cca8227f3 0x0000559e1d703adc 0x0000559e1d70424a 0x0000559e17db68ac 0x0000559e17dc012b 0x0000559e2186e926 0x0000559e218808eb 0x0000559e223bee78 0x0000559e223bcfb4 0x0000559e223bc707 0x0000559e223bc365 0x0000559e227316e9 0x0000559e2273539b 0x0000559e21f8925e 0x0000559e21f876be 0x0000559e2189366b 0x0000559e21870c7c 0x0000559e223be6d7 0x0000559e223bcfb4 0x0000559e223bc707 0x0000559e223bc365 0x0000559e227316e9 0x0000559e2272d18b 0x0000559e24389121 0x0000559e243a1299 0x0000559e28696447 0x0000559e2869691e 0x0000559e28643f92 0x0000559e28641bcf 0x00007f3cca88eac3 0x00007f3cca920850
[ 24522 ] {} BaseDaemon: 0.0. inlined from /build/src/Common/StackTrace.cpp:381: StackTrace::tryCapture()
[ 24522 ] {} BaseDaemon: 0. /build/src/Common/StackTrace.cpp:350: StackTrace::StackTrace(ucontext_t const&) @ 0x000000000f84c1c8
[ 24522 ] {} BaseDaemon: 1. /build/src/Common/SignalHandlers.cpp:106: signalHandler(int, siginfo_t*, void*) @ 0x000000000fb10625
[ 24522 ] {} BaseDaemon: 2. ? @ 0x00007f3cca83c520
[ 24522 ] {} BaseDaemon: 3. ? @ 0x00007f3cca8909fd
[ 24522 ] {} BaseDaemon: 4. ? @ 0x00007f3cca83c476
[ 24522 ] {} BaseDaemon: 5. ? @ 0x00007f3cca8227f3
[ 24522 ] {} BaseDaemon: 6. /build/src/Common/Exception.cpp:48: DB::abortOnFailedAssertion(String const&, void* const*, unsigned long, unsigned long) @ 0x000000000f7c4adc
[ 24522 ] {} BaseDaemon: 7.0. inlined from /build/src/Common/Exception.cpp:70: DB::handle_error_code(String const&, int, bool, std::vector> const&)
[ 24522 ] {} BaseDaemon: 7. /build/src/Common/Exception.cpp:114: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000f7c524a
[ 24522 ] {} BaseDaemon: 8.0. inlined from /build/src/Common/Exception.h:112: Exception
[ 24522 ] {} BaseDaemon: 8. /build/src/Common/Exception.h:56: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000009e778ac
[ 24522 ] {} BaseDaemon: 9. /build/src/Common/Exception.h:130: DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000009e8112b
[ 24522 ] {} BaseDaemon: 10. /build/src/Databases/DatabaseReplicated.cpp:970: DB::DatabaseReplicated::assertDigestWithProbability(std::shared_ptr const&) const @ 0x000000001392f926
[ 24522 ] {} BaseDaemon: 11. /build/src/Databases/DatabaseReplicated.cpp:1724: DB::DatabaseReplicated::dropTable(std::shared_ptr, String const&, bool) @ 0x00000000139418eb
[ 24522 ] {} BaseDaemon: 12. /build/src/Interpreters/InterpreterDropQuery.cpp:310: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x000000001447fe78
[ 24522 ] {} BaseDaemon: 13. /build/src/Interpreters/InterpreterDropQuery.cpp:119: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x000000001447dfb4
[ 24522 ] {} BaseDaemon: 14. /build/src/Interpreters/InterpreterDropQuery.cpp:92: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x000000001447d707
[ 24522 ] {} BaseDaemon: 15. /build/src/Interpreters/InterpreterDropQuery.cpp:73: DB::InterpreterDropQuery::execute() @ 0x000000001447d365
[ 24522 ] {} BaseDaemon: 16. /build/src/Interpreters/executeQuery.cpp:1458: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x00000000147f26e9
[ 24522 ] {} BaseDaemon: 17. /build/src/Interpreters/executeQuery.cpp:1762: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x00000000147f639b
[ 24522 ] {} BaseDaemon: 18. /build/src/Interpreters/DDLWorker.cpp:510: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&, bool) @ 0x000000001404a25e
[ 24522 ] {} BaseDaemon: 19. /build/src/Interpreters/DDLWorker.cpp:675: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&, bool) @ 0x00000000140486be
[ 24522 ] {} BaseDaemon: 20. /build/src/Databases/DatabaseReplicatedWorker.cpp:372: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr, bool) @ 0x000000001395466b
[ 24522 ] {} BaseDaemon: 21. /build/src/Databases/DatabaseReplicated.cpp:1136: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000013931c7c
[ 24522 ] {} BaseDaemon: 22. /build/src/Interpreters/InterpreterDropQuery.cpp:227: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x000000001447f6d7
[ 24522 ] {} BaseDaemon: 23. /build/src/Interpreters/InterpreterDropQuery.cpp:119: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x000000001447dfb4
[ 24522 ] {} BaseDaemon: 24. /build/src/Interpreters/InterpreterDropQuery.cpp:92: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x000000001447d707
[ 24522 ] {} BaseDaemon: 25. /build/src/Interpreters/InterpreterDropQuery.cpp:73: DB::InterpreterDropQuery::execute() @ 0x000000001447d365
[ 24522 ] {} BaseDaemon: 26. /build/src/Interpreters/executeQuery.cpp:1458: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x00000000147f26e9
[ 24522 ] {} BaseDaemon: 27. /build/src/Interpreters/executeQuery.cpp:1625: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000147ee18b
[ 24522 ] {} BaseDaemon: 28. /build/src/Server/TCPHandler.cpp:664: DB::TCPHandler::runImpl() @ 0x000000001644a121
[ 24522 ] {} BaseDaemon: 29. /build/src/Server/TCPHandler.cpp:2629: DB::TCPHandler::run() @ 0x0000000016462299
[ 24522 ] {} BaseDaemon: 30. /build/base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x000000001a757447
[ 24522 ] {} BaseDaemon: 31. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000001a75791e
[ 24522 ] {} BaseDaemon: 32. /build/base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x000000001a704f92
[ 24522 ] {} BaseDaemon: 33. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001a702bcf
[ 24522 ] {} BaseDaemon: 34. ? @ 0x00007f3cca88eac3
[ 24522 ] {} BaseDaemon: 35. ? @ 0x00007f3cca920850
[ 24522 ] {} BaseDaemon: Integrity check of the executable successfully passed (checksum: E2760E4AFAB740F65363F13915CF48E9)
[ 24522 ] {} BaseDaemon: Report this error to https://github.com/ClickHouse/ClickHouse/issues
[ 24522 ] {} BaseDaemon: Changed settings: min_compress_block_size = 2196138, max_compress_block_size = 746707, max_block_size = 82285, min_external_table_block_size_bytes = 0, max_joined_block_size_rows = 59323, max_insert_threads = 1, max_threads = 1, max_read_buffer_size = 582704, connect_timeout_with_failover_ms = 2000, connect_timeout_with_failover_secure_ms = 3000, idle_connection_timeout = 36000, s3_max_get_rps = 1000000, s3_max_get_burst = 2000000, s3_max_put_rps = 1000000, s3_max_put_burst = 2000000, s3_check_objects_after_upload = true, use_uncompressed_cache = true, max_remote_read_network_bandwidth = 1000000000000, max_remote_write_network_bandwidth = 1000000000000, max_local_read_bandwidth = 1000000000000, max_local_write_bandwidth = 1000000000000, stream_like_engine_allow_direct_select = true, replication_wait_for_inactive_replica_timeout = 30, compile_aggregate_expressions = false, min_count_to_compile_aggregate_expression = 0, group_by_two_level_threshold = 1000000, group_by_two_level_threshold_bytes = 2479725, distributed_aggregation_memory_efficient = false, enable_memory_bound_merging_of_aggregation_results = false, allow_nonconst_timezone_arguments = true, group_by_use_nulls = true, input_format_parallel_parsing = false, min_chunk_bytes_for_parallel_parsing = 2692548, output_format_parallel_formatting = false, merge_tree_coarse_index_granularity = 31, min_bytes_to_use_direct_io = 1371860538, min_bytes_to_use_mmap_io = 4583558604, log_queries = true, insert_quorum_timeout = 60000, table_function_remote_max_addresses = 200, merge_tree_read_split_ranges_into_intersecting_and_non_intersecting_injection_probability = 0.7900000214576721, http_response_buffer_size = 8291653, fsync_metadata = false, query_plan_join_swap_table = false, http_send_timeout = 60., http_receive_timeout = 60., use_index_for_in_with_subqueries_max_values = 1000000000, enable_zstd_qat_codec = true, enable_deflate_qpl_codec = true, opentelemetry_start_trace_probability = 0.10000000149011612, max_rows_to_read = 20000000, max_bytes_to_read = 1000000000000, max_bytes_to_read_leaf = 1000000000000, max_rows_to_group_by = 10000000000, max_bytes_before_external_group_by = 10737418240, max_bytes_ratio_before_external_group_by = 0., max_rows_to_sort = 10000000000, max_bytes_to_sort = 10000000000, prefer_external_sort_block_bytes = 100000000, max_bytes_before_external_sort = 10737418240, max_bytes_ratio_before_external_sort = 0., max_bytes_before_remerge_sort = 998009756, max_result_rows = 1000000000, max_result_bytes = 1000000000, max_execution_time = 60., max_execution_time_leaf = 600., max_execution_speed = 100000000000, max_execution_speed_bytes = 10000000000000, timeout_before_checking_execution_speed = 300., max_estimated_execution_time = 600., max_columns_to_read = 20000, max_temporary_columns = 20000, max_temporary_non_const_columns = 20000, max_rows_in_set = 10000000000, max_bytes_in_set = 10000000000, max_rows_in_join = 10000000000, max_bytes_in_join =
[ 4001 ] {} Application: Child process was terminated by signal 6.

Metadata

Metadata

Labels

fuzzProblem found by one of the fuzzers

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions