-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Stateless tests: "Internal query (CREATE/DROP DATABASE) failed" (DatabaseReplicated) #82676
Copy link
Copy link
Closed
Labels
flaky testflaky test found by CIflaky test found by CIred CIVERY flaky testVERY flaky testtestingSpecial issue with list of bugs found by CISpecial issue with list of bugs found by CI
Description
Symptoms: A group of concurrent tests fail because they cannot create a new database to run the check.
Example:
FAIL 02255_broken_parts_chain_on_start 123.79
2025-06-26 06:38:29 Reason: Internal query (CREATE/DROP DATABASE) failed:
2025-06-26 06:38:29 HTTPError
2025-06-26 06:38:29 Code: 408. Code: 159. DB::Exception: Distributed DDL task /clickhouse/task_queue/ddl/query-0000002203 is not finished on 1 of 3 hosts (1 of them are currently executing the task, 0 are inactive). They are going to execute the query in background. Was waiting for 120.646216797 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED) (version 25.7.1.1)
2025-06-26 06:38:29
2025-06-26 06:38:29 File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/clickhouse-test", line 2179, in run
2025-06-26 06:38:29 self._cleanup(result.status == TestStatus.OK)
2025-06-26 06:38:29
2025-06-26 06:38:29 File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/clickhouse-test", line 2242, in _cleanup
2025-06-26 06:38:29 self._cleanup_database(args, timeout)
2025-06-26 06:38:29
2025-06-26 06:38:29 File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/clickhouse-test", line 2277, in _cleanup_database
2025-06-26 06:38:29 clickhouse_execute(
2025-06-26 06:38:29
2025-06-26 06:38:29 File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/clickhouse-test", line 504, in clickhouse_execute
2025-06-26 06:38:29 return clickhouse_execute_http(
2025-06-26 06:38:29
2025-06-26 06:38:29 File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/clickhouse-test", line 490, in clickhouse_execute_http
2025-06-26 06:38:29 raise HTTPError(data.decode(), res.status)
2025-06-26 06:38:29
2025-06-26 06:38:29 Settings used in the test: --max_insert_threads 2 --group_by_two_level_threshold 448377 --group_by_two_level_threshold_bytes 1 --distributed_aggregation_memory_efficient 0 --fsync_metadata 1 --output_format_parallel_formatting 1 --input_format_parallel_parsing 1 --min_chunk_bytes_for_parallel_parsing 11621573 --max_read_buffer_size 515726 --prefer_localhost_replica 1 --max_block_size 26489 --max_joined_block_size_rows 60002 --max_threads 1 --optimize_append_index 0 --use_hedged_requests 0 --optimize_if_chain_to_multiif 0 --optimize_if_transform_strings_to_enum 0 --optimize_read_in_order 1 --optimize_or_like_chain 0 --optimize_substitute_columns 1 --enable_multiple_prewhere_read_steps 0 --read_in_order_two_level_merge_threshold 52 --optimize_aggregation_in_order 1 --aggregation_in_order_max_block_bytes 29390299 --use_uncompressed_cache 0 --min_bytes_to_use_direct_io 884455121 --min_bytes_to_use_mmap_io 10737418240 --local_filesystem_read_method read --remote_filesystem_read_method threadpool --local_filesystem_read_prefetch 0 --filesystem_cache_segments_batch_size 0 --read_from_filesystem_cache_if_exists_otherwise_bypass_cache 1 --throw_on_error_from_cache_on_write_operations 0 --remote_filesystem_read_prefetch 1 --allow_prefetched_read_pool_for_remote_filesystem 0 --filesystem_prefetch_max_memory_usage 128Mi --filesystem_prefetches_limit 0 --filesystem_prefetch_min_bytes_for_single_read_task 16Mi --filesystem_prefetch_step_marks 50 --filesystem_prefetch_step_bytes 100Mi --compile_expressions 1 --compile_aggregate_expressions 0 --compile_sort_description 1 --merge_tree_coarse_index_granularity 14 --optimize_distinct_in_order 0 --max_bytes_before_remerge_sort 702331688 --min_compress_block_size 1814454 --max_compress_block_size 2102903 --merge_tree_compact_parts_min_granules_to_multibuffer_read 86 --optimize_sorting_by_input_stream_properties 0 --http_response_buffer_size 9802242 --http_wait_end_of_query False --enable_memory_bound_merging_of_aggregation_results 0 --min_count_to_compile_expression 3 --min_count_to_compile_aggregate_expression 0 --min_count_to_compile_sort_description 0 --session_timezone America/Mazatlan --use_page_cache_for_disks_without_file_cache False --page_cache_inject_eviction True --merge_tree_read_split_ranges_into_intersecting_and_non_intersecting_injection_probability 0.8 --prefer_external_sort_block_bytes 1 --cross_join_min_rows_to_compress 0 --cross_join_min_bytes_to_compress 100000000 --min_external_table_block_size_bytes 1 --max_parsing_threads 1 --optimize_functions_to_subcolumns 1 --parallel_replicas_local_plan 1 --query_plan_join_swap_table true --enable_vertical_final 1 --optimize_extract_common_expressions 0 --use_query_condition_cache 0 --secondary_indices_enable_bulk_filtering 0 --use_skip_indexes_if_final 0 --max_bytes_before_external_sort 0 --max_bytes_before_external_group_by 0 --max_bytes_ratio_before_external_sort 0.0 --max_bytes_ratio_before_external_group_by 0.0 --use_skip_indexes_if_final_exact_mode 0
2025-06-26 06:38:29
2025-06-26 06:38:29 MergeTree settings used in test: --ratio_of_defaults_for_sparse_serialization 1.0 --prefer_fetch_merged_part_size_threshold 10737418240 --vertical_merge_algorithm_min_rows_to_activate 1 --vertical_merge_algorithm_min_columns_to_activate 47 --allow_vertical_merges_from_compact_to_wide_parts 0 --min_merge_bytes_to_use_direct_io 10737418240 --index_granularity_bytes 2557545 --merge_max_block_size 18922 --index_granularity 51802 --min_bytes_for_wide_part 892093879 --marks_compress_block_size 59952 --primary_key_compress_block_size 29152 --replace_long_file_name_to_hash 1 --max_file_name_length 69 --min_bytes_for_full_part_storage 468102463 --compact_parts_max_bytes_to_buffer 433523504 --compact_parts_max_granules_to_buffer 246 --compact_parts_merge_max_bytes_to_prefetch_part 12371589 --cache_populated_by_fetch 0 --concurrent_part_removal_threshold 17 --old_parts_lifetime 480 --prewarm_mark_cache 0 --use_const_adaptive_granularity 0 --enable_index_granularity_compression 0 --enable_block_number_column 1 --enable_block_offset_column 1 --use_primary_key_cache 1 --prewarm_primary_key_cache 1
2025-06-26 06:38:29
2025-06-26 06:38:29 Database: test_vj5kb6b6
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
flaky testflaky test found by CIflaky test found by CIred CIVERY flaky testVERY flaky testtestingSpecial issue with list of bugs found by CISpecial issue with list of bugs found by CI