Skip to content

broken_parts == broken_parts_to_enqueue_fetches_on_loading assertion in ReplicatedMergeTreeQueue::createLogEntriesToFetchBrokenParts #101070

@azat

Description

@azat

CI: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=101035&sha=30ab84bf715ae87971ed44529099c8d5643a50f2&name_0=PR&name_1=Stress%20test%20%28amd_debug%29

https://pastila.nl/?0000a0b9/2a125db53d6b59094990e5d881187901#bhCL1xMTrYr4xdw2kzMChQ==GCM

/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.398321 [ 7018 ] {BgSchPool::701f35d0-8a7b-4a12-8b9d-16e8ed854e9a} <Fatal> : Logical error: 'false'.
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.416479 [ 7018 ] {BgSchPool::701f35d0-8a7b-4a12-8b9d-16e8ed854e9a} <Fatal> : Stack trace (when copying this message, always include the lines below):
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.432406 [ 6994 ] {} <Fatal> BaseDaemon: ########## Short fault info ############
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.433652 [ 6994 ] {} <Fatal> BaseDaemon: (version 26.4.1.360, build id: 0AD53762F81F6EA460092E13EDFEB9DEDBF079E3, git hash: 26936c0eb741845822d349a7ae74b7eb5e9bb735, architecture: x86_64) (from thread 7018) Received signal 6 (internal)
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.433980 [ 6994 ] {} <Fatal> BaseDaemon: Signal description: Aborted
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.434687 [ 6994 ] {} <Fatal> BaseDaemon: Sent by tkill.
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.450307 [ 6994 ] {} <Fatal> BaseDaemon: Stack trace: 0x00007fe22b9d89fd 0x00007fe22b984476 0x00007fe22b96a7f3 0x0000560ac55c211d 0x0000560ac55c2e47 0x0000560acd7079da 0x0000560acde8f078 0x0000560acdec75c7 0x0000560acdec6119 0x0000560acdec532e 0x0000560ac95d7557 0x0000560ac95dca03 0x0000560ac95ded0a 0x0000560ac574d3c3 0x0000560ac575490e 0x00007fe22b9d6ac3 0x00007fe22ba688d0
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.453789 [ 6994 ] {} <Fatal> BaseDaemon: ########################################
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.454590 [ 6994 ] {} <Fatal> BaseDaemon: (version 26.4.1.360, build id: 0AD53762F81F6EA460092E13EDFEB9DEDBF079E3, git hash: 26936c0eb741845822d349a7ae74b7eb5e9bb735) (from thread 7018) (query_id: BgSchPool::701f35d0-8a7b-4a12-8b9d-16e8ed854e9a) (query: ) Received signal Aborted (6)
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.454939 [ 6994 ] {} <Fatal> BaseDaemon: Sent by tkill.
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.456832 [ 6994 ] {} <Fatal> BaseDaemon: Stack trace: 0x00007fe22b9d89fd 0x00007fe22b984476 0x00007fe22b96a7f3 0x0000560ac55c211d 0x0000560ac55c2e47 0x0000560acd7079da 0x0000560acde8f078 0x0000560acdec75c7 0x0000560acdec6119 0x0000560acdec532e 0x0000560ac95d7557 0x0000560ac95dca03 0x0000560ac95ded0a 0x0000560ac574d3c3 0x0000560ac575490e 0x00007fe22b9d6ac3 0x00007fe22ba688d0
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.459100 [ 6994 ] {} <Fatal> BaseDaemon: 3. pthread_kill@@GLIBC_2.34 @ 0x00000000000969fd
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.464465 [ 6994 ] {} <Fatal> BaseDaemon: 4. raise @ 0x0000000000042476
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.465377 [ 6994 ] {} <Fatal> BaseDaemon: 5. __ieee754_lgamma_r @ 0x00000000000287f3
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.540380 [ 6994 ] {} <Fatal> BaseDaemon: 6. /ClickHouse/src/Common/Exception.cpp:60:5: DB::abortOnFailedAssertion(String const&, std::basic_string_view<char, std::char_traits<char>>, void* const*, unsigned long, unsigned long) @ 0x000000001370c11d
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.554835 [ 6994 ] {} <Fatal> BaseDaemon: 7. /ClickHouse/src/Common/Exception.cpp:66:5: ? @ 0x000000001370ce47
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:19.957369 [ 6994 ] {} <Fatal> BaseDaemon: 8. /ClickHouse/src/Storages/StorageReplicatedMergeTree.cpp:1839:13: DB::StorageReplicatedMergeTree::paranoidCheckForCoveredPartsInZooKeeperOnStart(std::vector<String, std::allocator<String>> const&, std::vector<String, std::allocator<String>> const&) const @ 0x000000001b8519da
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:20.181773 [ 6994 ] {} <Fatal> BaseDaemon: 9. /ClickHouse/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:247:13: DB::ReplicatedMergeTreeQueue::createLogEntriesToFetchBrokenParts() @ 0x000000001bfd9078
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:20.330429 [ 6994 ] {} <Fatal> BaseDaemon: 10. /ClickHouse/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:211:27: DB::ReplicatedMergeTreeRestartingThread::tryStartup() @ 0x000000001c0115c7
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:20.405394 [ 6994 ] {} <Fatal> BaseDaemon: 11. /ClickHouse/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:168:10: DB::ReplicatedMergeTreeRestartingThread::runImpl() @ 0x000000001c010119
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:20.490337 [ 6994 ] {} <Fatal> BaseDaemon: 12. /ClickHouse/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:94:34: DB::ReplicatedMergeTreeRestartingThread::run() @ 0x000000001c00f32e
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:20.528329 [ 6994 ] {} <Fatal> BaseDaemon: 13.0. inlined from /ClickHouse/contrib/llvm-project/libcxx/include/__functional/function.h:508: ?
/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/server_log/clickhouse-server.err.log:2026.03.28 22:46:20.529579 [ 6994 ] {} <Fatal> BaseDaemon: 13.1. inlined from /ClickHouse/contrib/llvm-project/libcxx/include/__functional/function.h:772: ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    comp-replicationReplicatedMergeTree* + replication log/state transitions, eventual consistency mechanics.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions