Skip to content

Fix createTableSharedID again#43458

Merged
tavplubix merged 3 commits intomasterfrom
tavplubix-patch-5
Nov 24, 2022
Merged

Fix createTableSharedID again#43458
tavplubix merged 3 commits intomasterfrom
tavplubix-patch-5

Conversation

@tavplubix
Copy link
Copy Markdown
Member

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

https://s3.amazonaws.com/clickhouse-test-reports/42265/66f101e451754e76a1ecfd4d518a1f55b6086eea/stress_test__ubsan_.html

2022.11.19 20:19:48.998420 [ 1569 ] {} <Error> DatabaseCatalog: Cannot drop table test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2). Will retry later.: Code: 999. Coordination::Exception: Session expired, path: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):
2022.11.19 20:19:53.998642 [ 1644 ] {} <Information> DatabaseCatalog: Have 1 tables in drop queue (0 of them are in use), will try drop test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2)
2022.11.19 20:19:53.999495 [ 1644 ] {} <Information> test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2): Removing replica /clickhouse/tables/01076_parallel_alter_replicated_zookeeper_test_65ruc19z/concurrent_mutate_mt/replicas/1, marking it as lost
2022.11.19 20:19:54.000672 [ 1644 ] {} <Information> test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2): Removing replica /clickhouse/tables/01076_parallel_alter_replicated_zookeeper_test_65ruc19z/concurrent_mutate_mt/replicas/1 does not exist
2022.11.19 20:19:54.000699 [ 1644 ] {} <Trace> test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2): dropAllData: waiting for locks.
2022.11.19 20:19:54.000749 [ 1644 ] {} <Trace> test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2): dropAllData: removing data parts (count 14) from filesystem.
2022.11.19 20:19:54.000775 [ 1644 ] {} <Debug> test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2): Removing 14 parts from filesystem: all_0_0_0, all_1_1_0, all_3_3_0, all_4_4_0, all_5_5_0, all_6_6_0, all_7_7_0, all_8_8_0, all_9_9_0, all_10_10_0, all_11_11_0, all_12_12_0, all_16_16_0, all_18_18_0
2022.11.19 20:19:54.000884 [ 1644 ] {} <Trace> test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2): Part all_0_0_0 local references is zero, will check blobs can be removed in zookeeper
2022.11.19 20:19:54.001913 [ 1644 ] {} <Debug> test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2): Creating shared ID for table test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2)
2022.11.19 20:19:54.002451 [ 1644 ] {} <Warning> test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2): dropAllData: got exception removing parts from disk, removing successfully removed parts from memory.
2022.11.19 20:19:54.002690 [ 1644 ] {} <Error> DatabaseCatalog: Cannot drop table test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2). Will retry later.: Code: 999. Coordination::Exception: Session expired, path: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

2022.11.19 20:19:18.969388 [ 1685 ] {} <Error> DatabaseCatalog: Cannot drop table test_65ruc19z.concurrent_mutate_mt_1 (88eeef91-2f33-4a65-a361-2b79af91bde2). Will retry later.: Code: 999. Coordination::Exception: Session expired, path: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/../contrib/libcxx/include/exception:134: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, int) @ 0x2f3efa36 in /usr/bin/clickhouse
1. ./build_docker/../src/Common/Exception.cpp:69: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, int, bool) @ 0x21039bcf in /usr/bin/clickhouse
2. ./build_docker/../src/Common/ZooKeeper/IKeeper.cpp:26: Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Coordination::Error, int) @ 0x2ec072c9 in /usr/bin/clickhouse
3. ./build_docker/../src/Common/ZooKeeper/IKeeper.cpp:0: Coordination::Exception::Exception(Coordination::Error, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) @ 0x2ec07858 in /usr/bin/clickhouse
4. ./build_docker/../src/Common/ZooKeeper/ZooKeeperImpl.cpp:1071: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x2ec9a206 in /usr/bin/clickhouse
5. ./build_docker/../src/Common/ZooKeeper/ZooKeeperImpl.cpp:1208: Coordination::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::function<void (Coordination::GetResponse const&)>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0x2ec9a9dd in /usr/bin/clickhouse
6. ./build_docker/../src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::asyncTryGetNoThrow(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0x2ec10b27 in /usr/bin/clickhouse
7. ./build_docker/../src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::getImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, Coordination::Stat*, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0x2ec10519 in /usr/bin/clickhouse
8. ./build_docker/../src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetWatch(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, Coordination::Stat*, std::__1::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x2ec10f95 in /usr/bin/clickhouse
9. ./build_docker/../src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x2ec10d84 in /usr/bin/clickhouse
10. ./build_docker/../src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::createTableSharedID() const @ 0x2d8b7756 in /usr/bin/clickhouse
11. ./build_docker/../src/Storages/StorageReplicatedMergeTree.cpp:7634: DB::StorageReplicatedMergeTree::getTableSharedID() const @ 0x2d97af9a in /usr/bin/clickhouse
12. ./build_docker/../src/Storages/StorageReplicatedMergeTree.cpp:7850: DB::StorageReplicatedMergeTree::unlockSharedData(DB::IMergeTreeDataPart const&, std::__1::shared_ptr<DB::ZooKeeperWithFaultInjection> const&) const @ 0x2d980a1d in /usr/bin/clickhouse
13. ./build_docker/../src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::unlockSharedData(DB::IMergeTreeDataPart const&) const @ 0x2d8ffd79 in /usr/bin/clickhouse
14. ./build_docker/../src/Storages/MergeTree/IMergeTreeDataPart.cpp:0: DB::IMergeTreeDataPart::canRemovePart() const @ 0x2dca66e2 in /usr/bin/clickhouse
15. ./build_docker/../src/Storages/MergeTree/IMergeTreeDataPart.cpp:1641: DB::CanRemoveDescription std::__1::__function::__policy_invoker<DB::CanRemoveDescription ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::IMergeTreeDataPart::remove()::$_7, DB::CanRemoveDescription ()>>(std::__1::__function::__policy_storage const*) @ 0x2dcae406 in /usr/bin/clickhouse
16. ./build_docker/../contrib/libcxx/include/__functional/function.h:0: DB::DataPartStorageOnDisk::remove(std::__1::function<DB::CanRemoveDescription ()>&&, DB::MergeTreeDataPartChecksums const&, std::__1::list<DB::IDataPartStorage::ProjectionChecksums, std::__1::allocator<DB::IDataPartStorage::ProjectionChecksums>>, bool, DB::MergeTreeDataPartState, Poco::Logger*) @ 0x2dc1398d in /usr/bin/clickhouse
17. ./build_docker/../src/Storages/MergeTree/IMergeTreeDataPart.cpp:0: DB::IMergeTreeDataPart::remove() @ 0x2dc8e430 in /usr/bin/clickhouse
18. ./build_docker/../src/Storages/MergeTree/MergeTreeData.cpp:1976: DB::MergeTreeData::clearPartsFromFilesystemImpl(std::__1::vector<std::__1::shared_ptr<DB::IMergeTreeDataPart const>, std::__1::allocator<std::__1::shared_ptr<DB::IMergeTreeDataPart const>>> const&, std::__1::unordered_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>*) @ 0x2dd5d576 in /usr/bin/clickhouse
19. ./build_docker/../src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::clearPartsFromFilesystem(std::__1::vector<std::__1::shared_ptr<DB::IMergeTreeDataPart const>, std::__1::allocator<std::__1::shared_ptr<DB::IMergeTreeDataPart const>>> const&, bool, std::__1::unordered_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>*) @ 0x2dd5cd3b in /usr/bin/clickhouse
20. ./build_docker/../src/Storages/MergeTree/MergeTreeData.cpp:2206: DB::MergeTreeData::dropAllData() @ 0x2dd65e29 in /usr/bin/clickhouse
21. ./build_docker/../src/Storages/StorageReplicatedMergeTree.cpp:891: DB::StorageReplicatedMergeTree::drop() @ 0x2d8c0f86 in /usr/bin/clickhouse
22. ./build_docker/../src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableFinally(DB::DatabaseCatalog::TableMarkedAsDropped const&) @ 0x2c22cf17 in /usr/bin/clickhouse
23. ./build_docker/../src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableDataTask() @ 0x2c22c4a9 in /usr/bin/clickhouse

@robot-ch-test-poll robot-ch-test-poll added the pr-not-for-changelog This PR should not be mentioned in the changelog label Nov 21, 2022
@Avogar Avogar self-assigned this Nov 21, 2022
@tavplubix
Copy link
Copy Markdown
Member Author

CI: Exception: 429 Client Error: too many requests for url

@tavplubix
Copy link
Copy Markdown
Member Author

PullRequestCI / FastTest (pull_request) - WARNING:root:Killing process 63889, timeout 2400 exceeded - for some reason 40 minutes were not enough to build clickhouse, probably because of 0% ccache hits. Hope it's a result of some big change in master that invalidated the cache, let's try to restart.

@tavplubix
Copy link
Copy Markdown
Member Author

AST fuzzer (debug) - #43622
Performance Comparison Aarch64 - I don't understand why do we run them for each commit in all PRs - #43623
Stateless tests (release, s3 storage) [2/2] - #43626
Stress test (tsan) - #37664
Stress test (ubsan) - #39174 (comment)

@tavplubix tavplubix merged commit 6531ff7 into master Nov 24, 2022
@tavplubix tavplubix deleted the tavplubix-patch-5 branch November 24, 2022 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-not-for-changelog This PR should not be mentioned in the changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants