And now CH won't start up and we're seeing error where local PK metadata differ from ZK metadata.
Existing table metadata in ZooKeeper differs in primary key. Stored in ZooKeeper: mac, port_id, topology_type, first_seen_at, last_seen_at, local: mac, port_id, topology_type, first_seen_at, last_seen_at, version
1. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x95c923d in /usr/bin/clickhouse
2. ? @ 0xe5d7da6 in /usr/bin/clickhouse
3. DB::ReplicatedMergeTreeTableMetadata::checkEquals(DB::ReplicatedMergeTreeTableMetadata const&, DB::ColumnsDescription const&, DB::Context const&) const @ 0xe5d5c49 in /usr/bin/clickhouse
4. DB::StorageReplicatedMergeTree::checkTableStructure(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xe27d184 in /usr/bin/clickhouse
5. DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, DB::StorageID const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::StorageInMemoryMetadata const&, DB::Context&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::MergeTreeData::MergingParams const&, std::__1::unique_ptr<DB::MergeTreeSettings, std::__1::default_delete<DB::MergeTreeSettings> >, bool) @ 0xe2a4e79 in /usr/bin/clickhouse
6. ? @ 0xe5e382d in /usr/bin/clickhouse
7. std::__1::__function::__func<std::__1::shared_ptr<DB::IStorage> (*)(DB::StorageFactory::Arguments const&), std::__1::allocator<std::__1::shared_ptr<DB::IStorage> (*)(DB::StorageFactory::Arguments const&)>, std::__1::shared_ptr<DB::IStorage> (DB::StorageFactory::Arguments const&)>::operator()(DB::StorageFactory::Arguments const&) @ 0xe5e7a37 in /usr/bin/clickhouse
8. DB::StorageFactory::get(DB::ASTCreateQuery const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, DB::Context&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, bool) const @ 0xe1e7d2d in /usr/bin/clickhouse
9. DB::createTableFromAST(DB::ASTCreateQuery, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool) @ 0xdba5749 in /usr/bin/clickhouse
10. ? @ 0xdb98528 in /usr/bin/clickhouse
11. ? @ 0xdb98ee2 in /usr/bin/clickhouse
12. ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) @ 0x95f7987 in /usr/bin/clickhouse
13. ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()::operator()() const @ 0x95f80fa in /usr/bin/clickhouse
14. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x95f6e97 in /usr/bin/clickhouse
15. ? @ 0x95f5383 in /usr/bin/clickhouse
16. start_thread @ 0x74a4 in /lib/x86_64-linux-gnu/libpthread-2.24.so
17. __clone @ 0xe8d0f in /lib/x86_64-linux-gnu/libc-2.24.so
(version 20.5.2.7 (official build))
2020.07.02 16:40:02.369840 [ 4862 ] {} <Error> Application: DB::Exception: Existing table metadata in ZooKeeper differs in primary key. Stored in ZooKeeper: mac, port_id, topology_type, first_seen_at, last_seen_at, local: mac, port_id, topology_type, first_seen_at, last_seen_at, version: Cannot attach table `default`.`map___mac__port` from metadata file /var/lib/clickhouse/metadata/default/map___mac__port.sql from query ATTACH TABLE map___mac__port (`first_seen_at` DateTime, `last_seen_at` DateTime, `mac` UInt64, `port_id` UUID, `topology_type` Enum8('Edge' = 1, 'Infrastructure' = 2), `sign` Int8, `version` UInt32) ENGINE = ReplicatedVersionedCollapsingMergeTree('/clickhouse/tables/{shard}/map___mac__port', '{replica}', sign, version) PARTITION BY toDate(first_seen_at) ORDER BY (mac, port_id, topology_type, first_seen_at, last_seen_at) SETTINGS index_granularity = 8192```
We have upgraded to v20.5.2.7 from v20.1.11.73
And now CH won't start up and we're seeing error where local PK metadata differ from ZK metadata.
Existing table metadata in ZooKeeper differs in primary key. Stored in ZooKeeper: mac, port_id, topology_type, first_seen_at, last_seen_at, local: mac, port_id, topology_type, first_seen_at, last_seen_at, version
As you can see, CH somehow added
versionfield to the PK in table with ReplicatedVersionedCollapsingMergeTree engineHow to resolve this?
Stacktrace: